JP2011005180A - Sewing machine - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a sewing machine capable of correlating exactly the coordinates of moving means for an embroidery frame with the coordinates of imaging means.SOLUTION: Parameter calculation processing is executed while an object to be worked is held by a holding member. A working data is acquired in the parameter calculation processing (S10). The holding member is moved according to the working data (S30), and a plurality of feature points are formed in the object to be worked which is held by the holding member (S40). The object to be worked which is formed with the plurality of feature points is imaged therein (S70). The plurality of feature points is extracted from an image of the object to be worked, and feature point coordinates are calculated (S80). Coordinates acquired in S85 is correlated with the feature point coordinates (S90). A parameter for correlating the coordinates of the moving means with the coordinates of the imaging means is calculated based on a correlation result (S100).

Description

  The present invention relates to a sewing machine, and more particularly, to a sewing machine including an imaging unit.

  Conventionally, various sewing machines provided with an imaging means have been proposed. For example, in the sewing machine described in Patent Document 1, a stitch corresponding to the type of the base line is formed on the base line based on the image photographed by the imaging unit. Specifically, the base line drawn on the work cloth is imaged by the imaging means. The obtained captured image is analyzed, and the position and type of the base line on the work cloth are specified. The embroidery frame is moved to the specified base line position, and a stitch corresponding to the type of the base line is formed at the base line position.

  In general, in an apparatus in which an image captured by a camera is analyzed and the position (coordinates) of the imaging target is acquired, camera calibration is performed before the image is analyzed. Camera calibration is to calculate internal parameters and external parameters. Internal parameters are determined by the configuration of the camera itself, such as focal length and principal point coordinates. The external parameter is determined by the installation state of the camera such as the imaging direction of the camera. By using the parameters calculated by the camera calibration, it is possible to know the position where the point on the three-dimensional coordinate (the actual imaging target) should be projected on the two-dimensional coordinate (the captured image). Various studies have been conducted on camera calibration (parameter calculation) methods. For example, an automatic calibration device for a visual sensor described in Patent Document 2 has been proposed. Also in Patent Document 1, calibration of the imaging means is indispensable in order to accurately specify the position and thickness of the base line drawn on the work cloth.

JP 2007-289653 A Japanese Patent No. 3138080

  However, with the conventional sewing machine, even when the calibration of the imaging unit is performed accurately, the correspondence between the coordinate system of the moving unit that moves the embroidery frame and the coordinate system of the imaging unit cannot be accurately performed. There is. For example, there may be a case where the two-dimensional coordinate axis of the coordinate system of the moving unit is inclined with respect to the two-dimensional coordinate axis of the coordinate system of the imaging unit due to imperfect mounting of the embroidery frame. In such a case, for example, the position where the base line is drawn even if the position of the base line drawn on the work cloth is specified from the image picked up by the image pickup unit like the sewing machine described in Patent Document 1. The embroidery frame cannot be moved. Therefore, there is a problem in that a desired sewing result cannot be obtained when the user uses a sewing machine to form a stitch at a position specified from an image picked up by the image pickup means.

  The present invention has been made to solve the above-described problems, and an object thereof is to provide a sewing machine in which the correspondence between the coordinate system of the embroidery frame moving means and the coordinate system of the imaging means can be accurately taken. To do.

  The sewing machine according to claim 1 acquires a processing member that holds a workpiece, a moving unit that moves the holding member, and machining data that instructs to form a plurality of feature points on the workpiece. According to the processing data acquired by the data acquisition means and the processing data acquisition means, the moving member moves the holding member to form the plurality of feature points on the workpiece held by the holding member. A feature point forming unit, an imaging unit that is disposed at a position where the image can be captured on the sewing bed, and that images the workpiece on which the plurality of feature points are formed, and the workpiece that is imaged by the imaging unit A feature point coordinate calculator that extracts the plurality of feature points from an image of an object and calculates two-dimensional coordinates as feature point coordinates for each of the extracted feature points. A reference coordinate acquisition unit that acquires reference coordinates that are three-dimensional coordinates of the plurality of feature points measured in advance, the reference coordinates acquired by the reference coordinate acquisition unit, and the feature point coordinate calculation unit Parameter calculating means for associating the coordinated feature point coordinates with each other and a parameter for associating the coordinate system of the moving means and the coordinate system of the imaging means based on the result of the association by the associating means Means.

  The sewing machine according to claim 2 includes needle bar driving means for driving a needle bar having a sewing needle attached to a lower end portion thereof in the vertical direction according to the processing data, in addition to the configuration of the invention according to claim 1, The forming means moves the holding member by the moving means and drives the needle bar driving means according to the machining data acquired by the machining data acquiring means, and drives the needle bar driving means to form a plurality of workpieces on the workpiece by the sewing needle. A through hole is formed as the plurality of feature points.

  In the sewing machine according to claim 3, in addition to the configuration of the invention according to claim 1 or 2, the workpiece is white paper.

  A sewing machine according to a fourth aspect of the present invention is the sewing machine according to any one of the first to third aspects, wherein the plurality of feature points are regularly arranged on the workpiece within an imageable range of the imaging means. Is done.

  A sewing machine according to a fifth aspect, in addition to the configuration of the invention according to any one of the first to fourth aspects, processes an image picked up by the image pickup means using the parameter calculated by the parameter calculation means. Image processing means is provided.

  A sewing machine according to claim 6 is a tertiary on an image captured by the imaging unit using the parameter calculated by the parameter calculation unit in addition to the configuration of the invention according to any one of claims 1 to 5. Three-dimensional coordinate calculation means for calculating original coordinates is provided.

  According to a seventh aspect of the present invention, in addition to the configuration of the invention according to any one of the first to sixth aspects, the sewing machine according to the seventh aspect uses the parameter calculated by the parameter calculating means to calculate the coordinates of the point indicated by the three-dimensional coordinates. Two-dimensional coordinate calculation means for calculating the coordinates of the points projected on the two-dimensional coordinate system is provided.

  In the sewing machine according to the first aspect, a parameter for associating the coordinate system of the moving unit and the coordinate system of the imaging unit is calculated. By using the obtained parameters, the coordinate system of the moving unit and the coordinate system of the imaging unit are associated with each other, for example, based on an image captured by the imaging unit (hereinafter referred to as “captured image”). The holding member can be moved so that the seam is formed at the position. Therefore, when the user wants to form a stitch at the designated position in the holding member using the sewing machine, a desired sewing result can be obtained. Further, the feature points for calculating the parameters are formed on the workpiece by the feature point forming means. Therefore, when calculating the parameters, it is only necessary to prepare the feature point forming means and the workpiece, and the parameters can be calculated easily.

  Further, in the sewing machine according to claim 2, the through hole formed in the workpiece by the sewing needle is a feature point. That is, the feature point indicates a needle drop point at a position where the holding member is moved by the moving unit. For this reason, in addition to the effect of the invention described in claim 1, it is possible to accurately calculate a parameter for associating the coordinate system of the moving means and the coordinate system of the imaging means. The feature points for calculating the parameters are formed by sewing needles. Therefore, when calculating the parameters, the user only needs to prepare the workpiece, and the preparation for causing the sewing machine to calculate the parameters is simple.

  In the sewing machine according to the third aspect, since the workpiece is white paper, it is easy to obtain contrast with the feature points. Therefore, the sewing machine according to claim 3 can accurately extract the feature points formed on the workpiece in addition to the effects of the invention according to claim 1 or 2.

  Further, in addition to the effect of the invention according to any one of claims 1 to 3, the sewing machine according to a fourth aspect includes a coordinate system of the moving means and an imaging from feature points formed at regular positions on the workpiece. A parameter for associating the coordinate system of the means can be calculated with high accuracy.

  The sewing machine according to a fifth aspect processes a captured image using a parameter that associates the coordinate system of the moving unit with the coordinate system of the imaging unit. Therefore, in addition to the effect of the invention according to any one of claims 1 to 4, the sewing machine according to claim 5 can generate an image in which the coordinate system of the moving means and the coordinate system of the imaging means are taken. it can. Thus, for example, the sewing machine according to claim 5 easily determines the amount of movement of the holding member for forming the seam at a position specified based on a captured image obtained by imaging a workpiece such as a work cloth. be able to.

  In addition to the effect of the invention according to any one of claims 1 to 5, the sewing machine according to claim 6 eliminates the characteristic of the image pickup means and the influence of the installation state of the image pickup means. A point can be converted to a point in a three-dimensional coordinate system. For example, the sewing machine according to claim 6 can grasp the position of the point of the captured image or the distance between the two points of the captured image using the acquired three-dimensional coordinates.

    In addition to the effect of the invention according to any one of claims 1 to 6, the sewing machine according to claim 7 eliminates the characteristic of the image pickup means and the influence of the installation state of the image pickup means. Can be converted into coordinates in a two-dimensional coordinate system. The sewing machine according to claim 7 can specify, for example, a predetermined region on the captured image or a predetermined point on the captured image using the acquired two-dimensional coordinates.

1 is a perspective view of a sewing machine 1. FIG. FIG. 3 is a view of the vicinity of the sewing needle 7 as viewed from the left side of the sewing machine 1. FIG. 4 is a plan view of the embroidery frame 32. 2 is a block diagram showing an electrical configuration of the sewing machine 1. FIG. 4 is an explanatory diagram of a storage area of a RAM 63. FIG. It is explanatory drawing of the two-dimensional coordinate storage area. 3 is an explanatory diagram of a storage area of an EEPROM 64. FIG. It is explanatory drawing of the reference coordinate storage area. It is explanatory drawing of the coordinate of the feature point which process data instruct | indicates. It is a flowchart of a parameter calculation process. It is a top view of the to-be-processed object 34 in which the feature point 301 was formed. It is a flowchart of an embroidery pattern formation process. 13 is a flowchart of start point acquisition processing executed in the embroidery pattern formation processing of FIG. 12. It is a flowchart of the two-dimensional coordinate calculation process performed by the starting point acquisition process of FIG. It is a flowchart of the three-dimensional coordinate calculation process performed by the starting point acquisition process of FIG. It is a flowchart of the starting point acquisition process of a modification. It is a flowchart of the image processing process performed by the starting point acquisition process of FIG.

  Hereinafter, first and second embodiments of the present invention will be described in order with reference to the drawings. These drawings are used for explaining the technical features that can be adopted by the present invention, and the configuration of the apparatus and the flowcharts of various processes described are not intended to be limited to the drawings. This is just an illustrative example.

  First, a physical configuration and an electrical configuration of the sewing machine 1 common to the first and second embodiments will be described with reference to FIGS. 1 to 4. In FIG. 1, the direction of the arrow X is the right direction, the opposite direction is the left direction, the direction of the arrow Y is the front direction, and the opposite direction is the rear direction. As shown in FIG. 1, the sewing machine 1 includes a sewing machine bed 2, a pedestal column portion 3, and an arm portion 4. The sewing machine bed 2 extends long in the left-right direction. The pedestal 3 is erected upward from the right end of the sewing machine bed 2. The arm portion 4 extends leftward from the upper end of the pedestal column portion 3. There is a head 5 at the left end of the arm 4. A liquid crystal display (hereinafter referred to as “LCD”) 10 having a touch panel 16 on the surface is provided on the front surface of the pedestal 3. The LCD 10 displays sewing patterns, input keys for sewing conditions, and the like. The user performs sewing by pressing the position of the touch panel 16 corresponding to the position of the input key or the like displayed on the LCD 10 with a finger or a dedicated touch pen (hereinafter, this operation is referred to as “panel operation”). Patterns and sewing conditions can be selected.

  Inside the sewing machine bed 2, a feed dog longitudinal movement mechanism (not shown), a feed dog vertical movement mechanism (not shown), a feed amount adjusting pulse motor 78 (see FIG. 4), and a shuttle (not shown). Are stored. The feed dog longitudinal movement mechanism and the feed dog vertical movement mechanism drive a feed dog (not shown). The feed amount adjusting pulse motor 78 adjusts the feed amount of a work cloth (not shown) by the feed dog. The hook stores a bobbin (not shown) around which a lower thread (not shown) is wound. An embroidery device 30 is attached to the left end of the sewing bed 2. When the embroidery device 30 is not used, an auxiliary table (not shown) is attached to the left end of the sewing machine bed 2. When the embroidery device 30 is mounted on the left end of the sewing machine bed 2, the embroidery device 30 is electrically connected to the sewing machine 1. Details of the embroidery device 30 will be described later.

  Inside the pedestal 3 and the arm 4, there are a sewing machine motor 79 (see FIG. 4), a main shaft (not shown), a needle bar 6 (see FIG. 2), and a needle bar vertical movement mechanism (not shown). And a needle swing mechanism (not shown). As shown in FIG. 2, a sewing needle 7 is attached to the lower end portion of the needle bar 6. The needle bar vertical movement mechanism moves the needle bar 6 up and down using the sewing machine motor 79 as a drive source. The needle swing mechanism swings the needle bar 6 in the left-right direction using a needle swing pulse motor 77 (see FIG. 4) as a drive source. As shown in FIG. 2, a presser bar 45 extending in the vertical direction is provided behind the needle bar 6. A presser holder 46 is fixed to the lower end portion of the presser bar 45. A presser foot 47 for holding a work cloth (not shown) is attached to the presser holder 46.

  An opening / closing cover 21 is attached to the arm portion 4. The opening / closing cover 21 is provided in the longitudinal direction of the arm portion 4, and is pivotally supported by the upper rear end portion of the arm portion 4 so as to be openable and closable around a left-right axis. A thread accommodating portion 23 is provided in the vicinity of the upper center of the arm portion 4 with the opening / closing cover 21 opened. The thread accommodating portion 23 is a recess for accommodating the yarn spool 20 that supplies the sewing machine 1 with a yarn. On the inner wall surface of the thread accommodating portion 23 on the side of the pedestal portion 3, a thread stand bar 22 that protrudes toward the head 5 is provided. The thread spool 20 is mounted by inserting an insertion hole (not shown) provided in the thread spool 20 into the spool pin 22. Although not shown, the thread of the thread spool 20 is supplied as an upper thread to the sewing needle 7 (see FIG. 2) attached to the needle bar 6 via a plurality of thread hooks provided on the head 5. . Examples of the thread hook portion include a thread tension device, a thread take-up spring, and a balance. The thread tensioner and the thread take-up spring adjust the thread tension. The balance is reciprocated up and down to pull up the upper thread.

  A pulley (not shown) is provided on the right side surface of the sewing machine 1. The pulley manually rotates the main shaft and moves the needle bar 6 up and down. A front cover 59 is provided in front of the head 5 and the arm portion 4. The front cover 59 is provided with a sewing start / stop switch 41, a speed adjustment knob 43, and other operation switches. The sewing start / stop switch 41 is a switch for instructing the start and stop of sewing. When the sewing start / stop switch 41 is pressed while the sewing machine 1 is stopped, the operation of the sewing machine 1 is started. When the sewing start / stop switch 41 is pressed while the sewing machine 1 is operating, the operation of the sewing machine 1 is stopped. . The speed adjustment knob 43 adjusts the rotational speed of the main shaft (not shown). An image sensor 50 (see FIG. 2) is installed in the front cover 59 at a position obliquely upward to the right as viewed from the sewing needle 7. Details of the image sensor 50 will be described later.

  Next, the image sensor 50 will be described with reference to FIG. The image sensor 50 is a well-known CMOS image sensor, and is attached to a position where the sewing bed 2 and the needle plate 80 provided on the sewing bed 2 can be imaged. In the present embodiment, the image sensor 50 is attached to a support frame 51 attached to a frame (not shown) of the sewing machine 1.

  Next, the embroidery device 30 will be described with reference to FIGS. 1 and 3. The embroidery device 30 includes a carriage (not shown), a carriage cover 33, a forward / backward movement mechanism (not shown), a left / right movement mechanism (not shown), and an embroidery frame 32. The carriage holds the embroidery frame 32 in a detachable manner. A concave groove (not shown) in which the embroidery frame 32 is mounted is provided on the right side of the carriage. The concave groove extends along the longitudinal direction of the carriage. The carriage cover 33 generally has a rectangular parallelepiped shape that is long in the front-rear direction and accommodates the carriage. A forward / backward movement mechanism (not shown) is provided inside the carriage cover 33. The back-and-forth moving mechanism moves the carriage on which the embroidery frame 32 is mounted in the front-rear direction using a Y-axis motor 82 (see FIG. 4) as a drive source. The left / right moving mechanism is provided in the main body of the embroidery device 30. The left / right movement mechanism moves the carriage on which the embroidery frame 32 is mounted, the front / rear movement mechanism, and the carriage cover 33 in the left / right direction using an X-axis motor 81 (see FIG. 4) as a drive source. Driving commands for the Y-axis motor 82 and the X-axis motor 81 are output by a CPU 61 (see FIG. 4) described later.

  The embroidery frame 32 will be described with reference to FIG. The embroidery frame 32 includes a guide 321, an outer frame 322, an inner frame 323, and an adjustment screw 324. The guide 321 has a substantially rectangular shape in plan view. A projecting portion (not shown) extending along the longitudinal direction is provided substantially at the center of the lower surface of the guide 321. The embroidery frame 32 is mounted on the carriage by engaging the protrusion with a concave groove (not shown) provided on the carriage (not shown) of the embroidery device 30. When the embroidery frame 32 is mounted on the carriage, the protruding portion is urged in a direction to be pressed by the groove portion by an elastic urging spring (not shown) provided on the carriage. For this reason, the embroidery frame 32 and the carriage are securely engaged with each other, and the embroidery frame 32 moves integrally with the carriage. Inside the outer frame 322, an inner frame 323 whose outer peripheral shape is formed to be substantially the same as the inner peripheral shape of the outer frame 322 is fitted. A workpiece 34 such as a work cloth is sandwiched between the outer frame 322 and the inner frame 323. The work piece 34 is held by the embroidery frame 32 by tightening the adjustment screw 324 provided on the outer frame 322. A rectangular sewing area 325 is set inside the inner frame 323. The embroidery pattern is formed in the sewing area 325. The embroidery frame 32 is not limited to the size shown in FIG. 1, but various sizes of embroidery frames are prepared (not shown).

  Next, the main electrical configuration of the sewing machine 1 will be described with reference to FIG. As shown in FIG. 4, the sewing machine 1 includes a CPU 61, a ROM 62, a RAM 63, an EEPROM 64, an external access RAM 68, an input interface 65, and an output interface 66, which are connected to each other by a bus 67. Yes.

  The CPU 61 manages the main control of the sewing machine 1 and executes various calculations and processes according to programs stored in the ROM 62 and the like. The ROM 62 includes a plurality of storage areas including an embroidery data storage area and a program storage area. In the embroidery data storage area, embroidery data used when embroidery is performed by the sewing machine 1 is stored. In the program storage area, a plurality of programs including a parameter calculation program executed by the CPU 61 and an embroidery pattern formation program are stored. The RAM 63 is a storage element that can be arbitrarily read and written. The storage area of the RAM 63 will be described later. The EEPROM 64 is a readable / writable storage element, and stores various parameters used when various programs stored in the program storage area are executed. The storage area of the EEPROM 64 will be described later. A card slot 17 is connected to the external access RAM 68. The card slot 17 can be connected to the memory card 18. If the card slot 17 and the memory card 18 are connected, information in the memory card 18 can be read and written.

  To the input interface 65, a sewing start / stop switch 41, a speed adjustment knob 43, the touch panel 16, and an image sensor 50 are connected. Drive circuits 70 to 75 are connected to the output interface 66. The drive circuit 70 drives a needle swing pulse motor 77. The needle swing pulse motor 77 is a drive source of a needle swing mechanism (not shown). The drive circuit 71 drives the feed amount adjusting pulse motor 78. The drive circuit 72 drives the sewing machine motor 79. The sewing machine motor 79 is a drive source for a main shaft (not shown). The drive circuit 73 drives the X axis motor 81. The drive circuit 74 drives the Y-axis motor 82. The drive circuit 75 drives the LCD 10. Other components (not shown) are connected to the input interface 65 and the output interface 66 as appropriate.

Next, the storage area of the RAM 63 will be described with reference to FIG. As shown in FIG. 5, the RAM 63 includes a captured image storage area 631, a two-dimensional coordinate storage area 632, and other storage areas 633. A captured image is stored in the captured image storage area 631. As shown in FIG. 6, in the two-dimensional coordinate storage area 632, a plurality of feature points extracted from a captured image in a parameter calculation process to be described later are imaged by the image sensor 50 and a number for distinguishing the feature points. The two-dimensional coordinates (u _c , v _c ) of the image coordinate system 200 (hereinafter referred to as “image coordinate system 200”) are stored in association with each other. The image coordinate system 200 is a virtual coordinate system set for the captured image, and FIG. 2 schematically shows the image coordinate system 200. The feature points extracted from the captured image will be described later. In the other storage area 633, calculation results and the like calculated by the CPU 61 are stored.

  A storage area of the EEPROM 64 will be described with reference to FIG. The EEPROM 64 includes a reference coordinate storage area 641, an internal parameter storage area 642, an external parameter storage area 643, and a movement condition storage area 644. In the reference coordinate storage area 641, known three-dimensional coordinates of feature points in the world coordinate system 100 are stored as reference coordinates. The world coordinate system 100 is a coordinate system that indicates the entire space, and is a coordinate system that is not affected by the center of gravity of the object. In general, the world coordinate system 100 is used to indicate the position of an object in space or to compare coordinates for different objects. In the first and second embodiments, the world coordinate system 100 is set as follows. As shown in FIG. 1, the upper surface of the workpiece 34 sandwiched between the embroidery frames 32 is defined as an XY plane. In the world coordinate system 100, the X axis is set in the left-right direction of the sewing machine 1, the Y axis is set in the front-rear direction, and the Z axis is set in the up-down direction. The origin (X, Y, Z) = (0, 0, 0) of the world coordinate system 100 is a needle drop point. The needle drop point is a point at which the sewing needle 7 pierces the workpiece 34 when the needle bar 6 is moved downward from a state where the sewing needle 7 is on the workpiece 34.

  The reference coordinate storage area 641 provided in the EEPROM 64 will be described in detail with reference to FIG. As shown in FIG. 8, the reference coordinate storage area 641 stores a number, processed data, and reference coordinates in association with each other. The number is an ID for distinguishing feature points. The number of the reference coordinate storage area 641 corresponds to the number of the two-dimensional coordinate storage area 632. The processed data is data indicating the position of the feature point corresponding to the number. The drive amounts of the X-axis motor 81 and the Y-axis motor 82 when moving the embroidery frame 32 are determined according to the processing data. The processing data is indicated by, for example, the world coordinate system 100 (see FIG. 1). The reference coordinates are, for example, parameters described later in a state where a coordinate system 300 (see FIG. 3) (hereinafter referred to as “embroidery coordinate system 300”) of the embroidery device 30 described later and the image coordinate system 200 are calibrated. Calculation processing is executed and calculated. The embroidery coordinate system 300 will be described later. The reference coordinates are three-dimensional coordinates (Xw, Yw, Zw) of the world coordinate system 100. The reference coordinates are used in a process of comparing with feature point coordinates calculated based on a captured image, which is executed in a parameter calculation process described later.

  Next, machining data stored in the reference coordinate storage area 641 will be described with reference to FIGS. In FIG. 9, a position where a feature point indicated by the processing data is formed is indicated by a point 101. An origin 110 that is the intersection of the X axis 103 and the Y axis 102 of the world coordinate system 100 is located at the center of the imageable range 104 of the image sensor 50. The origin 110 may be a position other than the center position of the imageable range 104. In the point 101, the density of feature points (number of feature points per unit area) is substantially uniform within the imageable range 104 in a state where the embroidery frame 32 is moved to the origin position of the embroidery coordinate system 300 described later. So that they are regularly arranged. More specifically, in the present embodiment, the point 101 is arranged at the apex position of a 17 × 17 grid.

  The storage area of the EEPROM 64 will be described with reference to FIG. The internal parameter storage area 642 stores internal parameters calculated from the captured image and the three-dimensional coordinates of the feature points. The internal parameters are parameters for correcting the focal length, the deviation of the principal point coordinates, and the distortion of the captured image that are generated based on the characteristics of the image sensor 50, respectively. The internal parameter storage area 642 includes, as internal parameters, an X-axis focal length, a Y-axis focal length, an X-axis principal point coordinate, a Y-axis principal point coordinate, a first distortion coefficient, and a second distortion coefficient. Remembered. The X-axis focal length indicates a shift in the focal length of the image sensor 50 in the X-axis direction. The Y-axis focal length indicates a shift in focal length in the Y-axis direction. The X-axis principal point coordinates indicate the deviation of the principal point of the image sensor 50 in the X-axis direction. The Y-axis principal point coordinate indicates a deviation of the principal point in the Y-axis direction. Each of the first distortion coefficient and the second distortion coefficient indicates distortion due to the inclination of the lens of the image sensor 50. The internal parameters are used, for example, when the sewing machine 1 converts a captured image into a normalized image. A normalized image is an image captured by a normalized camera. A normalized camera is a camera whose unit length is the distance from the optical center to the screen surface.

  The external parameter storage area 643 stores external parameters calculated from the captured image and the three-dimensional coordinates of the feature points. The external parameter is a parameter indicating the installation state (position and orientation) of the image sensor 50 with respect to the world coordinate system 100. That is, the external parameter is a parameter indicating a deviation between the image coordinate system 200 and the world coordinate system 100. The external parameter storage area 643 stores an X-axis rotation vector, a Y-axis rotation vector, a Z-axis rotation vector, an X-axis translation vector, a Y-axis translation vector, and a Z-axis translation vector as external parameters. The The X axis rotation vector indicates rotation around the X axis with respect to the world coordinate system 100 of the image coordinate system 200, the Y axis rotation vector indicates rotation around the Y axis, and the Z axis rotation vector indicates rotation around the Z axis. Show. The X-axis rotation vector, the Y-axis rotation vector, and the Z-axis rotation vector are a conversion matrix for converting from the world coordinate system 100 to the image coordinate system 200, and a conversion matrix for converting from the image coordinate system 200 to the world coordinate system 100. Used in making decisions. The X-axis translation vector indicates a shift in the X-axis direction of the image coordinate system 200 with respect to the world coordinate system 100, the Y-axis translation vector indicates a shift in the Y-axis direction, and the Z-axis translation vector indicates a shift in the Z-axis direction. The X-axis translation vector, the Y-axis translation vector, and the Z-axis translation vector are the translation vector that the sewing machine 1 converts from the world coordinate system 100 to the image coordinate system 200, or the image coordinate system 200 to the world coordinate system 100. Used in determining the translation vector.

  In the movement condition storage area 644, as the movement condition of the embroidery frame 32, the movement amount of the embroidery frame 32 indicated by data such as processing data and embroidery data (hereinafter referred to as “instruction data”), and the X-axis motor 81 are displayed. The correspondence relationship with the drive amount of the Y-axis motor 82 (see FIG. 4) is stored. Further, the movement condition storage area 644 stores the coordinates of the origin of the embroidery coordinate system 300 for each type of the embroidery frame 32. The movement condition is used for processing for moving the embroidery frame 32 in accordance with the instruction data. The embroidery coordinate system 300 is a coordinate system determined based on the movement amount specified by the instruction data and the actual movement amount of the embroidery frame 32 when the embroidery frame 32 is moved according to the instruction data. That is, the embroidery coordinate system 300 is a coordinate system of the X-axis motor 81 and the Y-axis motor 82 that move the embroidery frame 32. In the embroidery coordinate system 300, the left-right direction of the sewing machine 1 is the X-axis direction, and the front-rear direction of the sewing machine 1 is the Y-axis direction. In the present embodiment, in the embroidery coordinate system 300, the origin (X, Y, Z) = (0, 0, 0) is the position where the center of the sewing area of the embroidery frame 32 coincides with the needle drop point. Note that the embroidery device 30 of the present embodiment does not move the embroidery frame 32 in the Z direction (the vertical direction of the sewing machine 1), so that the thickness of the workpiece 34 such as a work cloth is within a negligible range. The upper surface of the object 34 is set to Z = 0.

  Next, the parameter calculation processing of the first embodiment will be described with reference to the flowchart of FIG. In the parameter calculation process, a parameter for associating the image coordinate system 200 and the embroidery coordinate system 300 is calculated. In the first embodiment, in the parameter calculation process, internal parameters and external parameters of the image sensor 50 are calculated using feature points (through holes) having coordinates of the embroidery coordinate system 300 formed on the workpiece 34. The The parameter calculation process shown in FIG. 10 is executed by the CPU 61 according to the parameter calculation program stored in the ROM 62. The parameter calculation process is executed when an instruction to perform “parameter calculation” is input by the user. The execution instruction is input by, for example, a panel operation. For example, when the sewing machine 1 includes a dedicated button for executing parameter calculation, an execution instruction is input by pressing the dedicated button by the user.

  As shown in FIG. 10, in the parameter calculation process of the first embodiment, first, it is determined whether or not the preparation for executing the parameter calculation process is completed (S5). In the first embodiment, when executing a parameter calculation process, if a thread (not shown) is set on the sewing needle 7, the user removes the thread. Further, the user sandwiches white paper as the workpiece 34 in the embroidery frame 32 and attaches the embroidery frame 32 to the embroidery device 30. Whether or not the preparation is completed is determined based on, for example, an instruction input by a panel operation. If the preparation is not completed (S5: No), the process waits until the preparation is completed. If the preparation is completed (S5: Yes), the number and the machining data stored in the reference coordinate storage area 641 are stored. The acquired machining data is stored in the RAM 63 (S10).

  Next, the machining data of number i is read out of the machining data acquired in S10 (S20). The initial value of i is 1, and is incremented when S20 is repeatedly executed. Next, a drive instruction is output to the drive circuits 73 and 74, and the embroidery frame 32 is moved to the position indicated by the processing data read in S20 (S30). In S <b> 30, the sewing needle 7 is on the workpiece 34. Next, a drive instruction is output to the drive circuit 72, and the sewing machine motor 79 is driven. As a result, the needle bar 6 is moved up and down, and a through hole is formed as a feature point in the workpiece 34 by the sewing needle 7 (S40). Next, it is determined whether all the machining data acquired in S10 has been read in S20 (S50). If there is processed data that has not been read (S50: No), the incremented i is stored in the RAM 63, and the process returns to S20. When all the processing data has been read (S50: Yes), a drive instruction is output to the drive circuits 73 and 74, and the embroidery frame 32 is moved to the origin position of the embroidery coordinate system 300 (S60). In S 60, the sewing needle 7 is on the workpiece 34. In S10, when processing data instructing that feature points are formed at the positions shown in FIG. 9 is acquired, 324 feature points 301 shown in FIG. When the embroidery frame 32 is at the origin position of the embroidery coordinate system 300, the feature point 301 is inside the imageable range 104 of the image sensor 50. The feature points 301 are regularly arranged so that the density of feature points (the number of feature points per unit area) is substantially uniform. More specifically, in this embodiment, the feature points 301 are arranged at the positions of the vertices of a 17 × 17 grid.

  Next, the workpiece 34 on which the feature point 301 is formed is imaged by the image sensor 50, and the captured image is stored in the captured image storage area 631 (S70). In the example illustrated in FIG. 11, 324 feature points 301 inside the imageable range 104 are shown in the captured image. Next, the feature point 301 is extracted from the captured image captured in S70, and the two-dimensional coordinates of the image coordinate system 200 of the feature point 301 are stored in the two-dimensional coordinate storage area 632 as the feature point coordinates (S80). In S80, feature points are extracted from the captured image based on a known technique. For example, the captured image is subjected to Hough transform, and a Hough transform image is created. Next, non-maximum suppression processing is performed on the Hough transform image, and a bright spot is extracted locally (within the mask) of the Hough transform image. Next, threshold processing for extracting only brighter points than the predetermined threshold among the extracted bright points is performed, and feature points are extracted. The center of the through hole is extracted as a feature point, and feature point coordinates are calculated.

  Next, the reference coordinates stored in the reference coordinate storage area 641 are acquired, and the reference coordinates are stored in the RAM 63 (S85). Next, the feature point coordinates calculated in S80 are associated with the feature point numbers, and the correspondence is stored in the two-dimensional coordinate storage area 632 (S90). The correspondence between the feature point coordinates and the feature point numbers is determined, for example, as follows. In a state where the embroidery coordinate system 300 and the image coordinate system 200 (see FIG. 2) are calibrated, a parameter calculation process described later is executed, and the two-dimensional coordinates of the calculated feature points 301 are calculated in advance. It is assumed that the calculated two-dimensional coordinates are stored in a predetermined storage area in association with the feature point number. The feature having the feature point coordinate closest to the known two-dimensional coordinate of number n acquired in S85 is compared with the known two-dimensional coordinate stored in the predetermined storage area and the feature point coordinate calculated in S80. Let the point 301 be a feature point associated with the number n. The feature point coordinates are associated with the reference coordinates by the feature point number.

  Next, parameters are calculated, and the calculated parameters are stored in the internal parameter storage area 642 and the external parameter storage area 643 of the EEPROM 64 (S100). The parameters are calculated using a known camera calibration parameter calculation method. In the parameter calculation method, an object (workpiece) including a feature point whose three-dimensional coordinates are known is imaged, and the two-dimensional coordinates of the feature points in the captured image are calculated. A projection matrix is obtained based on the known three-dimensional coordinates of the feature points and the calculated two-dimensional coordinates, and parameters are calculated from the projection matrix. As for the parameter calculation method, a known technique (for example, Japanese Patent No. 3138080) may be appropriately used. Following S100, the parameter calculation process ends.

  As described above, the parameter calculation process of the first embodiment is executed. The embroidery frame 32 that holds the workpiece 34 corresponds to the holding member of the present invention. Based on the drive instruction output to the drive circuits 73 and 74, the X-axis motor 81 and the Y-axis motor 82 that move the embroidery frame 32 function as the moving means of the present invention. In the parameter calculation process of FIG. 10, the machining data is acquired (S10). The CPU 61 functions as the machining data acquisition means of the present invention. The EEPROM 64 that stores the machining data corresponds to machining data storage means. The sewing machine motor 79 that drives the needle bar 6 with the sewing needle 7 attached to the lower end in the vertical direction according to the processing data functions as the needle bar driving means of the present invention. In accordance with the machining data acquired in S10, a driving instruction is output to the driving circuits 72 to 74, the embroidery frame 32 is moved (S30), and the needle bar 6 is driven in the vertical direction (S40). It functions as a feature point forming means. The image sensor 50 corresponds to the imaging unit of the present invention. In S80 of the parameter calculation process, a plurality of feature points are extracted from the captured image and two-dimensional coordinates are calculated as feature point coordinates (S80). The CPU 61 functions as a feature point coordinate calculation unit of the present invention. In S85 of the parameter calculation process, the CPU 61 that acquires the reference coordinates functions as a reference coordinate acquisition unit of the present invention. The EEPROM 64 that stores the reference coordinates corresponds to reference coordinate storage means. The CPU 61 that associates the reference coordinates acquired in S85 with the feature point coordinates calculated in S80 functions as an association unit of the present invention. Based on the result of S90, a parameter for associating the embroidery coordinate system 300 and the image coordinate system 200 is calculated (S100). The CPU 61 functions as a parameter calculation means of the present invention. The embroidery coordinate system 300 corresponds to the coordinate system of the moving means of the present invention. The image coordinate system 200 corresponds to the coordinate system of the imaging means of the present invention.

  In the parameter calculation process of the first embodiment, internal parameters and external parameters can be calculated as parameters for associating the embroidery coordinate system 300 with the image coordinate system 200. Based on the obtained parameters, the image coordinate system 200 is set based on the embroidery coordinate system 300. That is, the embroidery coordinate system 300 becomes the world coordinate system 100. In the first embodiment, white paper is used as the workpiece 34, and a through hole opened in the workpiece 34 with the sewing needle 7 is used as the feature point 301. Therefore, when calculating the parameters, the user only needs to prepare white paper as the workpiece 34, and preparation for causing the sewing machine 1 to calculate the parameters is simple. Since the sewing machine 1 uses white paper as the workpiece 34, the white paper has a relatively large difference in brightness between the area where the characteristic points are not formed and the characteristic points. Therefore, in S80 of the parameter calculation process, the feature points can be extracted with higher accuracy than in the case where there is little difference in brightness between the area where the feature points are not formed and the feature points. Further, since white paper is a general-purpose product, the user can easily prepare a workpiece. As shown in FIG. 11, the feature points 301 are regularly formed in the imageable range 104 in a state where the embroidery frame 32 is moved to the origin position. Therefore, compared with the case where the feature points 301 are formed so as to be biased toward a part of the area within the imageable range 104, the parameters for associating the embroidery coordinate system 300 with the image coordinate system 200 can be calculated with high accuracy. Since the sewing machine 1 uses a small and inexpensive CMOS sensor as the image pickup means, a space for installing the image pickup means in the sewing machine can be reduced, and the cost of the photographing apparatus can be kept low.

  Next, a usage example of a parameter calculated by the parameter calculation process of the first embodiment (hereinafter referred to as “first parameter”) will be described. The first parameter is used for, for example, two-dimensional coordinate calculation processing, three-dimensional coordinate calculation processing, and image processing processing. The two-dimensional coordinate calculation process is a process of converting the three-dimensional coordinates of the points in the world coordinate system 100 into the two-dimensional coordinates of the points projected on the captured image plane of the image coordinate system 200. The three-dimensional coordinate calculation process is a process of converting the two-dimensional coordinates of the points on the captured image of the image coordinate system 200 into the three-dimensional coordinates of the points in the world coordinate system 100. The image processing process is a process for processing an image captured by the image sensor 50. Taking an embroidery pattern forming process as an example, a two-dimensional coordinate calculation process, a three-dimensional coordinate calculation process, and an image processing process will be described. In the embroidery pattern forming process, an embroidery pattern is formed starting from a position specified by a user on a workpiece such as a work cloth. The embroidery pattern forming process shown in FIG. 12 is executed by the CPU 61 according to the embroidery pattern forming program stored in the ROM 62 when an instruction of “embroidery pattern forming” is input by the user.

In the following description, the two-dimensional coordinate of the point p on the captured image in the image coordinate system 200 is m _c (u _c , v _c ), and the three-dimensional coordinate of the point p in the image coordinate system 200 is M _c (X _c , Y _c , Z _c ). A point obtained by converting the point p into the world coordinate system 100 is defined as a point P, and a three-dimensional coordinate of the point P in the world coordinate system 100 is defined as M _w (X _w , Y _w , Z _w ). For the internal parameters stored in the internal parameter storage area 642, the X-axis focal length is fx, the Y-axis focal length is fy, the X-axis principal point coordinate is cx, the Y-axis principal point coordinate is cy, and the first distortion coefficient is k. _1, the second distortion coefficient is _{k 2.} For the external parameters stored in the external parameter storage area 643, the X-axis rotation vector is r ₁ , the Y-axis rotation vector is r ₂ , the Z-axis rotation vector is r ₃ , the X-axis translation vector is t ₁ , and the Y-axis translation vector Is t ₂ and the Z-axis translation vector is t ₃ . “Rw” is a 3 × 3 rotation matrix determined based on external parameters (X-axis rotation vector r ₁ , Y-axis rotation vector r ₂ , Z-axis rotation vector r ₃ ), and “t _w ” is an external parameter ( A 3 × 1 translation vector determined based on the X-axis translation vector t ₁ , the Y-axis translation vector t ₂ , and the Z-axis translation vector t ₃ ).

  As shown in FIG. 12, in the embroidery pattern formation process, first, a storage area such as the ROM 62 is referred to, embroidery data is acquired and stored in the RAM 63 (S110). In S110, the embroidery data of the embroidery pattern selected by the user through the panel operation is acquired. Next, a start point acquisition process is executed (S120). In the start point acquisition process, the position at which sewing of the embroidery pattern starts according to the embroidery pattern acquired in S110 is acquired as the start point, and the three-dimensional coordinates of the start point world coordinate system 100 are stored in the RAM 63. Details of the start point acquisition process will be described later. Next, it is determined whether or not a starting point has been acquired in S120 (S130). If the start point has not been acquired (S130: No), an error is notified (S160), and the embroidery pattern formation process ends. When the start point is acquired (S130: Yes), a drive instruction is output to the drive circuits 73 and 74, and the embroidery frame 32 is moved to a position where the start point acquired in S120 becomes the needle drop point. (S140). Next, an embroidery pattern is formed based on the embroidery data acquired in S110 (S150). In S150, the formation of the embroidery pattern is started at the position moved in S140. The process for forming the embroidery pattern is the same as that for sewing a normal embroidery pattern. Next, the embroidery pattern forming process ends.

  Next, the start point acquisition process executed in the embroidery pattern formation process will be described with reference to FIG. In the starting point acquisition process of the first embodiment, a predetermined mark (for example, an X mark) drawn on the workpiece 34 sandwiched between the embroidery frame 32 is detected, and the position where the predetermined mark is drawn is detected. This is the starting point. It is assumed that the user marks the starting point of the workpiece 34 with a chaco pen or the like before executing the embroidery pattern forming process.

  As shown in FIG. 13, in the start point acquisition process, first, the embroidery frame 32 is moved to the nth position (S210). The process of S210 is a process for imaging the sewing area of the embroidery frame 32 in m times. For n, integers from 1 to m are set in order. Next, the workpiece 34 is imaged and the captured image is stored in the RAM 63 (S220). Next, a two-dimensional coordinate calculation process is executed (S230). In the two-dimensional coordinate calculation process, the three-dimensional coordinates of the world coordinate system 100 of the boundary line of the sewing area 325 are converted into the two-dimensional coordinates of the image coordinate system 200. The process of S230 is executed to specify the sewing area in the captured image acquired in S220. In the specified sewing area, a detection process for detecting a mark indicating the start point is executed. Details of the two-dimensional coordinate calculation processing will be described with reference to FIG.

In the two-dimensional coordinate calculation process shown in FIG. 14, the three-dimensional coordinates of the world coordinate system 100 of the points forming the boundary of the sewing area 325 of the embroidery frame 32 stored in the EEPROM 64 are sequentially read as the points P and the following processing is performed. Is executed. First, the three-dimensional coordinates _M w of P point in the world coordinate system _{100 (X w, Y w,} Z w) based on the three-dimensional coordinates _M c when the point P is converted into point p of the image coordinate system 200 ( X _c , Y _c , Z _c ) is calculated, and the three-dimensional coordinates M _c (X _c , Y _c , Z _c ) are stored in the RAM 63 (S310). The three-dimensional coordinates M _c (X _c , Y _c , Z _c ) are calculated by “M _c = R _w × M _w + t _w ”. Next, the two-dimensional coordinates (x ′, y ′) of the point p on the normalized image of the image coordinate system 200 are calculated based on the three-dimensional coordinates of the point p of the image coordinate system 200 calculated in S310. The dimension coordinates (x ′, y ′) are stored in the RAM 63 (S320). The two-dimensional coordinates of the point p on the normalized image of the image coordinate system 200 are calculated by “x ′ = X _c / Z _c ” and “y ′ = Y _c / Z _c ”.

Next, based on the two-dimensional coordinates (x ′, y ′) of the point p on the normalized image of the image coordinate system 200, the distortion of the lens of the image sensor 50 is added to the normalized camera at the point p. Dimensional coordinates (x ", y") are calculated (S330). For the calculation of the two-dimensional coordinates (x ″, y ″), the first distortion coefficient k ₁ and the second distortion coefficient k _{2 which} are internal parameters are used. Specifically, the two-dimensional coordinates (x ″, y ″) are expressed by the expression “x ″ = x ′ × (1 + k ₁ × r ² + k ₂ × r ⁴ )” and “y” = y ′ × (1 + k _1). × r ² + k ₂ × r ⁴ ) ”. Note that “r ² = x ′ ² + y ′ ² ”. Next, two-dimensional coordinates (u _c , v _c ) on the captured image of the point p in the image coordinate system 200 are calculated, and the two-dimensional coordinates (u _c , v _c ) are stored in the RAM 63 (S340). For the calculation of the two-dimensional coordinates (u _c , v _c ), the two-dimensional coordinates (x ″, y ″), the internal parameters X-axis focal length fx, Y-axis focal length fy, and X-axis principal point coordinates cx and Y-axis principal point coordinates cy are used. Specifically, it is calculated by the equation “u _c = fx × x” + cx ”and“ v _c = fy × y ”+ cy”. The above processing is executed for points on the boundary line of the sewing area 325 of the embroidery frame 32, and the two-dimensional coordinates of the image coordinate system 200 of the boundary line of the sewing area 325 are obtained. Next, the two-dimensional coordinate calculation process ends.

The start point acquisition process will be described with reference to FIG. Next to S230, it is determined whether or not there is a mark indicating the start point in the sewing area 325 obtained in S230 among the images captured in S220 (S240). If there is no start point (S240: No), it is determined whether the entire range of the sewing area 325 has been imaged (S260). If the entire range has not been imaged (S260: No), n is incremented and the process returns to S210. In S240, when there is a start point (S240: Yes), a three-dimensional coordinate calculation process is performed (S250). In the three-dimensional coordinate calculation process, the position of the mark (starting point) on the captured image is set as a point p, and the three-dimensional coordinates M _w (X _w , Y _w ) when the point p is converted to a point P in the world coordinate system 100. , Z _w ) is calculated. Details of the three-dimensional coordinate calculation processing will be described with reference to FIG.

As shown in FIG. 15, in the three-dimensional coordinate calculation process, first, a point on the normalized image of the image coordinate system 200 is calculated from the two-dimensional coordinates (u _c , v _c ) of the point p on the captured image of the image coordinate system 200. The two-dimensional coordinates (x ″, y ″) of p are calculated, and the two-dimensional coordinates (x ″, y ″) are stored in the RAM 63 (S410). The two-dimensional coordinates (x ″, y ″) are calculated by “x ″ = (u−cx) / fx” and “y ″ = (v−cy) / fy”. The two-dimensional coordinates (x ″, y ″) calculated in S210 are the internal parameters X-axis focal length fx, Y-axis focal length fy, X-axis principal point coordinate cx, and Y-axis principal point coordinate cy. Is a coordinate that takes into account. Next, based on the two-dimensional coordinates (x ″, y ″) of the point p, the two-dimensional coordinates (x ′, y ′) of the point p on the normalized image from which the lens distortion has been removed are calculated, and the two-dimensional The coordinates (x ′, y ′) are stored in the RAM 63 (S420). For the calculation of the two-dimensional coordinates (x ′, y ′), the first distortion coefficient k ₁ and the second distortion coefficient k _{2 which} are internal parameters are used. Specifically, the expression “x ′ = x” −x ”× (1 + k ₁ × r ² + k ₂ × r ⁴ )” and “y ′ = y” −y ”× (1 + k ₁ × r ² + k ₂ ×) r ⁴ ) ”.

Next, based on the two-dimensional coordinates (x ′, y ′) calculated in S420, the three-dimensional coordinates M _c (X _c , Y _c , Z _c ) of the point p in the image coordinate system 200 are calculated, and the three-dimensional The coordinates M _c (X _c , Y _c , Z _c ) are stored in the RAM 63 (S430). The three-dimensional coordinates M _c (X _c , Y _c , Z _c ) are calculated using the following relational expression. X _c and Y _c have a relationship of “X _c = x ′ × Z _c ” and “Y _c = y ′ × Z _c ”. Further, between the three-dimensional coordinates M _c (X _c , Y _c , Z _c ) of the image coordinate system 200 and the three-dimensional coordinates M _w (X _w , Y _w , Z _w ) of the world coordinate system 100, The relationship “M _w = R _w ^T (M _c −t _w )” is established. It should be noted _that the _R ^{w T} is a transposed matrix of _{R w.} Since the XY plane of the world coordinate system 100 is set on the upper surface of the workpiece 34 sandwiched between the embroidery frames 32, the sewing machine 1 sets Z _w = 0, “X _c = x ′ × Z _c ”, Solve “Y _c = y ′ × Z _c ” and “M _w = R _w ^T (M _c −t _w )” simultaneously. As a result, _Zc is calculated, and _Xc and _Yc are calculated. Next, based on the three-dimensional coordinates M _c (X _c , Y _c , Z _c ) of the point p in the image coordinate system 200, the three-dimensional coordinates M _w when the point p is converted to the point P in the world coordinate system 100. (X _w , Y _w , Z _w ) is calculated, and M _w (X _w , Y _w , Z _w ) is stored in the RAM 63 (S440). The three-dimensional coordinates M _w (X _w , Y _w , Z _w ) of the world coordinate system 100 are calculated by “M _w = R _w ^T (M _c −t _w )”. Next, the three-dimensional coordinate calculation process ends.

  The start point acquisition process will be described with reference to FIG. In S260, when the entire range is imaged (S260: Yes), or after S250, the start point acquisition process ends.

  As described above, the sewing machine 1 that executes the embroidery pattern forming process uses the first parameter to form an embroidery pattern starting from the position of the mark drawn on the workpiece 34 by the user. 13, the point P is converted to the point p of the image coordinate system 200 based on the point P on the boundary line of the sewing area indicated by the three-dimensional coordinates of the world coordinate system 100 in S230 of the start point acquisition process of FIG. The CPU 61 that calculates the two-dimensional coordinates of the case functions as the two-dimensional coordinate calculation means of the present invention. In S250 of the start point acquisition process, the CPU 61 that calculates the three-dimensional coordinates when the point p is converted into the point P of the world coordinate system 100 based on the two-dimensional coordinates of the point p on the captured image of the image coordinate system 200. It functions as the three-dimensional coordinate calculation means of the present invention.

  In the embroidery pattern formation process, the sewing area 325 on the captured image is specified based on the two-dimensional coordinates of the point p of the image coordinate system 200 calculated using the first parameter in S240 of the start point acquisition process. Since the sewing machine 1 calibrates the image sensor 50 using the first parameter, the points of the image coordinate system 200 are taken into consideration in consideration of the characteristic of the image sensor 50 and the influence of the installation state of the image sensor 50. The two-dimensional coordinates of p can be calculated. Since the image coordinate system 200 is set based on the embroidery coordinate system 300, the sewing machine 1 can accurately specify the sewing area 325 on the captured image. Further, in S240 of FIG. 13, a mark detection process for instructing the start point is executed for the pixels in the specified sewing area. Therefore, the sewing machine 1 can detect a mark more efficiently than when the detection process is performed on all pixels of the captured image.

  In the embroidery pattern formation process, the two-dimensional coordinates of the points on the captured image in the image coordinate system 200 can be converted into the three-dimensional coordinates of the points in the world coordinate system 100 in S250 of the start point acquisition process in FIG. . The first parameter has a function as a parameter used for calibration of the image sensor 50. Therefore, the sewing machine 1 eliminates the influence of the characteristic inherent to the image sensor 50 and the installation state of the image sensor 50, and uses the point on the captured image as a point in the actual three-dimensional coordinate system (world coordinate system 100). Can be calculated. In the first embodiment, the first parameter is calculated so that the embroidery coordinate system 300 becomes the world coordinate system 100. Therefore, the sewing machine 1 can issue an accurate movement instruction to the embroidery frame 32 using the three-dimensional coordinates calculated in the three-dimensional coordinate calculation process. That is, the sewing machine 1 can accurately move the embroidery frame 32 so that the position of the mark drawn by the user on the workpiece 34 such as a work cloth becomes a needle drop point.

  In the first embodiment, the position of the mark drawn by the user on the workpiece 34 is set as the start point. However, the start point may be acquired by another method. For example, the sewing machine 1 may display a captured image obtained by imaging the workpiece on the LCD 10 and acquire a point on the captured image designated by the user through a panel operation as a start point. The start point acquisition process of a modification is demonstrated with reference to FIG.

  As shown in FIG. 16, in the start point acquisition process of the modification, first, the workpiece 34 sandwiched between the embroidery frame 32 is imaged (S205). Next, it is determined whether there is an instruction to move the viewpoint (S215). The instruction to move the viewpoint instructs that the captured image is converted into a virtual image viewed from an arbitrary viewpoint and displayed on the LCD 10. As shown in FIG. 2, the image sensor 50 is attached at a position where the needle plate 80 is imaged from above. In other words, the viewpoint is on the needle plate 80 (in the negative direction of the Z axis). The instruction to move the viewpoint instructs to change the viewpoint to, for example, the upper right side of the needle plate 80 and to process the captured image into the viewpoint changed image in which the workpiece 34 is captured from the upper right side. The instruction to move the viewpoint is input by a panel operation together with the changed viewpoint, for example. If there is an instruction to move the viewpoint (S215: Yes), image processing is executed (S225). In the image processing process, a process is performed in which the captured image captured by the image sensor 50 is processed and converted into a virtual image from the viewpoint instructed in S215. Details of the image processing will be described with reference to FIG.

In the description of the image processing, the three-dimensional coordinates of the point p ′ in the coordinate system (not shown) of the moved viewpoint (hereinafter referred to as “moving viewpoint coordinate system”) are represented by M _cm (X _cm , Y _cm , Z _cm ). Let the two-dimensional coordinates of the point p ′ on the viewpoint change image in the moving viewpoint coordinate system be (u _cm , v _cm ). The three-dimensional coordinates M _w (X _w , Y _w , Z _w ) of the point P in the world coordinate system are converted into the three-dimensional coordinates M _cm (X _cm , Y _cm , Z _cm ) of the point p ′ in the moving viewpoint coordinate system. In this case, R _w2 (3 × 3 rotation matrix) and t _w2 (3 × 1 translation vector) are used. These determinants are determined based on which point in the world coordinate system 100 the destination viewpoint corresponds to. A determinant for converting the three-dimensional coordinates M _cm (X _cm , Y _cm , Z _cm ) of the moving viewpoint coordinate system into the three-dimensional coordinates M _c (X _c , Y _c , Z _c ) of the image coordinate system 200 is R ₂₁ (3 × 3 rotation matrix) and t ₂₁ (3 × 1 translation vector).

As shown in FIG. 17, in the image processing, first, determinants R ₂₁ and t ₂₁ are calculated, and the determinants R ₂₁ and t ₂₁ are stored in the RAM 63 (S510). And R _w, and _{R w2,} and _{R 21,} and _{t w,} and _{t w2,} to and _{t 21,} the following relationship is established. “M _c = R _w × M _w + t _w (from the world coordinate system 100 to the image coordinate system 200)” and “M _cm = R _w2 × M _w + t _w2 (from the world coordinate system 100 to the moving viewpoint coordinate system)” And “M _c = R ₂₁ × M _cm + t ₂₁ (from the moving viewpoint coordinate system to the image coordinate system 200)”. From these expressions, “R ₂₁ = R _w × R _w2 ^T ” and “t ₂₁ = −R _w × R _w2 ^T × t _w2 + t _w ” are obtained. And R _w, and _{R w2,} since the _{t w} and _{t w2} is already fixed value _calculated, R _{21, t} 21 is uniquely determined.

Next, based on the two-dimensional coordinates (u _cm , v _cm ) of the point p ′ on the viewpoint change image, the two-dimensional coordinates (u _c , v _c ) when the point p ′ is converted to the point p on the captured image. ) Is calculated, and the correspondence between the two-dimensional coordinates (u _cm , v _cm ) and the two-dimensional coordinates (u _c , v _c ) is stored in the RAM 63 (S520). The process of S520 is performed for all the pixels of the viewpoint change image, and the correspondence relationship between the pixel (u _c , v _c ) of the captured image and the pixel (u _cm , v _cm ) of the viewpoint change image is stored in the RAM 63. The Specifically, first, based on the two-dimensional coordinates (u _cm , v _cm ) of the point p ′ on the viewpoint change image, the two-dimensional coordinates (x ₂ ”of the point p ′ on the normalized image of the moving viewpoint coordinate system are used. , Y ₂ ″) is calculated. For the calculation of the two-dimensional coordinates (x ₂ ″, y ₂ ″), the X-axis focal length fx, the Y-axis focal length fy, the X-axis principal point coordinate cx, and the Y-axis principal point coordinate cy are internal parameters. Is used. Specifically, the two-dimensional coordinates (x ₂ ″, y ₂ ″) are “x ₂ ” = (u _cm −cx) / fx ”and“ y ₂ ”= (v _cm −cy) / fy”. Is calculated by Next, two-dimensional coordinates (x ₂ ′, y ₂ ′) are calculated by adding lens distortion to the two-dimensional coordinates (x ₂ ″, y ₂ ″) of the point p ′ on the normalized image in the moving viewpoint coordinate system. Then, the two-dimensional coordinates (x ₂ ′, y ₂ ′) are stored in the RAM 63. For calculation of the two-dimensional coordinates (x ₂ ′, y ₂ ′), the first distortion coefficient k ₁ and the second distortion coefficient k _{2 that} are internal parameters are used. Specifically, the two-dimensional coordinates (x ₂ ′, y ₂ ′) are “x ₂ ′ = x ₂ ” −x ₂ ”× (1 + k ₁ × r ² + k ₂ × r ⁴ )” and “y ₂ '= Y ₂ "-y ₂ " × (1 + k ₁ × r ² + k ₂ × r ⁴ ) ”. Note that “r ² = x ₂ ′ ² + y ₂ ′ ² ”.

Next, the three-dimensional coordinates M _cm (X _cm , Y _cm ) of the point p ′ of the moving viewpoint coordinate system from the two-dimensional coordinates (x ₂ ′, y ₂ ′) of the point p ′ on the normalized image of the moving viewpoint coordinate system. , Z _cm ) and the three-dimensional coordinates M _cm (X _cm , Y _cm , Z _cm ) are stored in the RAM 63. The three-dimensional coordinates M _cm (X _cm , Y _cm , Z _cm ) are calculated using the following relational expression. Regarding X _cm and Y _cm, they are “X _cm = x ₂ ′ × Z _cm ” and “Y _cm = y ₂ ′ × Z _cm ”. Furthermore, since the upper surface of the sewing machine bed 2 is the XY plane of the world coordinate system 100, Z _w = 0 in “M _cm = R _w2 × M _w + t _w2 ”. Then, by calculating the solution of the simultaneous equations, the three-dimensional coordinates M _cm (X _cm , Y _cm , Z _cm ) of the point p ′ in the moving viewpoint coordinate system are calculated.

Next, the three-dimensional coordinates when the point p ′ is converted to the point p of the image coordinate system 200 based on the three-dimensional coordinates M _cm (X _cm , Y _cm , Z _cm ) of the point p ′ of the moving viewpoint coordinate system. M _c (X _c , Y _c , Z _c ) is calculated, and the three-dimensional coordinates M _c (X _c , Y _c , Z _c ) are stored in the RAM 63. The three-dimensional coordinates M _c (X _c , Y _c , Z _c ) are calculated by substituting M _cm (X _cm , Y _cm , Z _cm ) into the formula “M _c = R ₂₁ × M _cm + t ₂₁ ”. Next, based on the three-dimensional coordinates M _c (X _c , Y _c , Z _c ) of the point p in the image coordinate system 200, the two-dimensional coordinates (x ₁ ′, x y ₁ ′) is calculated, and the three-dimensional coordinates M _cm (X _cm , Y _cm , Z _cm ) are stored in the RAM 63. The two-dimensional coordinates (x ₁ ′, y ₁ ′) are calculated by “x ₁ ′ = x ₁ / z ₁ ” and “y ₁ ′ = y ₁ / z ₁ ”. Then, the two-dimensional coordinates which lens distortion is taken into account _{(x 1 ", y 1"} ) is calculated, two-dimensional coordinates _{(x 1 ", y 1"} ) is stored in the RAM 63. The two-dimensional coordinates (x ₁ ″, y ₁ ″) are “x ₁ ” = x ₁ ′ × (1 + k ₁ × r ² + k ₂ × r ⁴ ) ”and“ y ₁ ”= y ₁ ′ × (1 + k _1). × r ² + k ₂ × r ⁴ ) ”. Note that “r ² = x ₁ ′ ² + y ₁ ′ ² ”. Next, based on the two-dimensional coordinates (x ₁ ″, y ₁ ″) of the point p on the normalized image of the image coordinate system 200, the two-dimensional coordinates (u _c , v _c ) is calculated, and the two-dimensional coordinates (u _c , v _c ) are stored in the RAM 63. The two-dimensional coordinates (u _c , v _c ) are calculated by “u _c = fx × x ₁ ” + cx ”and“ v _c = fy × y ₁ ”+ cy”.

Next, a viewpoint change image is created from the captured image based on the correspondence relationship between the pixel (u _c , v _c ) of the captured image and the pixel (u _cm , v _cm ) of the viewpoint changed image stored in the RAM 63. The generated viewpoint change image is stored in the RAM 63 (S530). Next, the image processing process ends.

Next, the start point acquisition process of the modification will be described with reference to FIG. In S215, when there is no instruction to move the viewpoint (S215: No), or after S225, the captured image captured in S205 or the viewpoint change image created in S225 is displayed on the LCD 10 (S235). Next, it is determined whether or not there is an instruction to specify a start point (S245). When a point on the captured image or viewpoint change image displayed on the LCD 10 is selected by a panel operation, it is determined that there is an instruction to specify the start point. If there is no instruction, the process waits until there is an instruction (S245: No). If there is an instruction (S245: Yes), a three-dimensional coordinate calculation process is executed (S250). In the three-dimensional coordinate calculation process, the three-dimensional coordinates of the world coordinate system 100 at the start point specified in S245 are calculated. When the viewpoint change image is displayed in S235, based on the correspondence between the pixel (u _c , v _c ) of the captured image obtained in S225 and the pixel (u _cm , v _cm ) of the viewpoint change image, After specifying the two-dimensional coordinates of the image coordinate system 200, a three-dimensional coordinate calculation process is executed. Since the three-dimensional coordinate calculation process is the same as that of the first embodiment, the description thereof is omitted. Next, the start point acquisition process ends.

  As described above, the start point acquisition process of the modification is executed. In S225 of the start point acquisition process of the modification, the CPU 61 that processes the captured image to create the viewpoint change image using the first parameter functions as the image processing unit of the present invention.

  The sewing machine 1 can process the image picked up by the image sensor 50 using the first parameter to create a viewpoint change image. In the image processing, a viewpoint-changed image is created that excludes the characteristics inherent to the image sensor 50 and the influence of the installation state of the image sensor 50 based on the captured image. When the sewing machine 1 obtains the three-dimensional coordinates of the point on the viewpoint change image, after specifying the two-dimensional coordinates of the image coordinate system 200 based on the correspondence between the pixel of the captured image and the pixel of the viewpoint change image. The three-dimensional coordinate calculation process is executed. In the three-dimensional coordinate calculation process, the three-dimensional coordinates of the world coordinate system 100 (embroidery coordinate system 300) are calculated using the first parameter. Therefore, the sewing machine 1 can move the embroidery frame 32 to the position designated by the user based on the three-dimensional coordinates of the world coordinate system 100 calculated by the start point acquisition process of the modification.

  By the way, in the parameter calculation process of the first embodiment, the sewing machine 1 calculates the first parameter with the embroidery coordinate system 300 as the world coordinate system 100. However, for example, when the movement condition stored in the movement condition storage area 644 is not set appropriately, the embroidery frame 32 does not move by the amount indicated by the instruction data. More specifically, even if it is instructed to move the embroidery frame 32 by (X, Y) = (5, 0) by the processing data, the embroidery frame 32 is actually (X, Y) = (4 .5,0) May be moved. In such a case, when the image coordinate system 200 is set based on the embroidery coordinate system 300, for example, there arises a problem that the actual distance between two points on the captured image cannot be accurately calculated. In such a case, for example, as in the second embodiment, the embroidery coordinate system 300 and the world coordinate system 100 may be set separately.

  In the second embodiment, the process of associating the image coordinate system 200 and the world coordinate system 100 is performed separately according to a known calibration process. On the other hand, the second parameter for associating the image coordinate system 200 and the embroidery coordinate system 300 is calculated by the same process as that of the first embodiment. The second parameter is calculated in the same manner as the first parameter, but is used only for processing for converting the coordinates of the image coordinate system 200 or the world coordinate system 100 into the coordinates of the embroidery coordinate system 300. That is, the second parameter is not used for the process of converting the coordinates of the image coordinate system 200 into the coordinates of the world coordinate system 100 and the process of converting the coordinates of the world coordinate system 100 into the coordinates of the image coordinate system 200. For example, when it is instructed to move the embroidery frame 32 by a predetermined amount according to the processing data indicated by the coordinates of the world coordinate system 100, the sewing machine 1 temporarily embroiderys the processing data using the second parameter. It is converted into data of the coordinate system 300. The sewing machine 1 moves the embroidery frame 32 based on the processing data converted into the embroidery coordinate system 300. In this way, the sewing machine 1 can move the embroidery frame 32 to a position intended by the user even when the movement conditions stored in the movement condition storage area 644 are not appropriately set. In the second embodiment, the CPU 61 that calculates the second parameter functions as a parameter calculation unit of the present invention.

  The embroidery data creation device of the present invention is not limited to the above-described embodiment, and various modifications may be made without departing from the gist of the present invention. For example, the following modifications (A) to (I) may be added as appropriate.

  (A) The image pickup means provided in the sewing machine 1 may be a CCD camera or another image pickup device. The image sensor 50 as the imaging means may be any as long as it can capture the image on the sewing bed 2, and the mounting position thereof can be changed as appropriate.

  (B) The workpiece on which the feature points are formed may be paper other than white, for example, in addition to white paper. For example, the workpiece on which the feature points are formed may be a resin or a work cloth.

  (C) The feature point 301 formed on the workpiece is not limited to the through hole formed by the sewing needle 7 exemplified in the above embodiment. For example, the feature point formed on the workpiece may be a point drawn with a marker. In this case, the marker is fixed to the lower end of the needle bar 6. The attachment position of the marker in the vertical direction is a position where the pen tip of the marker contacts the workpiece when the needle bar 6 moves to the lowest end. In this way, the sewing machine 1 can draw feature points corresponding to the needle drop points by moving the needle bar 6 up and down. From the viewpoint of accurately extracting feature points, the color of the marker is preferably a color having a contrast with the workpiece. In another example, the feature point formed on the workpiece may be a stitch. When the workpiece is a work cloth of the same type as the work cloth that forms the embroidery pattern, and the feature point is a stitch formed by the thread used to form the embroidery pattern, the sewing machine of the modified example has The parameter can be obtained in consideration of the thickness and the amount of shrinkage. In addition, when forming a through-hole in the work piece as a feature point, when the work piece is imaged, a sheet-like member having a relatively large difference in brightness from the work piece is provided below the work piece. You may arrange. If it does in this way, a sheet-like member will be reflected in a portion in which a penetration hole is formed in a picked-up image. For this reason, the difference in brightness between the workpiece and the through hole in the captured image can be increased.

  (D) In the above embodiment, the feature points 301 are regularly formed within the imageable range 104 of the image sensor 50. However, the arrangement of the feature points can be changed as appropriate. As another example in which the feature points are regularly arranged within the imageable range, for example, the feature points may be arranged radially within the imageable range 104. For example, the feature points 301 may be irregularly formed. However, from the viewpoint of calculating the parameters with high accuracy, it is preferable that five or more feature points 301 are provided within the imageable range when the embroidery frame 32 is moved in S60 of FIG.

  (E) The sewing machine 1 may calibrate the embroidery coordinate system 300 using the second parameter. Specifically, the sewing machine 1 may correct the movement condition stored in the movement condition storage area 644 so that the embroidery coordinate system 300 matches the world coordinate system 100 using the second parameter. In this case, the image coordinate system 200 is set based on the embroidery coordinate system 300 as in the first embodiment. Therefore, the sewing machine 1 of the modified example (E) can move the embroidery frame 32 to a position intended by the user.

  (F) The procedure for calculating the first or second parameter can be changed as appropriate. For example, although the sewing machine 1 has extracted feature points using Hough transform in S80 of FIG. 10, the feature points may be extracted using other methods. The sewing machine 1 associates the feature point coordinates and the reference coordinates using the known two-dimensional coordinates and the two-dimensional coordinates calculated in S80 in S90 of FIG. 10, but other methods may be used. Good. For example, when forming a plurality of feature points having different shapes (for example, a circle and a square), the sewing machine 1 may execute a process of associating the feature point coordinates with the reference coordinates according to the shape of the feature points. Good. The known two-dimensional coordinates and reference coordinates need only be determined before the parameter calculation process is executed, and the known two-dimensional coordinates and reference coordinates calculation method can be changed as appropriate.

  (G) The embroidery coordinate system and the image coordinate system only need to be associated with each other by the parameter calculated by the parameter calculation process, and the setting method of each coordinate system can be changed as appropriate. For example, the embroidery coordinate system may be set so that the upper side in the vertical direction of the sewing machine 1 is positive on the Z axis.

  (H) Although the embroidery device 30 includes the X-axis motor 81 and the Y-axis motor 82 as moving means, any one of them may be used. The embroidery device 30 may include a Z-axis motor that moves the embroidery frame 32 in the Z-axis direction in addition to at least one of the X-axis motor 81 and the Y-axis motor 82. The embroidery device 30 includes the embroidery frame 32 as a holding member, but may include other holding members.

  (I) Although the two-dimensional coordinate calculation process, the three-dimensional coordinate calculation process, and the image processing process are exemplified as the process used by the first or second parameter, the parameter may be used for other processes. . In addition, each of the two-dimensional coordinate calculation process, the three-dimensional coordinate calculation process, and the image processing process may be executed by simplifying the process or using another known technique as necessary. .

1 sewing machine 30 embroidery device 32 embroidery frame 34 work piece 50 image sensor 61 CPU
79 Sewing machine motor 81 X-axis motor 82 Y-axis motor 104 Imageable range 200 Image coordinate system 300 Embroidery coordinate system 301 Features

Claims (7)

A holding member for holding a workpiece;
Moving means for moving the holding member;
Processing data acquisition means for acquiring processing data for instructing to form a plurality of feature points on the workpiece;
Feature point forming means for moving the holding member by the moving means according to the processing data acquired by the processing data acquiring means and forming the plurality of feature points on the workpiece held by the holding member; ,
An image pickup unit that is arranged at a position where an image can be picked up on the sewing bed and that picks up an image of the workpiece on which the plurality of feature points are formed,
Feature point coordinate calculation for extracting the plurality of feature points from the image of the workpiece imaged by the imaging means and calculating two-dimensional coordinates as feature point coordinates for each of the extracted feature points Means,
Reference coordinate acquisition means for acquiring reference coordinates which are three-dimensional coordinates of the plurality of feature points measured in advance;
Associating means for associating the reference coordinates acquired by the reference coordinate acquiring means with the feature point coordinates calculated by the feature point coordinate calculating means;
A sewing machine comprising: parameter calculation means for calculating a parameter for associating the coordinate system of the moving means and the coordinate system of the imaging means based on a result of the association by the association means. Needle bar driving means for driving a needle bar with a sewing needle attached to the lower end in the vertical direction according to the processing data;
The feature point forming means includes:
In accordance with the processing data acquired by the processing data acquisition means, the holding member is moved by the moving means and the needle bar driving means is driven,
The sewing machine according to claim 1, wherein a plurality of through holes are formed as the plurality of characteristic points in the workpiece by the sewing needle.   The sewing machine according to claim 1 or 2, wherein the workpiece is white paper.   The sewing machine according to any one of claims 1 to 3, wherein the plurality of feature points are regularly arranged on the workpiece within an imageable range of the imaging unit.   The sewing machine according to any one of claims 1 to 4, further comprising an image processing unit that processes an image captured by the imaging unit using the parameter calculated by the parameter calculation unit.   6. The apparatus according to claim 1, further comprising: a three-dimensional coordinate calculation unit that calculates a three-dimensional coordinate on an image captured by the imaging unit using the parameter calculated by the parameter calculation unit. The sewing machine according to Crab.   Using two-dimensional coordinate calculation means for calculating the coordinates of a point obtained by projecting the coordinates of the point indicated by the three-dimensional coordinates on the two-dimensional coordinate system using the parameter calculated by the parameter calculation means; The sewing machine according to any one of claims 1 to 6.

Images