CN115066313A

CN115066313A - Workpiece mounting method, workpiece mounting support system, and workpiece mounting support program for machining device

Info

Publication number: CN115066313A
Application number: CN202080095937.5A
Authority: CN
Inventors: 堀部和也; 丸田和正; 山本博雅; 朴炫九; 伊藤雅敏; 水上龙一
Original assignee: Yamazaki Mazak Corp
Current assignee: Yamazaki Mazak Corp
Priority date: 2020-02-14
Filing date: 2020-02-14
Publication date: 2022-09-16
Anticipated expiration: 2040-02-14
Also published as: US20220371143A1; EP4104968A4; JP6736798B1; EP4104968B1; WO2021161530A1; EP4104968A1; JPWO2021161530A1; CN115066313B

Abstract

The present invention provides a workpiece mounting method, a workpiece mounting support system, and a workpiece mounting support program for a machining apparatus, the workpiece mounting method for the machining apparatus including: a method of adjusting the attitude of a workpiece includes acquiring a reference image representing a reference workpiece whose attitude has been adjusted, setting a plurality of workpiece reference lines at a boundary of a first image region occupied by the reference workpiece in the reference image, acquiring a measurement image representing the workpiece, and generating a measurement composite image by a processor, the plurality of workpiece reference lines being respectively represented at a plurality of positions identical to the positions of the plurality of workpiece reference lines in the measurement composite image, and adjusting the attitude of the workpiece so that the boundary of a second image region occupied by the workpiece is substantially parallel to or substantially coincident with the plurality of workpiece reference lines in the measurement composite image.

Description

Workpiece mounting method, workpiece mounting support system, and workpiece mounting support program for machining device

Technical Field

The present invention relates to a workpiece mounting method, a workpiece mounting support system, and a workpiece mounting support program for a machining apparatus.

Background

In the case of machining a workpiece with a numerically controlled machine tool, the workpiece needs to be placed at a position and in a posture assumed in a machining program. Therefore, a mounting operation of placing the workpiece on the table and adjusting the position and posture thereof is performed before the machining. Patent document 1 discloses one of the following methods: a workpiece set on a table is photographed by a vision device, a workpiece name is deduced from an image thereof, and a rough position of the workpiece on the table is calculated from the image data. Further, patent document 1 discloses a method of: a contact probe for measurement is attached to a main spindle, an accurate position of a workpiece and an accurate position of a feature point of the workpiece are automatically searched and found from the rough position, a rough posture of the workpiece is found from a straight line connecting the feature points of the workpiece, and correction is performed so that a reference coordinate system of a machining program coincides with a coordinate system of the workpiece on a table.

Documents of the prior art

Patent document 1: japanese patent publication No. 2011180

When the method described in patent document 1 is applied to a workpiece such as a casting having an uneven surface, there is a deviation in posture between the posture of the workpiece estimated from straight lines connecting characteristic points measured from the uneven surface and the actual posture of the workpiece, and due to the existence of the deviation in posture, a mounting error greater than or equal to a machining allowance is generated in a part of the workpiece. Further, when the workpiece is large, the estimated attitude of the workpiece deviates greatly from the actual attitude of the workpiece, so that there is a high possibility that an attachment error greater than or equal to the machining allowance is generated. Therefore, the method described in patent document 1 is insufficient, and when a plurality of workpieces requiring the same machining are machined, it is necessary to determine and correct the posture of the workpiece by the contact probe for measurement for each workpiece, check whether or not the above-described posture deviation exists for each workpiece, determine the posture of the workpiece again by the contact probe for measurement as necessary, and adjust the correction amount. Therefore, a more efficient method of adjusting the attitude of the workpiece is required.

Disclosure of Invention

The present invention has been made in an effort to provide a workpiece mounting method, a workpiece mounting support system, and a workpiece mounting support program for a machining apparatus capable of efficiently adjusting the postures of a plurality of workpieces.

A first aspect of the present invention is a method for mounting a workpiece for a machining apparatus, including: placing a reference workpiece on a placing table, adjusting the posture of the reference workpiece on the placing table so that the posture of the reference workpiece becomes a workpiece reference posture, setting a camera to a target arrangement in which a viewpoint position and a visual line direction of a camera which photographs the placing table become a reference viewpoint position and a reference visual line direction, respectively, acquiring a reference image which represents the reference workpiece after the posture is adjusted and which is photographed by the camera which is set to the target arrangement, setting a plurality of workpiece reference lines at a boundary of a first image region which the reference workpiece occupies in the reference image, storing a plurality of positions in the reference image in which the plurality of workpiece reference lines are located, respectively, as a plurality of workpiece reference line positions in a memory, placing a workpiece having a shape and a size which are substantially the same as the shape and the size of the reference workpiece on one of the placing table from which the reference workpiece is taken off and an additional placing table which is arranged in place of the placing table and is different from the placing table, a measurement image representing a workpiece captured by a camera set to a target arrangement is acquired, a measurement composite image in which a plurality of workpiece reference lines are superimposed on the measurement image is generated by a processor, the plurality of workpiece reference lines are respectively represented at a plurality of positions identical to the plurality of workpiece reference lines in the measurement composite image, and the attitude of the workpiece is adjusted on one of the stages so that the boundaries of a second image region occupied by the workpiece in the measurement composite image are substantially parallel to or substantially coincident with the plurality of workpiece reference lines.

According to a second aspect of the present invention, in the workpiece mounting method according to the first aspect, the placement table and the additional placement table are movable to a processing position where the workpiece is processed by the processing apparatus and a preparation position away from the processing position. The camera photographs any one of a reference workpiece and a workpiece placed on a stage moved to the preparation position. The attitude of the workpiece is adjusted on the placing table or the additional placing table moved to the preparation position.

According to a third aspect of the present invention, in the workpiece mounting method according to the second aspect, when the placing table on which the reference workpiece whose posture is adjusted so as to be the reference posture of the workpiece is placed is moved to the machining position, all of the positions of the plurality of reference feature points, which are the plurality of feature points of the shape of the reference workpiece, are within a predetermined allowable range.

According to a fourth aspect of the present invention, the method of mounting a workpiece according to the third aspect further comprises: the one of the tables on which the workpiece whose posture is adjusted is placed is moved to a machining position, positions of a plurality of machining reference points, which are a plurality of feature points of the workpiece having substantially the same shape corresponding to the plurality of reference feature points, are measured by a position measuring sensor of the machining device, and it is determined by the machining device whether or not the positions of the plurality of machining reference points are within an allowable range.

According to a fifth aspect of the present invention, the method of mounting a workpiece of the fourth aspect further comprises: when the position of at least one of the plurality of machining reference points is determined not to be within the allowable range, the determination result that machining is not permitted is output, and the one of the tables on which the workpiece is placed that has been moved to the machining position is moved to the preparation position.

According to a sixth aspect of the present invention, in the workpiece mounting method of any one of the first to fifth aspects, the attitude of the reference workpiece and the attitude of the workpiece are defined by a rotation angle around a coordinate axis of a coordinate system set for causing the machining device to execute the machining program or a rotation center axis of the placing table.

According to a seventh aspect of the present invention, in the workpiece mounting method according to any one of the first to sixth aspects, the workpiece reference line is an edge of a boundary of the first image region obtained by image processing performed by the processor.

According to an eighth aspect of the present invention, in the workpiece mounting method of any one of the first to seventh aspects, the workpiece reference line is a straight line or a circle.

According to a ninth aspect of the present invention, the workpiece mounting method according to any one of the first to eighth aspects further includes setting a plurality of camera setting reference lines at a boundary of a third image area occupied by a stationary object in a background of the reference image. In the workpiece mounting method, a plurality of positions in the reference image at which the plurality of camera setting reference lines are respectively located are stored in the memory as a plurality of camera setting reference line positions, and the processor generates a measurement composite image in which the plurality of camera setting reference lines are superimposed on the measurement image, wherein the plurality of camera setting reference lines are respectively indicated in the measurement composite image at a plurality of positions identical to the plurality of camera setting reference line positions. The cameras are set to the target arrangement by adjusting the viewpoint positions and the line-of-sight directions of the cameras so that the boundaries of the fourth image area occupied by the stationary objects in the measurement composite image coincide with the plurality of camera setting reference lines.

According to a tenth aspect of the present invention, in the workpiece mounting method according to the ninth aspect, the camera setting reference line is an edge of a boundary of the third image area obtained by image processing performed by the processor. According to an eleventh aspect of the present invention, in the workpiece mounting method according to the ninth or tenth aspect, the camera setting reference line is a straight line.

According to a twelfth aspect of the present invention, the method of mounting a workpiece according to any one of the first to eleventh aspects, further comprises: setting a first additional camera to a first additional target configuration in which a viewpoint position and a visual line direction of the first additional camera on the imaging table are set to a first additional reference position different from a reference viewpoint position and a first additional reference visual line direction non-parallel to the reference visual line direction, respectively, setting a second additional camera to a second additional target configuration in which a viewpoint position and a visual line direction of the second additional camera on the imaging table are set to a second additional reference position different from the reference viewpoint position and the first additional reference position, respectively, and a second additional reference visual line direction non-parallel to the reference visual line direction and the first additional reference visual line direction, respectively, acquiring a first additional reference image, which is captured by the first additional camera set to the first additional target configuration and represents a reference workpiece whose posture is adjusted, and acquiring a second additional reference image, which is captured by the second additional camera set to the second additional target configuration and represents a reference workpiece whose posture is adjusted A second additional reference image in which a plurality of first additional workpiece reference lines are set at a boundary of a fifth image region occupied by the reference workpiece in the first additional reference image, a plurality of second additional workpiece reference lines are set at a boundary of a sixth image region occupied by the reference workpiece in the second additional reference image, a plurality of positions in the first additional reference image where the plurality of first additional workpiece reference lines are respectively located are stored in a memory as a plurality of first additional workpiece reference line positions, a plurality of positions in the second additional reference image where the plurality of second additional workpiece reference lines are respectively located are stored in the memory as a plurality of second additional workpiece reference line positions, a first additional measurement image representing the workpiece captured by the first additional camera set to the first additional target configuration is acquired, and a second additional measurement image representing the workpiece captured by the second additional camera set to the second additional target configuration is acquired, generating, by a processor, a first additional measurement composite image in which a plurality of first additional workpiece reference lines are superimposed on a first additional measurement image, the plurality of first additional workpiece reference lines being respectively indicated in the first additional measurement composite image at a plurality of positions identical to the positions of the plurality of first additional workpiece reference lines, generating, by the processor, a second additional measurement composite image in which a plurality of second additional workpiece reference lines are superimposed on a second additional measurement image, the plurality of second additional workpiece reference lines being respectively indicated in the second additional measurement composite image at a plurality of positions identical to the positions of the plurality of second additional workpiece reference lines, adjusting the attitude of the workpiece on one of the stages so that the boundary of a seventh image region occupied by the workpiece in the first additional measurement composite image is substantially parallel to or substantially coincident with the plurality of first additional workpiece reference lines, adjusting the attitude of the workpiece on the one of the stages, the boundary of the eighth image region occupied by the workpiece in the second additional measurement composite image is made substantially parallel to or substantially coincident with the plurality of second additional workpiece reference lines. According to a thirteenth aspect of the present invention, in the workpiece mounting method according to the twelfth aspect, two of the reference line-of-sight direction, the first additional reference line-of-sight direction, and the second additional reference line-of-sight direction are substantially perpendicular to each other, and an angle formed by a plane parallel to both of the two line-of-sight directions and the remaining line-of-sight directions other than the two line-of-sight directions is larger than 45 degrees.

According to a fourteenth aspect of the present invention, the workpiece mounting method of the fifth aspect further comprises: determining, by a machine learning model learned using training data having a measurement image and a plurality of workpiece reference lines as input, whether or not all positions of a plurality of first additional machining reference points, which are a plurality of feature points of a first additional workpiece having substantially the same shape and size as those of a reference workpiece, respectively corresponding to a plurality of reference feature points, exist within an allowable range when one of tables on which the first additional workpiece is placed is moved to a machining position, based on a third additional measurement image representing the first additional workpiece having substantially the same shape and size as those of the reference workpiece captured by a camera set to a target arrangement, and outputting at least one of the determination result, the result of determining whether each of the plurality of machining reference points is within the allowable range, and the amount of deviation of each of the plurality of machining reference points from the center value of the allowable range. According to a fifteenth aspect of the present invention, in the workpiece mounting method according to the fourteenth aspect, the machine learning model is learned using training data that also takes as input at least one of an edge detected from the measurement image, a deviation amount from an image center of a plurality of workpiece reference line positions, a focal length of a lens of the camera, and a distortion parameter of the camera.

According to a sixteenth aspect of the present invention, in the workpiece mounting method according to any one of the first to fifteenth aspects, a second additional workpiece having a shape and a size substantially identical to those of the reference workpiece is placed on the other of the placement table from which the reference workpiece is removed and the additional placement table disposed in place of the placement table, a fourth additional measurement image representing the second additional workpiece captured by the camera set to the target disposition is acquired, a fourth additional measurement composite image in which a plurality of workpiece reference lines are superimposed on the fourth additional measurement image is generated by the processor, the plurality of workpiece reference lines are respectively represented in the fourth additional measurement composite image at a plurality of positions identical to the positions of the plurality of workpiece reference lines, the posture of the second additional workpiece is adjusted on the other table, and the boundary of a ninth image region occupied by the second workpiece and the plurality of workpiece reference lines are made substantially identical to those of the plurality of workpiece reference lines in the fourth additional measurement composite image Either substantially parallel or substantially coincident.

According to a seventeenth aspect of the present invention, there is provided a work mounting support system including a placement table, a camera, an input device, a memory, a processor, and a display. The placing table is configured to place the reference workpiece or the workpiece alternatively in order to adjust the posture of the workpiece and the posture of the reference workpiece, the workpiece having a shape and a size substantially the same as those of the reference workpiece. The camera is configured to photograph the reference workpiece and the workpiece on the placing table. A plurality of workpiece reference lines are set at the boundary of a first image region occupied by a reference workpiece in a reference image of the reference workpiece captured by a camera and indicating the reference workpiece adjusted so that the posture of the reference workpiece is the workpiece reference posture, by an input device. The memory is configured to store a plurality of positions in the reference image at which the plurality of workpiece reference lines are respectively located as a plurality of workpiece reference line positions. The processor is configured to generate a measurement composite image in which a plurality of workpiece reference lines, which are respectively indicated at a plurality of positions identical to the positions of the plurality of workpiece reference lines in the measurement composite image, are superimposed on a measurement image representing the workpiece captured by the camera when the posture of the workpiece is adjusted on the placing stage. The display is configured to display the measurement composite image when the measurement image is captured.

According to an eighteenth aspect of the present invention, the workpiece mounting support system according to the seventeenth aspect is configured such that, when capturing the reference image, the cameras are set to a target arrangement in which the viewpoint position and the visual line direction of the cameras are the reference viewpoint position and the reference visual line direction, respectively, and when capturing the measurement image, the cameras are set to the target arrangement.

According to a nineteenth aspect of the present invention, the workpiece mounting support system of the eighteenth aspect further includes a machining device and an additional placement table. The machining device is configured to machine a workpiece. The additional placing table is configured to place the reference workpiece or the workpiece alternatively. The placing table and the additional placing table can be moved to a processing position where the workpiece is processed by the processing device and a preparation position away from the processing position. The camera shoots the platform of one of the placing platform moved to the preparation position and the additional placing platform. The attitude of the reference workpiece is adjusted on the placing table moved to the preparation position. The attitude of the workpiece is adjusted on the placing table or the additional placing table moved to the preparation position.

According to a twentieth aspect of the present invention, in the workpiece mounting support system according to the nineteenth aspect, the machining device includes: a position measurement sensor configured to: measuring the positions of a plurality of reference feature points, which are a plurality of feature points of the shape of the reference workpiece, when the placing table on which the reference workpiece is placed, the posture of which is adjusted so as to be the reference posture of the workpiece, is moved to the machining position; and an electronic circuit configured to: it is determined whether or not the positions of all of the plurality of reference feature points are within a predetermined allowable range.

According to a twenty-first aspect of the present invention, in the workpiece mounting support system according to the twentieth aspect, when one of the tables on which the workpiece whose posture is adjusted is placed is moved to the machining position, the position measurement sensor measures positions of a plurality of machining reference points, which are a plurality of feature points of the workpiece having substantially the same shape corresponding to the plurality of reference feature points, respectively, and the electronic circuit determines whether or not the positions of the plurality of machining reference points are within the allowable ranges, respectively.

According to a twenty-second aspect of the present invention, in the workpiece mounting support system according to the twenty-first aspect, the electronic circuit is configured to output a determination result of permission of machining to cause the machining device to machine the workpiece when it is determined that all of the positions of the plurality of machining reference points are within the allowable range. The electronic circuit is configured to output a determination result that machining is not permitted when it is determined that the position of at least one of the plurality of machining reference points does not exist within the allowable range, and cause the machining device to move one of the tables on which the moving workpiece is placed to the preparation position.

According to a twenty-third aspect of the present invention, a work mounting support system according to any one of the eighteenth aspect to the twenty-second aspect is configured to: the attitude of the reference workpiece and the attitude of the workpiece are defined by a rotation angle around a coordinate axis of a coordinate system set for causing the machining device to execute the machining program or a rotation center axis of the placing table.

According to a twenty-fourth aspect of the present invention, the work mounting support system according to any one of the eighteenth aspect to the twenty-third aspect is configured such that: the workpiece reference line is an edge of a boundary of the first image region obtained by image processing performed by the processor. According to a twenty-fifth aspect of the present invention, the work mounting support system according to any one of the eighteenth aspect to the twenty-fourth aspect is configured to: the workpiece reference line is a straight line or a circle.

According to a twenty-sixth aspect of the present invention, the work mounting support system according to any one of the eighteenth aspect to the twenty-fourth aspect is configured to: a plurality of camera setting reference lines are set in the background of the reference image at the boundary of a third image area occupied by a stationary object by an input device. The memory is configured to further store a plurality of positions in the reference image where the plurality of camera setting reference lines are respectively located as a plurality of camera setting reference line positions. The processor is configured to generate a measurement composite image in which a plurality of camera setting reference lines are superimposed on a measurement image, and the plurality of camera setting reference lines are respectively indicated at a plurality of positions that are the same as the plurality of camera setting reference lines in the measurement composite image. The boundary of the fourth image area occupied by the stationary object in the measurement composite image coincides with the plurality of camera setting reference lines.

According to a twenty-seventh aspect of the present invention, the work mounting support system of the twenty-sixth aspect is configured to: the camera setting reference line is an edge of a boundary of the third image region obtained by the image processing performed by the processor. According to a twenty-eighth aspect of the present invention, a work mounting support system according to the twenty-sixth or twenty-seventh aspect is configured: the camera setting reference line is a straight line.

According to a twenty-ninth aspect of the present invention, the workpiece mounting support system according to any one of the eighteenth aspect to the twenty-eighth aspect further includes a first additional camera and a second additional camera. The first additional camera is configured to photograph the reference workpiece and the workpiece on the placing table. The second additional camera is configured to photograph the reference workpiece and the workpiece on the placing table. The first additional camera is set to a first additional target arrangement in which the viewpoint position and the visual line direction of the first additional camera are set to a first additional reference position different from the reference viewpoint position and a first additional reference visual line direction non-parallel to the reference visual line direction, respectively. The second additional camera is configured to be a second additional target arrangement in which the viewpoint position and the line-of-sight direction of the second additional camera are set to a second additional reference position different from the reference viewpoint position and the first additional reference position, and a second additional reference line-of-sight direction non-parallel to the reference line-of-sight direction and the first additional reference line-of-sight direction, respectively. A plurality of first additional workpiece reference lines are set by an input device at the boundary of a fifth image area occupied by the reference workpiece in a first additional reference image, which is captured by a first additional camera and indicates the reference workpiece adjusted so that the posture of the reference workpiece is the workpiece reference posture. A plurality of second additional workpiece reference lines are set by the input device at the boundary of a sixth image area occupied by the reference workpiece in a second additional reference image, which is captured by a second additional camera and indicates the reference workpiece adjusted so that the posture of the reference workpiece is the workpiece reference posture. The memory is configured to store a plurality of positions in the first additional reference image where the plurality of first additional workpiece reference lines are located, respectively, as a plurality of first additional workpiece reference line positions. The memory is configured to store a plurality of positions in the second additional reference image where the plurality of second additional workpiece reference lines are respectively located as a plurality of second additional workpiece reference line positions. The processor is configured to generate a first additional measurement composite image in which a plurality of first additional workpiece reference lines are superimposed on a first additional measurement image representing a workpiece captured by a first additional camera, the plurality of first additional workpiece reference lines being respectively represented in the first additional measurement composite image at a plurality of positions identical to the plurality of first additional workpiece reference line positions. The processor is configured to generate a second additional measurement composite image in which a plurality of second additional workpiece reference lines are superimposed on a second additional measurement image representing the workpiece captured by the second additional camera, the plurality of second additional workpiece reference lines being respectively represented in the second additional measurement composite image at a plurality of positions identical to the plurality of second additional workpiece reference line positions. The display is configured to display the first additional measurement composite image when the first additional measurement image is captured, and to display the second additional measurement composite image when the second additional measurement image is captured. According to a thirtieth aspect of the present invention, the work mounting support system of the twenty-ninth aspect is configured to: two of the reference line-of-sight direction, the first additional reference line-of-sight direction, and the second additional reference line-of-sight direction are substantially perpendicular to each other, and an angle formed by a plane parallel to both of the two line-of-sight directions and the remaining line-of-sight directions other than the two line-of-sight directions is larger than 45 degrees.

According to a thirty-first aspect of the present invention, in the work mounting support system according to the twenty-second aspect, the work mounting support system is configured to: the processor determines, based on a third additional measurement image representing a first additional workpiece having a shape and a size substantially identical to those of the reference workpiece captured by a camera set to a target configuration, whether or not all positions of a plurality of first additional processing reference points, which are a plurality of feature points of the first additional workpiece having a substantially identical shape and a plurality of reference feature points, respectively, exist within an allowable range when one of the tables on which the first additional workpiece is placed is moved to a processing position, using a machine learning model learned using training data having a measurement image and a plurality of workpiece reference lines as input, and outputting at least one of the determination result, the result of determining whether each of the plurality of machining reference points is within the allowable range, and the amount of deviation of each of the plurality of machining reference points from the center value of the allowable range. According to a thirty-second aspect of the present invention, a work mounting support system of the thirty-first aspect is configured to: the machine learning model is learned using training data that also takes as input at least one of edges detected from the measurement image, deviation amounts of the plurality of workpiece reference line positions from the image center, a focal length of a lens of the camera, and a distortion parameter of the camera.

A workpiece mounting support program according to a thirteenth aspect of the present invention causes a processor to execute: acquiring a reference image representing a reference workpiece on a placing table adjusted so that a posture thereof is a workpiece reference posture, the reference image being captured by a camera of a target arrangement in which a viewpoint position and a visual line direction of the camera are set to a reference viewpoint position and a reference visual line direction, respectively, setting a plurality of workpiece reference lines at a boundary of a first image region occupied by the reference workpiece in the reference image, storing a plurality of positions within the reference image in which the plurality of workpiece reference lines are respectively located in a memory as a plurality of workpiece reference line positions, and acquiring a measurement image representing a workpiece having a shape and a size substantially identical to those of the reference workpiece placed on one of the placing table from which the reference workpiece is removed and an additional placing table different from the placing table and disposed in place of the placing table, the measurement image being captured by the camera of the target arrangement, a measurement composite image is generated by superimposing a plurality of workpiece reference lines on a measurement image, the plurality of workpiece reference lines being respectively displayed at a plurality of positions identical to the plurality of workpiece reference lines in the measurement composite image, and the measurement composite image is displayed on a display.

According to a thirty-fourth aspect of the present invention, in the workpiece mounting support program of the thirty-third aspect, the attitude of the reference workpiece and the attitude of the workpiece are defined by a rotation angle about a coordinate axis of a coordinate system set for causing the machining device to execute the machining program or a rotation center axis of the placement table.

According to a thirty-fifth aspect of the present invention, the workpiece mounting support program of the thirty-third or thirty-fourth aspect causes the processor to further execute: the edge of the boundary of the first image region obtained by the image processing is detected as a workpiece reference line. According to a thirty-sixth aspect of the present invention, in the workpiece mounting support program according to any one of the thirty-third to thirty-fifth aspects, the workpiece reference line is a straight line or a circle.

According to a thirty-sixth aspect of the present invention, the workpiece mounting support program of any one of the thirty-third to thirty-fifth aspects causes the processor to further execute: a plurality of camera setting reference lines are set in the background of the reference image at the boundary of a third image area occupied by a stationary object, and a plurality of positions in the reference image where the plurality of camera setting reference lines are respectively located are stored in a memory as a plurality of camera setting reference line positions, and a measurement composite image is generated by overlapping the plurality of camera setting reference lines with the measurement image, wherein the plurality of camera setting reference lines are respectively indicated in the measurement composite image at a plurality of positions identical to the plurality of camera setting reference line positions.

According to a seventeenth aspect of the present invention, the workpiece mounting support program of the thirty-sixth aspect causes the processor to further execute: the edge of the boundary of the third image region obtained by the image processing is detected as a camera setting reference line. According to a thirty-eighth aspect of the present invention, in the workpiece mounting support program of the thirty-fifth aspect or the thirty-sixth aspect, the camera setting reference line is a straight line.

According to a thirty-ninth aspect of the present invention, the workpiece mounting support program of the thirty-third aspect to the thirty-eighteenth aspect causes the processor to further execute: acquiring a first additional reference image representing a reference workpiece whose posture is adjusted, the first additional reference image being captured by a first additional camera arranged so that a viewpoint position and a line-of-sight direction of the first additional camera are respectively a first additional reference position different from a reference viewpoint position and a first additional reference line-of-sight direction non-parallel to the reference line-of-sight direction, acquiring a second additional reference image representing the reference workpiece whose posture is adjusted, the second additional camera being arranged so that a viewpoint position and a line-of-sight direction of the second additional camera are respectively a second additional reference position different from the reference viewpoint position and the first additional reference position, and a second additional target arranged so that a viewpoint position and a line-of-sight direction of the reference line-of-sight direction and the first additional reference line-of-sight direction are non-parallel to the reference line-of-sight direction and the first additional reference line-of-sight direction, setting a plurality of first additional workpiece reference lines at a boundary of a fifth image area occupied by the reference workpiece in the first additional reference image, setting a plurality of second additional workpiece reference lines at a boundary of a sixth image region occupied by the reference workpiece in the second additional reference image, storing a plurality of positions in the first additional reference image, at which the plurality of first additional workpiece reference lines are respectively located, as a plurality of first additional workpiece reference line positions in a memory, storing a plurality of positions in the second additional reference image, at which the plurality of second additional workpiece reference lines are respectively located, as a plurality of second additional workpiece reference line positions in the memory, acquiring a first additional measurement image representing the workpiece captured by the first additional camera set to the first additional target configuration, acquiring a second additional measurement image representing the workpiece captured by the second additional camera set to the second additional target configuration, generating a first additional measurement composite image in which the plurality of first additional workpiece reference lines and the first additional measurement image are overlapped, the plurality of first additional workpiece reference lines are respectively displayed at a plurality of positions identical to the positions of the plurality of first additional workpiece reference lines in the first additional measurement composite image, a second additional measurement composite image obtained by overlapping a plurality of second additional workpiece reference lines with the second additional measurement image is generated, the plurality of second additional workpiece reference lines are respectively displayed at a plurality of positions identical to the positions of the plurality of second additional workpiece reference lines in the second additional measurement composite image, and the first additional measurement composite image and the second additional measurement composite image are displayed on the display. According to a fortieth aspect of the present invention, in the workpiece mounting support program of the thirty-ninth aspect, two of the reference line-of-sight direction, the first additional reference line-of-sight direction, and the second additional reference line-of-sight direction are substantially perpendicular to each other, and an angle formed by a plane parallel to both of the two line-of-sight directions and the remaining line-of-sight directions other than the two line-of-sight directions is larger than 45 degrees.

According to a fourteenth aspect of the present invention, the workpiece mounting support program of the thirteenth aspect causes the processor to further execute: determining, based on a third additional measurement image representing a first additional workpiece having a shape and a size substantially the same as those of a reference workpiece captured by a camera set to a target configuration, whether or not all positions of a plurality of first additional processing reference points, which are a plurality of feature points of the first additional workpiece corresponding to the plurality of reference feature points when one of tables on which the first additional workpiece is placed is moved to a processing position, are within an allowable range, using a machine learning model learned using training data, which has a measurement image and a plurality of workpiece reference lines as input, and which has at least one of a determination result of whether or not all positions of the plurality of processing reference points are within the allowable range, a result of determining whether or not the plurality of processing reference points are within the allowable range, and a deviation amount of the plurality of processing reference points from a center value of the allowable range, as input And (6) outputting. According to a forty-second aspect of the present invention, in the workpiece mounting support program of the forty-first aspect, the machine learning model is learned using training data that further has as input at least one of an edge detected from the measurement image, a deviation amount from an image center of a plurality of workpiece reference line positions, a focal length of a lens of the camera, and a distortion parameter of the camera.

In the workpiece mounting method according to the first aspect, the workpiece mounting support system according to the seventeenth aspect, and the workpiece mounting support program according to the thirteenth aspect, the attitude of the workpiece can be adjusted with reference to the plurality of reference line positions of the reference workpiece adjusted so that the attitude is the workpiece reference attitude. Therefore, the postures of the plurality of workpieces can be efficiently adjusted. In the workpiece mounting method according to the first aspect, the workpiece mounting support system according to the eighteenth aspect, and the workpiece mounting support program according to the thirteenth aspect, the cameras can be set to the target arrangement so that the deviation of the reference line position due to the deviation of the cameras does not occur. Therefore, the posture of the workpiece can be adjusted more efficiently.

In the workpiece mounting method according to the second aspect and the workpiece mounting support system according to the nineteenth aspect, the posture of another workpiece can be adjusted at the preparation position while the workpiece is being machined at the machining position. Thus, by adjusting the posture of the next workpiece to be machined at the preparation position during machining by the machining apparatus, the postures of the plurality of workpieces can be efficiently adjusted so as to shorten the machining stop time of the machining apparatus.

In the workpiece mounting method according to the third aspect and the workpiece mounting support system according to the twentieth aspect, the reference attitude of the workpiece can be appropriately set by setting the allowable range in accordance with the machining allowances of the reference workpiece and the workpiece. Therefore, the machining device can machine the reference workpiece set to the workpiece reference posture.

In the workpiece mounting method according to the fourth aspect and the workpiece mounting support system according to the twenty-first aspect, it is possible to determine whether or not the machining device can machine the workpiece whose posture has been adjusted. In this way, it is determined whether or not the posture of the workpiece is defective before machining, and therefore the posture of the workpiece can be efficiently adjusted.

In the workpiece mounting method according to the fifth aspect and the workpiece mounting support system according to the twenty-second aspect, the workpiece that is determined to be unable to be machined by the machining device is returned to the preparation position. Therefore, readjustment of the posture of the workpiece becomes easy.

In the workpiece mounting method according to the sixth aspect, the workpiece mounting support system according to the twenty-third aspect, and the workpiece mounting support program according to the thirty-fourth aspect, the amount of coordinate conversion in the machining program can be reduced by the rotation angle of the coordinate axis serving as a reference for the coordinate conversion in the machining program.

In the workpiece mounting method according to the seventh aspect, the workpiece mounting support system according to the twenty-fourth aspect, and the workpiece mounting support program according to the thirty-fifth aspect, the workpiece reference line can be set by the edge detection processing, so that the setting operation of the workpiece reference line becomes easy.

In the workpiece mounting method according to the eighth aspect, the workpiece mounting support system according to the twenty-fifth aspect, and the workpiece mounting support program according to the thirty-sixth aspect, since the workpiece reference line is a simple figure such as a circle or a straight line, it is easy to match the boundary of the second image area with the workpiece reference line.

In the workpiece mounting method according to the ninth aspect, the workpiece mounting support system according to the twenty-sixth aspect, and the workpiece mounting support program according to the thirty-sixth aspect, the camera can be adjusted to a target arrangement using the measurement composite image.

In the workpiece mounting method according to the tenth aspect, the workpiece mounting support system according to the twenty-seventh aspect, and the workpiece mounting support program according to the thirty-seventh aspect, the camera setting reference line can be set by the edge detection processing, so that the setting operation of the camera setting reference line becomes easy.

In the workpiece mounting method according to the eleventh aspect, the workpiece mounting support system according to the twenty-eighth aspect, and the workpiece mounting support program according to the thirty-eighth aspect, since the camera setting reference line is a straight line, it is easy to match the boundary of the fourth image area with the camera setting reference line.

In the workpiece mounting method according to the twelfth aspect, the workpiece mounting support system according to the twenty-ninth aspect, and the workpiece mounting support program according to the thirty-ninth aspect, the three-dimensional attitude adjustment of the workpiece can be performed.

In the workpiece mounting method according to the thirteenth aspect, the workpiece mounting support system according to the thirtieth aspect, and the workpiece mounting support program according to the fortieth aspect, since the three cameras are arranged so as to be substantially perpendicular to each other, three-dimensional highly accurate posture adjustment of the workpiece can be performed.

In the workpiece mounting method according to the fourteenth aspect, the workpiece mounting support system according to the thirty-first aspect, and the workpiece mounting support program according to the forty-first aspect, it is possible to determine whether or not the posture adjustment of the workpiece by the human hand is appropriate without moving one of the tables to the machining position. Therefore, the time required for the posture adjustment can be further shortened, and the postures of a plurality of workpieces that need to be processed in the same manner can be more efficiently adjusted.

In the workpiece mounting method according to the fifteenth aspect, the workpiece mounting support system according to the thirty-second aspect, and the workpiece mounting support program according to the forty-second aspect, the edge detected from the measurement image is input, and the machine learning model is learned. Since the positional relationship between the workpiece reference line and the edge is an important feature quantity indicating the posture of the workpiece, high-precision determination based on the machine learning model can be expected. Further, in the workpiece mounting method according to the fifteenth aspect, the workpiece mounting support system according to the thirty-second aspect, and the workpiece mounting support program according to the forty-second aspect, the amount of deviation of the workpiece reference line position from the image center, the focal length of the lens of the camera, and the distortion parameter of the camera are input, and the machine learning model is learned. When the workpiece reference line is a straight line, the farther the workpiece reference line is from the image center, the greater the influence of aberration (aberration) is exerted, and therefore the error becomes larger. Therefore, by learning these parameters using the machine learning model, highly accurate determination based on the machine learning model can be expected.

In the workpiece mounting method according to the sixteenth aspect and the workpiece mounting support system according to the nineteenth aspect, the workpiece can be placed on either the placement table or the additional placement table, and the attitude of the workpiece can be adjusted.

According to the technique disclosed by the present invention, for example, the attitude of the workpiece can be adjusted by referring to the plurality of reference line positions of the reference workpiece adjusted so that the attitude becomes the workpiece reference attitude. Therefore, the postures of the plurality of workpieces can be efficiently adjusted.

Drawings

Fig. 1 is a diagram showing a schematic configuration of a work mounting support system according to a first embodiment.

Fig. 2 shows an example of a reference image according to the first embodiment.

Fig. 3 shows an example of the first additional reference image.

Fig. 4 shows an example of the second additional reference image.

Fig. 5 shows an example of a measurement composite image according to the first embodiment.

Fig. 6 shows an example of a measurement composite image in which the orientation of the workpiece is adjusted according to the first embodiment.

Fig. 7 shows another example of the measurement composite image in which the orientation of the workpiece is adjusted according to the first embodiment.

Fig. 8 shows an example of a first additional measurement composite image.

Fig. 9 shows an example of a second additional measurement composite image.

Fig. 10 is a flowchart showing a workpiece mounting method according to the first embodiment.

Fig. 11 is a flowchart showing a method of adjusting the attitude of the reference workpiece to the workpiece reference attitude according to the first embodiment.

Fig. 12 shows a method for adjusting the attitude of a reference workpiece according to the first embodiment.

Fig. 13 shows a method for adjusting the attitude of a reference workpiece according to the first embodiment.

Fig. 14 shows a method for adjusting the attitude of a reference workpiece according to the first embodiment.

Fig. 15 is a diagram for explaining the reference feature points and the allowable range.

Fig. 16 is a view for explaining a machining reference point.

Fig. 17 is a block diagram of a workpiece mounting support system according to a modification of the first embodiment.

Fig. 18 is an example of a composite image showing the determination result of the third additional measurement image.

Fig. 19 shows an example of a fourth additional measurement composite image.

Fig. 20 is a diagram showing a schematic configuration of a work mounting support system according to a second embodiment.

Fig. 21 is a diagram for explaining the postures of the reference workpiece and the workpiece in the second embodiment.

Fig. 22 is a diagram for explaining the postures of the reference workpiece and the workpiece in the second embodiment.

Fig. 23 is a flowchart showing a workpiece mounting method according to a second embodiment.

Fig. 24 is a flowchart showing a method of adjusting the attitude of the reference workpiece to the workpiece reference attitude according to the second embodiment.

Fig. 25 shows a method for adjusting the attitude of a reference workpiece according to a second embodiment.

Fig. 26 shows a method for adjusting the attitude of a reference workpiece according to a second embodiment.

Fig. 27 shows a method for adjusting the attitude of a reference workpiece according to a second embodiment.

Fig. 28 shows a method for adjusting the attitude of a reference workpiece according to a second embodiment.

Fig. 29 shows an example of a reference image according to the second embodiment.

Fig. 30 shows an example of a measurement composite image according to the second embodiment.

Fig. 31 shows an example of a measurement composite image in which the orientation of the workpiece is adjusted in the middle of the second embodiment.

Fig. 32 shows an example of a measurement composite image in which the orientation of the workpiece is adjusted according to the second embodiment.

Detailed Description

The present invention will be specifically described below with reference to the drawings showing embodiments thereof. In the drawings, the same reference numerals denote corresponding or substantially the same components.

< first embodiment >

< System construction >

Fig. 1 shows a schematic configuration of a work mounting support system 1 according to an embodiment of the present invention. The work attachment support system 1 includes a processing device 10, a camera 4, a first additional camera 4A, a second additional camera 4B, an image processing device 200, an input device 7, and a display 8. The machining device 10 includes, for example, a machining center (machining center) capable of performing milling. The machining device 10 includes a numerical control device 100, a placement table 2, and an additional placement table 2A. Here, the additional placement table 2A may be disposed separately from the processing apparatus 10 and may be connected to the processing apparatus 10. The image processing apparatus 200 includes a processor 5 and a memory 6. The image processing apparatus 200 may be a dedicated image processing apparatus that processes images from the camera 4, the first additional camera 4A, and the second additional camera 4B, or may be a general-purpose computer. The image processing apparatus 200 captures images from the camera 4, the first additional camera 4A, and the second additional camera 4B via a known camera input/output interface 201 such as HDMI (registered trademark). Fig. 1 shows an example in which the input device 7 and the display 8 are implemented by a touch panel display connected to the image processing device 200 via a wireless network NW. However, the input device 7 and the display 8 may be separate terminals such as a display and a mouse, for example, or may be connected to the image processing apparatus 200 by a cable. An interface for transmitting a signal of the input device 7 to the image processing apparatus 200 may be referred to as a first communication interface 202. An interface through which a video processed by the image processing apparatus 200 or a video taken in by the image processing apparatus 200 via the camera input/output interface 201 is output to the display 8 may be referred to as a video output interface 203. In the example of fig. 1, the first communication interface 202 and the video output interface 203 are implemented by wireless communication interfaces. The processor 5, the memory 6, the camera input/output interface 201, the first communication interface 202, and the video output interface 203 are connected by a bus 205.

The machining device 10 includes a guide rail 2G1 extending in the first direction D1 and a guide rail 2G2 extending in the second direction D2. The mount table 2 and the additional mount table 2A can be moved to a machining position MP where the workpiece W is machined by the machining device 10 and a preparation position AP located apart from the machining position MP. More specifically, the processing position MP and the preparation position AP are separated from each other in the first direction D1, and the placing table 2 and the additional placing table 2A are movable along the guide rail 2G1 extending in the first direction D1. The platform 2 and the additional platform 2A may be further moved from the preparation position AP to the additional preparation position AAP. More specifically, the preparation position AP and the additional preparation position AAP are separated from each other in a second direction D2 substantially perpendicular to the first direction D1, and the placement table 2 and the additional placement table 2A are movable along a guide rail 2G2 extending in the second direction D2.

The placing table 2 is configured to place the reference workpiece RW or the workpiece W alternatively. The posture of the reference workpiece RW or the posture of the workpiece W can be adjusted on the placing table 2. The reference workpiece RW is used to determine a reference posture of the workpiece W to be referred to for adjusting the posture of the workpiece W. The workpiece W has substantially the same shape and size as the reference workpiece RW. This means that the difference between the shape and size of the workpiece W and the shape and size of the reference workpiece RW is within a range of error that may occur in the manufacturing process of the workpiece W and the reference workpiece RW. For example, when the reference workpiece RW and the workpiece W are cast products manufactured by the same mold, the difference is within a range of an error (for example, about 5mm in the case of a large-sized workpiece) that may be generated by the mold forming.

The reference workpiece RW and the workpiece W are placed on the placing table 2 (additional placing table 2A) moved to the additional preparatory position AAP, temporarily fixed to the placing table 2 (additional placing table 2A), and sent to the preparatory position AP. The posture of the reference workpiece RW is adjusted on the placing table 2 moved to the preparation position AP. The posture of the workpiece W is adjusted on the placing table 2 moved to the preparation position AP or the additional placing table 2A. The camera 4, the first additional camera 4A, and the second additional camera 4B are configured to capture the reference workpiece RW and the workpiece W on the placing table 2. More specifically, the camera 4, the first additional camera 4A, and the second additional camera 4B capture an image of one of the mount table 2 and the additional mount table 2A moved to the preparation position AP.

The cameras 4 are set in a target arrangement such that the viewpoint position and the visual line direction of the cameras 4 are the reference viewpoint position and the reference visual line direction, respectively. The first additional camera 4A is set so that the viewpoint position and the line-of-sight direction of the first additional camera 4A are disposed as a first additional target at a first additional reference position different from the reference viewpoint position and at a first additional reference line-of-sight direction non-parallel to the reference line-of-sight direction, respectively. The second additional camera 4B is set so that the viewpoint position and the line-of-sight direction of the second additional camera 4B are respectively arranged as a second additional target of a second additional reference position different from the reference viewpoint position and the first additional reference position and a second additional reference line-of-sight direction non-parallel to the reference line-of-sight direction and the first additional reference line-of-sight direction. In the example of fig. 1, the reference line-of-sight direction is parallel to the first direction D1, the first additional reference line-of-sight direction is parallel to the second direction D2, and the second additional reference line-of-sight direction (indicated as the third direction D3 in fig. 1) is inclined with respect to a plane parallel to each of the first direction D1 and the second direction D2. In this way, it is preferable that two of the reference line-of-sight direction, the first additional reference line-of-sight direction, and the second additional reference line-of-sight direction are substantially perpendicular to each other, and an angle formed by a plane parallel to both of the two line-of-sight directions and the remaining line-of-sight directions other than the two line-of-sight directions is greater than 45 degrees. The positions and the line of sight directions of the camera 4, the first additional camera 4A, and the second additional camera 4B are not limited to those shown in fig. 1, and may be changed.

The machining device 10 is configured to machine a workpiece W. The machining device 10 may machine the reference workpiece RW. The processing apparatus 10 includes: a main shaft 9A for mounting a tool for machining a workpiece; a spindle moving mechanism 9B; and a position measuring sensor 9. The numerical control apparatus 100 includes an electronic circuit 110 and an input interface 111. The position measuring sensor 9 is, for example, a contact probe. The position measuring sensor 9 is attached to the main shaft 9A, and the main shaft moving mechanism 9B moves the main shaft 9A. For example, the position measurement sensor 9 is attached to the spindle 9A by exchanging with a tool attached to the spindle 9A. The electronic circuit 110 is, for example, a controller that executes a machining program. The position measuring sensor 9, the electronic circuit 110, the input interface 111, the spindle 9A, and the spindle moving mechanism 9B are connected to each other by a bus 113 and/or a cable not shown. The position measuring sensor 9 is configured to measure the positions of a plurality of reference characteristic points, which are a plurality of characteristic points of the shape of the reference workpiece RW, when the placing table 2 on which the reference workpiece RW whose posture is adjusted to the workpiece reference posture is moved to the machining position MP. The position measuring sensor 9 is configured to measure positions of a plurality of machining reference points, which are a plurality of feature points of substantially the same shape of the workpiece corresponding to the plurality of reference feature points, when the one of the tables on which the workpiece W having the adjusted posture is placed is moved to the machining position MP. The details of the workpiece reference attitude and the plurality of reference feature points will be described later. The electronic circuit 110 is configured to determine whether or not all of the positions of the plurality of reference feature points are within a predetermined allowable range. The electronic circuit 110 is configured to determine whether or not the positions of the plurality of machining reference points are within the allowable ranges. The allowable range is set according to the machining allowance of the reference workpiece RW and the workpiece W. The process of determining by the electronic circuit 110 may also be referred to as a measurement process. The electronic circuit 110 is configured to output a determination result of permission of machining when it is determined that all the positions of the plurality of machining reference points are within the allowable range, and cause the machining device 10 to machine the workpiece W. The electronic circuit 110 is configured to output a determination result that machining is not permitted when it is determined that the position of at least one of the plurality of machining reference points does not exist within the allowable range, and move the one of the tables on which the workpiece W is placed, which has been moved to the machining position MP, to the preparation position AP.

The posture of the reference workpiece RW is adjusted on the placing table 2 moved to the machining position MP so that all the positions of the plurality of reference feature points are within a predetermined allowable range. The posture of the reference workpiece RW when it is determined that all the positions of the plurality of reference feature points are within the predetermined allowable range is the workpiece reference posture. Therefore, when the placing table 2 on which the reference workpiece RW whose posture is adjusted to the workpiece reference posture is placed is moved to the machining position MP, all the positions of the plurality of reference characteristic points, which are the plurality of characteristic points of the shape of the reference workpiece RW, are present within the predetermined allowable range.

After the posture of the reference workpiece RW is adjusted to the workpiece reference posture, the camera 4 captures (captures) a reference image IB of the reference workpiece RW showing that the posture of the reference workpiece RW is adjusted to the workpiece reference posture. The first additional camera 4A captures a first additional reference image IBA of the reference workpiece RW indicating that the posture of the reference workpiece RW is adjusted to the workpiece reference posture. The second additional camera 4B captures a second additional reference image IBB of the reference workpiece RW indicating that the posture of the reference workpiece RW is adjusted to the workpiece reference posture. The display 8 displays the reference image IB, the first additional reference image IBA, and the second additional reference image IBB.

First, in the subsequent shooting, the user sets a plurality of camera setting reference lines CRL1 to CRL2 on the background of the reference image IB at the boundary of the third image area IR3 occupied by the stationary object by using the input device 7 so as to check whether or not the arrangement (viewpoint position and visual line direction) of the cameras 4 is deviated from the target arrangement. That is, the plurality of camera setting reference lines CRL1 to CRL2 are set by the input device 7 at the boundary of the third image area IR3 occupied by the stationary object in the background of the reference image IB. Similarly, the user sets a plurality of first additional camera setting reference lines CRL3 to CRL4 on the background of the first additional reference image IBA at the boundary of the image area BG1 occupied by the stationary object by using the input device 7. That is, the plurality of first additional camera setting reference lines CRL3 to CRL4 are set on the boundary of the image area BG1 occupied by the stationary object in the background of the first additional reference image IBA by the input device 7. The user sets a plurality of second additional camera setting reference lines CRL5 to CRL6 on the background of the second additional reference image IBB at the boundary of the image area BG2 occupied by the stationary object by using the input device 7. That is, the plurality of second additional camera setting reference lines CRL5 to CRL6 are set on the boundary of the image area BG2 occupied by the stationary object in the background of the second additional reference image IBB by the input device 7. In fig. 2 to 4, the reference image IB, the first additional reference image IBA, and the second additional reference image IBB are hatched in a third image region IR3, an image region BG1, and an image region BG2 occupied by a still object, respectively.

As shown in fig. 2 to 4, the camera setting reference lines CRL1 to CRL6 are preferably straight lines. The camera setting reference lines CRL1 to CRL2 may be edges of the boundary of the third image region IR3 obtained by the image processing performed by the processor 5. Similarly, the first additional camera setting reference lines CRL3 to CRL4 may be edges of the boundary of the image region BG1 obtained by the image processing performed by the processor 5. The second additional camera setting reference lines CRL5 to CRL6 may be edges of the boundary of the image region BG2 obtained by the image processing performed by the processor 5. In this case, the regions of the reference image IB, the first additional reference image IBA, and the second additional reference image IBB for edge detection may be specified by the input device 7, or any one of a plurality of edges detected from the reference image IB, the first additional reference image IBA, and the second additional reference image IBB may be selected by the input device 7.

Next, the user sets a plurality of workpiece reference lines RL1 to RL3 in the reference image IB by the input device 7 at the boundary of the first image area IR1 occupied by the reference workpiece RW while viewing the reference image IB as shown in fig. 2 displayed on the display 8. That is, the input device 7 sets the plurality of workpiece reference lines RL1 to RL3 in the reference image IB at the boundary of the first image area IR1 occupied by the reference workpiece RW. While viewing the first additional reference image IBA shown in fig. 3 displayed on the display 8, the user sets the plurality of first additional workpiece reference lines RL4 to RL5 in the first additional reference image IBA at the boundary of the fifth image area IR5 occupied by the reference workpiece RW via the input device 7. That is, as shown in fig. 3, the plurality of first additional workpiece reference lines RL4 to RL5 are set in the first additional reference image IBA by the input device 7 at the boundary of the fifth image area IR5 occupied by the reference workpiece RW. While viewing the second additional reference image IBB as shown in fig. 4 displayed on the display 8, the user sets a plurality of second additional workpiece reference lines RL6 to RL7 in the second additional reference image IBB at the boundary of the sixth image area IR6 occupied by the reference workpiece RW by the input device 7. That is, the plurality of second additional workpiece reference lines RL6 to RL7 are set at the boundaries of the sixth image area IR6 occupied by the reference workpiece RW in the second additional reference image IBB by the input device 7. In fig. 2 to 4, in the reference image IB, the first additional reference image IBA, and the second additional reference image IBB, a dot pattern is added to the first image area IR1, the fifth image area IR5, and the sixth image area IR6 occupied by the reference workpiece RW, respectively.

As shown in fig. 2 to 4, the workpiece reference lines RL1 to RL7 are preferably straight lines or circles. Further, the workpiece reference lines RL1 to RL3 may be edges of the boundary of the first image region IR1 obtained by the image processing performed by the processor 5. Similarly, the first additional workpiece reference lines RL4 to RL5 may be edges of the boundary of the fifth image area IR5 obtained by the image processing performed by the processor 5. Similarly, the second additional workpiece reference lines RL6 to RL7 may be edges of the boundary of the sixth image area IR6 obtained by the image processing performed by the processor 5. In this case, the regions of the reference image IB, the first additional reference image IBA, and the second additional reference image IBB for edge detection may be specified by the input device 7, or any one of a plurality of edges detected from the reference image IB, the first additional reference image IBA, and the second additional reference image IBB may be selected by the input device 7.

The memory 6 is configured to store a plurality of positions in the reference image IB, at which the plurality of workpiece reference lines RL1 to RL3 are respectively located, as a plurality of workpiece reference line positions. The memory 6 is configured to store a plurality of positions in the reference image IB, at which the plurality of camera setting reference lines CRL1 to CRL2 are located, as a plurality of camera setting reference line positions. These reference line positions are defined by coordinates of end points thereof (RP 1 to RP4, CRP1 to CRP4 in the example of fig. 2) in the image coordinate system of the reference image IB, for example, in the case where the reference line is a straight line, and by coordinates of a point (for example, any one of the center RP5, the points RP6 to RP8 on the circumference, and the like) for specifying a circular shape in the image coordinate system of the reference image IB, in the example of fig. 2, in the case where the reference line is a circle. The coordinates of these points may be expressed in terms of sub-pixel units. Similarly, the memory 6 is configured to store a plurality of positions in the first additional reference image IBA where the plurality of first additional workpiece reference lines RL4 to RL5 are respectively located as a plurality of first additional workpiece reference line positions. The memory 6 is configured to further store a plurality of positions in the first additional reference image IBA where the plurality of first additional camera setting reference lines CRL3 to CRL4 are respectively located as a plurality of first additional camera setting reference line positions. The memory 6 is configured to store a plurality of positions in the second additional reference image IBB where the plurality of second additional workpiece reference lines RL6 to RL7 are respectively located as a plurality of second additional workpiece reference line positions. The memory 6 is configured to further store, as the plurality of second additional camera setting reference line positions, a plurality of positions in the second additional reference image IBB where the plurality of second additional camera setting reference lines CRL5 to CRL6 are respectively located.

After the above processing, the placing table 2 on which the reference workpiece RW is placed is moved to the machining position MP or the additional preparation position AAP. When the placing table 2 is moved to the machining position MP, the reference workpiece RW is machined by the machining device 10. Next, one of the additional placing table 2A on which the workpiece W is placed and the placing table 2 on which the workpiece W is placed in place of the reference workpiece RW is moved to the preparation position AP. When the posture of the workpiece W is adjusted on the one table, the camera 4 captures a measurement image representing the workpiece W. Similarly, the first additional camera 4A captures a first additional measurement image representing the workpiece W. The second additional camera 4B captures a second additional measurement image representing the workpiece W.

The processor 5 IS configured to generate a measurement composite image IS in which the plurality of camera setting reference lines CRL1 to CRL2 are superimposed on the measurement image, and to cause the plurality of camera setting reference lines CRL1 to CRL2 to be respectively indicated at a plurality of positions identical to the plurality of camera setting reference line positions in the measurement composite image IS. The processor 5 IS configured to generate a measurement composite image IS in which the plurality of workpiece reference lines RL1 to RL3 are superimposed on a measurement image representing the workpiece W captured by the camera 4 when the posture of the workpiece W IS adjusted on the one stage, and to cause the plurality of workpiece reference lines RL1 to RL3 to be respectively represented in the measurement composite image IS at a plurality of positions identical to the positions of the plurality of workpiece reference lines. The display 8 IS configured to display the measurement composite image IS when the measurement image IS captured.

Fig. 5 shows an example of the measurement composite image IS of the first embodiment. In fig. 5, the second image area IR2 occupied by the workpiece W IS represented by a dot pattern in the measurement composite image IS. In addition, the boundary of the fourth image area IR4 occupied by the stationary object corresponding to the third image area IR3 IS hatched in the measurement composite image IS. The user first adjusts the arrangement of the camera 4 while referring to the measurement composite image IS so that the boundary of the fourth image area IR4 occupied by the stationary object in the measurement composite image IS overlaps with the plurality of camera setting reference lines CRL1 to CRL 2. Thus, the camera 4 is set to the target arrangement when the measurement image is captured.

Next, the user adjusts the posture of the workpiece W on the one of the stages so that the boundary of the second image area IR2 occupied by the workpiece W in the measurement composite image IS substantially parallel to or substantially coincides with the plurality of workpiece reference lines RL1 to RL 3. Fig. 6 and 7 show examples of the measurement composite image IS showing the workpiece W whose posture IS adjusted in this way. Fig. 6 shows an example of a case where the boundaries between the plurality of workpiece reference lines RL1 to RL3 and the second image area IR2 actually coincide with each other. Fig. 7 shows an example in which the boundaries between the plurality of workpiece reference lines RL1 to RL3 and the second image area IR2 are substantially parallel to each other. In fig. 7, the boundary line BL1 between the workpiece reference line RL1 and the second image region IR2 is separated so as to be parallel to each other, but the boundary line BL2 between the workpiece reference line RL2 and the second image region IR2 actually coincides with each other. In this way, "the boundary of the second image region IR2 is substantially parallel to the plurality of workpiece reference lines RL1 to RL 3" may be such that the boundary of the second image region IR2 substantially coincides with a part of the plurality of workpiece reference lines RL1 to RL3, or the distances between the plurality of workpiece reference lines RL1 to RL3 and the boundary of the second image region IR2 may be different from each other.

The processor 5 and the display 8 perform the same processing on the first additional measurement image and the second additional measurement image. That is, as shown in fig. 8, the processor 5 is configured to generate a first additional measurement synthetic image ISA in which the plurality of first additional camera setting reference lines CRL3 to CRL4 are superimposed on the first additional measurement image, and to cause the plurality of first additional camera setting reference lines CRL3 to CRL4 to be respectively displayed at a plurality of positions identical to the plurality of first additional camera setting reference line positions in the first additional measurement synthetic image ISA. The processor 5 is configured to generate a first additional measurement synthetic image ISA in which the plurality of first additional workpiece reference lines RL4 to RL5 are superimposed on the first additional measurement image, and to cause the plurality of first additional workpiece reference lines RL4 to RL5 to be respectively displayed at a plurality of positions identical to the positions of the plurality of first additional workpiece reference lines in the first additional measurement synthetic image ISA. As shown in fig. 9, the processor 5 is configured to generate a second additional measurement synthetic image ISB in which the plurality of second additional camera setting reference lines CRL5 to CRL6 are superimposed on the second additional measurement image, and to cause the plurality of second additional camera setting reference lines CRL5 to CRL6 to be respectively displayed at a plurality of positions in the second additional measurement synthetic image ISB, the positions being the same as the positions of the plurality of second additional camera setting reference lines. The processor 5 is configured to generate a second additional measurement synthetic image ISB in which the plurality of second additional workpiece reference lines RL6 to RL7 are superimposed on the second additional measurement image, and to cause the plurality of second additional workpiece reference lines RL6 to RL7 to be respectively displayed at a plurality of positions identical to the positions of the plurality of second additional workpiece reference lines in the second additional measurement synthetic image ISB. The display 8 is configured to display the first additional measurement synthetic image ISA when the first additional measurement image is captured. The display 8 is configured to display the second additional measurement combined image ISB when the second additional measurement image is captured.

The user adjusts the arrangement of the first additional camera 4A so that the boundaries of the image area BG3 occupied by a still object (a still object corresponding to the image area BG 1) in the first additional measurement synthetic image ISA overlap the plurality of first additional camera setting reference lines CRL3 to CRL4 while referring to the first additional measurement synthetic image ISA. Thus, when the first additional measurement image is captured, the first additional camera 4A is set to the first additional target configuration. The user adjusts the arrangement of the second additional camera 4B so that the boundaries of the image area BG4 occupied by the still object (the still object corresponding to the image area BG 2) overlap the plurality of second additional camera setting reference lines CRL5 to CRL6 in the second additional measurement composite image ISB while referring to the second additional measurement composite image ISB. Thereby, when the second additional measurement image is captured, the second additional camera 4B is set to the second additional target arrangement. The user adjusts the posture of the workpiece W on the one of the above-described stages so that the boundary of the seventh image area IR7 occupied by the workpiece W in the first additional measurement synthetic image ISA is substantially parallel to or substantially coincides with the plurality of first additional workpiece reference lines RL4 to RL 5. The user adjusts the posture of the workpiece W on the one of the above-described stages so that the boundary of the eighth image area IR8 occupied by the workpiece W in the second additional measurement combined image ISB is substantially parallel to or substantially coincides with the plurality of second additional workpiece reference lines RL6 to RL 7.

Typically, the processor 5 executes the workpiece mounting support program 6p stored in the memory 6 to acquire images from the camera 4, the first additional camera 4A, and the second additional camera 4B, thereby realizing the processing of the processor 5. However, the processor 5 may be realized by a dedicated image processing processor or an integrated circuit. In the following, a case where the processor 5 executes the workpiece mounting support program 6p will be described as an example, and details of a workpiece mounting method using the workpiece mounting support system 1 will be described.

< method for mounting work >

Fig. 10 is a flowchart illustrating a workpiece mounting method according to the first embodiment. In this method, in step S1, the user places the reference workpiece RW on the placing table 2, and adjusts the posture of the reference workpiece RW on the placing table 2 so that the posture of the reference workpiece RW becomes the workpiece reference posture. Fig. 11 is a flowchart of a specific process of step S1. First, in step S101, the dial gauge 101 (see fig. 12 to 13) is attached to the spindle 9A (see fig. 1) of the machining device 10. In step S102, as shown in fig. 12, two points separated in the Y direction that should be located on a plane (X-Y reference plane) parallel to the X-Y plane of the workpiece coordinate system (coordinate system set for the machining apparatus 10 to execute the machining program) in the machining program are measured by the dial indicator 101. The dial gauge 101 indicates values corresponding to the Z coordinate of these two points. Preferably, the two points are separated as much as possible in the Y direction. If the dial gauge values at the two points are different (no in step S103), the adjustment is performed by the jack so that the dial gauge values are the same (the process of step S104 → S102 → S103 is repeated until yes in step S103). By the processing in steps S101 to S104, the angle (roll angle) formed by the X-Y plane when viewed from the X direction and the X-Y reference plane when moving in parallel so as to pass through the origin of the workpiece coordinate system becomes substantially 0 degree.

If the setting of the roll angle is finished (yes in step S103), then in step S105, as shown in fig. 13, two points separated in the X direction that should be located on a plane (X-Y reference plane) parallel to the X-Y plane of the workpiece coordinate system in the machining program are measured with the dial indicator 101. The X-Y reference plane of step S105 may be the same as or different from the X-Y reference plane of step S102. Preferably, the two points are separated as much as possible in the X direction. Even in this case, the dial gauge 101 indicates the values corresponding to the Z coordinate of the two points. If the dial gauge values at the two points are different (no in step S106), the adjustment is performed by the jack so that the dial gauge values are the same (the process of step S107 → S105 → S106 is repeated until yes in step S106). By the processing in steps S105 to S106, the angle (pitch angle) between the X-Y plane and the X-Y reference plane when the X-Y plane is moved in parallel so as to pass through the origin of the workpiece coordinate system as viewed in the Y direction becomes substantially 0 degree.

When the setting of the pitch angle is completed (yes in step S106), the rod 102 (see fig. 14) is attached to the main shaft 9A (see fig. 1) of the machining apparatus 10. In step S109, as shown in fig. 14, the workpiece RW is moved to a point where a portion (symmetric portion SP) that is to be arranged in the machining program to be plane-symmetric with respect to a plane parallel to the X-Z plane of the workpiece coordinate system is located. Preferably, the symmetric portion SP extends in the X direction. Next, in step S110, the distance between the bar 102 and the both side edges of the reference workpiece RW in the Y direction is measured by the scale 103. If the distance between the both side edges is different (no in step S111), the main shaft 9A is moved to be the same (the process of step S112 → S110 → S111 is repeated until yes in step S111). In fig. 14, a position where the both side edge distances become equal is illustrated as P1. If the both side edge distances become equal (yes in step S111), the main shaft 9A is moved in parallel from the position P1 in the X direction on the symmetrical portion SP in step S113. In step S114, the distance between the bar 102 and both side edges of the reference workpiece RW in the Y direction is measured with the ruler 103. If the distance between the side edges is different (no in step S115), the adjustment is performed by the jack so that the side edges are the same (the process of step S116 → S114 → S115 is repeated until yes in step S115). By the processing in steps S108 to S116, the angle (yaw angle) formed by the Z-X plane and the symmetry plane of the symmetric portion SP when viewed from the Z direction is substantially 0 degree. The posture of the reference workpiece RW set in this way is referred to as a workpiece reference posture. Therefore, the posture of the reference workpiece RW is defined by the rotation angles (roll angle, pitch angle, yaw angle) around the coordinate axes of a coordinate system (workpiece coordinate system) set for causing the machining device 10 to execute the machining program. Since the attitude of the workpiece W is also determined from the workpiece reference attitude, the attitude of the workpiece W is specified by the rotation angles (roll angle, pitch angle, yaw angle) around the coordinate axes of the coordinate system (workpiece coordinate system) set for causing the machining apparatus 10 to execute the machining program.

Returning to fig. 10, if the posture of the reference workpiece RW is adjusted to the workpiece reference posture in step S1, in step S2, the reference workpiece RW is set with respective allowable ranges of a plurality of reference characteristic points BP1 to BPn, which are a plurality of characteristic points of the shape of the reference workpiece RW and the machining origin MO. This setting will be described with reference to fig. 15. Fig. 15 shows only the machining origin MO and the reference feature points BP1 and BPn for convenience of explanation. In fig. 15, the ideal arrangement positions and arrangement postures of the reference workpiece RW and the workpiece W are indicated by broken lines. The reference workpiece RW and the workpiece W shown by the broken line in fig. 15 are manufactured according to the design values without manufacturing errors. The machining program executed by the machining device 10 has a command for cutting the machining allowance portion of the workpiece occupying the space indicated by the broken line. Such a model of a workpiece managed by a machining program is referred to as a workpiece model. However, since the reference workpiece RW and the workpiece W are set to such ideal arrangement positions and arrangement postures, the operation burden is large and manufacturing errors are actually caused, and therefore the machining device 10 operates to allow the parallel deviation of the positions. Specifically, a point which is not machined or a point which is easy to be a reference (for example, the highest point or the like) in the reference workpiece RW and the workpiece W is set as the machining origin MO, the coordinates of the machining origin MO in the workpiece coordinate system are measured by the position measuring sensor 9, coordinate transformation is performed by taking into consideration only parallel movement using the difference between the coordinates and the coordinates of the machining origin MO on the workpiece model, and the machining program is executed in the coordinate system in which the coordinate transformation is performed to execute the machining operation. In fig. 15, the original workpiece coordinate system is represented by an XYZ coordinate system, and the transformed workpiece coordinate system is represented by an X ' Y ' Z ' coordinate system.

Therefore, for example, if the position on the workpiece model of the reference feature point BP1 is BP1o (Xo1, Yo1, Zo1), ideally, the reference feature point BP1 should be located at the position BP1i represented by (Xo1, Yo1, Zo1) in the X ' Y ' Z ' coordinate system, but since the manufacturing error, the roll angle, the pitch angle, and the yaw angle of the reference workpiece RW and the workpiece W are deviated from 0 degree in microseconds, the actual position BP1r that becomes BP1 is deviated from BP1 i. Therefore, for example, if the coordinate values of BP1r expressed in the X ' Y ' Z ' coordinate system are (Xr1, Yr1, Zr1), machining can be performed at the reference feature point BP1 if the absolute values of (Xr1-Xo1), the absolute values of (Yr1-Yo1), and the absolute values of (Zr1-Zo1) are smaller than the threshold value determined from the machining allowance. In other words, if the absolute values of (Xr1-Xo1), of (Yr1-Yo1), and of (Zr1-Zo1) are values smaller than the above-described threshold value, it can be said that the position of the reference feature point BP1 exists within the predetermined allowable range. The allowable range can be similarly determined for the other reference feature points BP2 (not shown) to BPn.

However, if the reference feature point BPn is a point on the surface farthest from the machining origin MO, it is preferable to set the threshold value to a value smaller than half the machining allowance, for example. However, when there is a surface farther from the machining origin MO than the reference feature point BP1 like the reference feature point BP1, it is preferable to obtain an intersection point BF1o of a half-straight line L passing through BP1o from the MO on the workpiece model and a surface farthest from the machining origin MO, and set the threshold value to a value smaller than a value obtained by multiplying a distance DP 1/DF1 by a half of the machining allowance from the machining origin MO to the distance DP1 from the machining origin MO to the machining origin 1o and the distance DF1 from the machining origin MO to the BF1 o. This can prevent the surface farthest from the machining origin MO from being processed. The threshold value may be empirically determined in such a manner as to satisfy the above-described conditions. When the machining allowance differs in size depending on the X direction (X 'direction), the Y direction (Y' direction), and the Z direction (Z 'direction), the threshold value may be determined for each of the X direction (X' direction), the Y direction (Y 'direction), and the Z direction (Z' direction).

In step S2, a sufficient number of reference feature points BP1 to BPn are selected when it is determined whether machining is possible. Preferably, the reference feature points BP1 to BPn are edges and corners whose positions are easily recognized by the contact probe. Preferably, the reference feature points BP1 to BPn are points as far as possible from the machining origin MO. If the machining origin MO and the reference feature points BP1 to BPn are determined, the allowable range can be determined by determining a threshold value for each reference feature point by the above-described method. Further, since the reference workpiece RW on the mount table 2 adjusted so that the posture thereof becomes the workpiece reference posture is adjusted so that the roll angle, the pitch angle, and the yaw angle become 0 degrees, there is only an error of the degree of manufacturing error, when the mount table 2 on which the reference workpiece RW adjusted so that the posture thereof becomes the workpiece reference posture is moved to the machining position MP, all the positions of the plurality of reference characteristic points BP1 to BPn are within the allowable range.

In step S3, the camera 4 is set to the target arrangement, and when the reference image IB is captured, the viewpoint position and the line of sight direction of the camera 4 on the imaging platform 2 are set to the reference viewpoint position and the reference line of sight direction, respectively. Similarly, the first additional camera 4A is set to the first additional target arrangement, and when the first additional reference image IBA is captured, the viewpoint position and the line-of-sight direction of the first additional camera 4A on the imaging platform 2 are set to the first additional reference position different from the reference viewpoint position and the first additional reference line-of-sight direction non-parallel to the reference line-of-sight direction, respectively. Similarly, the second additional camera 4B is set to the second additional target arrangement, and when the second additional reference image IBB is captured, the viewpoint position and the line-of-sight direction of the second additional camera 4B on the imaging platform 2 are set to the second additional reference position different from the reference viewpoint position and the first additional reference position, and the second additional reference line-of-sight direction non-parallel to the reference line-of-sight direction and the first additional reference line-of-sight direction, respectively.

In step S4, a reference image IB representing the reference workpiece RW whose posture has been adjusted is acquired, which is captured by the camera 4 set to the target arrangement. Specifically, the workpiece mounting support program 6p causes the processor 5 to execute the following processing: a reference image IB is acquired which is captured by a camera 4 arranged so that the viewpoint position and the visual line direction of the camera 4 are respectively set as a reference viewpoint position and a reference visual line direction, and which indicates a reference workpiece RW on a mounting table 2 whose posture is adjusted to a workpiece reference posture. The first additional reference image IBA representing the reference workpiece RW whose posture is adjusted, which is captured by the first additional camera 4A set to the first additional target arrangement, is acquired. Specifically, the workpiece mounting support program 6p causes the processor 5 to execute the following processing: a first additional reference image IBA representing the reference workpiece RW with the adjusted posture is acquired, which is captured by a first additional camera 4A disposed so that the viewpoint position and the visual line direction of the first additional camera 4A are respectively a first additional reference position different from the reference viewpoint position and a first additional target of a first additional reference visual line direction non-parallel to the reference visual line direction. A second additional reference image IBB representing the reference workpiece RW whose posture is adjusted and which is captured by the second additional camera 4B set to the second additional target arrangement is acquired. Specifically, the workpiece mounting support program 6p causes the processor 5 to execute the following processing: a second additional reference image IBB representing the reference workpiece RW after the posture adjustment is acquired, which is captured by a second additional camera 4B disposed so that the viewpoint position and the line-of-sight direction of the second additional camera 4B become a second additional reference position different from the reference viewpoint position and the first additional reference position, and a second additional target of a second additional reference line-of-sight direction non-parallel to the reference line-of-sight direction and the first additional reference line-of-sight direction, respectively.

In step S5, a plurality of camera setting reference lines CRL1 to CRL2 are set in the background of the reference image IB at the boundary of the third image area IR3 occupied by the stationary object. The plurality of positions in the reference image IB where the plurality of camera setting reference lines CRL1 to CRL2 are located are stored in the memory 6 as a plurality of camera setting reference line positions. Specifically, the workpiece mounting support program 6p causes the processor 5 to execute the following processing: a plurality of camera setting reference lines CRL1 to CRL2 are set at the boundary of the third image area IR3 occupied by the stationary object in the background of the reference image IB, and a plurality of positions in the reference image IB where the plurality of camera setting reference lines CRL1 to CRL2 are respectively located are stored in the memory 6 as a plurality of camera setting reference line positions. Specifically, the workpiece mounting support program 6p causes the processor 5 to execute the following processing: when an input from the input device 7 is received, the camera setting reference lines CRL1 to CRL2 to be set by the user are displayed in a superimposed manner on the reference image IB, and if the input of the settings of the camera setting reference lines CRL1 to CRL2 to be displayed is received from the input device 7, a plurality of positions in the reference image IB where the plurality of camera setting reference lines CRL1 to CRL2 are located are stored in the memory 6 as a plurality of camera setting reference line positions. Alternatively, the workpiece mounting support program 6p causes the processor 5 to execute: the edges of the boundary of the third image region IR3 found by the image processing are detected as camera setting reference lines CRL1 to CRL 2. Further, the workpiece mounting support program 6p causes the processor 5 to execute the following processing: when the detected edge is displayed so as to overlap the reference image IB and an input for selecting an edge is received from the input device 7, a plurality of positions in the reference image IB where the plurality of camera setting reference lines CRL1 to CRL2 relating to the edge are respectively located are stored in the memory 6 as a plurality of camera setting reference line positions.

Similarly, a plurality of first additional camera setting reference lines CRL3 to CRL4 are set in the background of the first additional reference image IBA at the boundary of the image area BG1 occupied by the stationary object. A plurality of positions in the first additional reference image IBA where the plurality of first additional camera setting reference lines CRL3 to CRL4 are respectively located are stored in the memory 6 as a plurality of first additional camera setting reference line positions. A plurality of second additional camera setting reference lines CRL5 to CRL6 are set on the background of the second additional reference image IBB at the boundary of the image area BG2 occupied by the stationary object. A plurality of positions in the second additional reference image IBB where the plurality of second additional camera setting reference lines CRL5 to CRL6 are respectively located are stored in the memory 6 as a plurality of second additional camera setting reference line positions. Even in these cases, specifically, the workpiece installation support program 6p causes the processor 5 to execute the same processing as that of the workpiece installation support program 6p for setting the plurality of camera setting reference lines CRL1 to CRL2 and storing the same in the memory 6.

In step S6, a plurality of workpiece reference lines RL1 to RL3 are set in the reference image IB at the boundary of the first image area IR1 occupied by the reference workpiece RW. The plurality of positions in the reference image IB where the plurality of workpiece reference lines RL1 to RL3 are located are stored in the memory 6 as a plurality of workpiece reference line positions. Specifically, the workpiece mounting support program 6p causes the processor 5 to execute the following processing: a plurality of workpiece reference lines RL1 to RL3 are set at the boundary of the first image area IR1 occupied by the reference workpiece RW in the reference image IB, and a plurality of positions in the reference image IB where the plurality of workpiece reference lines RL1 to RL3 are located are stored in the memory 6 as a plurality of workpiece reference line positions. More specifically, the workpiece mounting support program 6p causes the processor 5 to execute the following processing: in response to an input from the input device 7, the workpiece reference lines RL1 to RL3 desired to be set by the user are displayed in superimposition with the reference image IB, and if an input of setting of the displayed workpiece reference lines RL1 to RL3 is received from the input device 7, a plurality of positions in the reference image IB where the plurality of workpiece reference lines RL1 to RL3 are respectively located are stored in the memory 6 as a plurality of workpiece reference line positions. Alternatively, the workpiece installation support program 6p causes the processor 5 to execute the following processing: the edges of the boundaries of the first image region IR1 found by the image processing are detected as workpiece reference lines RL1 to RL 3. Further, the workpiece mounting support program 6p causes the processor 5 to execute the following processing: when the detected edge is displayed in superimposition with the reference image IB and an input for selecting an edge is received from the input device 7, a plurality of positions in the reference image IB where the plurality of workpiece reference lines RL1 to RL3 relating to the edge are respectively located are stored in the memory 6 as a plurality of workpiece reference line positions.

Similarly, a plurality of first additional workpiece reference lines RL4 to RL5 are set in the first additional reference image IBA at the boundary of the fifth image area IR5 occupied by the reference workpiece RW. A plurality of positions in the first additional reference image IBA where the plurality of first additional workpiece reference lines RL4 to RL5 are respectively located are stored in the memory 6 as a plurality of first additional workpiece reference line positions. Specifically, the workpiece mounting support program 6p causes the processor 5 to execute the following processing: a plurality of first additional workpiece reference lines RL4 to RL5 are set at the boundary of the fifth image area IR5 occupied by the reference workpiece RW in the first additional reference image IBA, and a plurality of positions in the first additional reference image IBA where the plurality of first additional workpiece reference lines RL4 to RL5 are respectively located are stored in the memory 6 as a plurality of first additional workpiece reference line positions. In the second additional reference image IBB, a plurality of second additional workpiece reference lines RL6 to RL7 are set at the boundary of the sixth image area IR6 occupied by the reference workpiece RW. A plurality of positions in the second additional reference image IBB where the plurality of second additional workpiece reference lines RL6 to RL7 are respectively located are stored in the memory 6 as a plurality of second additional workpiece reference line positions. Further, the processor 5 is caused to execute the following processing: a plurality of second additional workpiece reference lines RL6 to RL7 are set at the boundary of the sixth image area IR6 occupied by the reference workpiece RW in the second additional reference image IBB, and a plurality of positions in the second additional reference image IBB where the plurality of second additional workpiece reference lines RL6 to RL7 are respectively located are stored in the memory 6 as a plurality of second additional workpiece reference line positions. Even in these cases, specifically, the workpiece attachment support program 6p causes the processor 5 to execute the same processing as that of the workpiece attachment support program 6p for setting the plurality of workpiece reference lines RL1 to RL3 and storing the set result in the memory 6.

In step S7, the workpiece W having substantially the same shape and size as those of the reference workpiece RW is placed on one of the placing table 2 from which the reference workpiece RW has been removed and the additional placing table 2A, which is different from the placing table 2 and is disposed in place of the placing table 2. As a typical example, the placing table 2 on which the reference workpiece RW is placed is moved to the processing position MP, and while the reference workpiece RW is being processed, the additional placing table 2A on which the workpiece W is placed is moved to the preparation position AP. Alternatively, the placing table 2 on which the reference workpiece RW is placed is moved to the processing position MP, the reference workpiece RW is processed, the processed reference workpiece RW is removed from the placing table 2, and then the placing table 2 on which the workpiece W is placed is moved to the preparation position AP.

In step S8, a measurement image representing the workpiece W captured by the camera 4 set to the target arrangement is acquired. Specifically, the workpiece mounting support program 6p causes the processor 5 to execute the following processing: a measurement image is acquired which is captured by the camera 4 set to the target arrangement and which indicates a workpiece W having a shape and a size substantially the same as those of the reference workpiece RW placed on one of the placing table 2 from which the reference workpiece RW has been removed and the additional placing table 2A different from the placing table 2 and which is arranged in place of the placing table 2. Further, a first additional measurement image representing the workpiece W captured by the first additional camera 4A set to the first additional target arrangement is acquired. Specifically, the workpiece mounting support program 6p causes the processor 5 to execute the following processing: a first additional measurement image representing the workpiece W captured by the first additional camera 4A set to the first additional target arrangement is acquired. A second additional measurement image representing the workpiece W captured by the second additional camera 4B set to the second additional target configuration is acquired. Specifically, the workpiece mounting support program 6p causes the processor 5 to execute the following processing: a second additional measurement image representing the workpiece W captured by the second additional camera 4B set to the second additional target arrangement is acquired.

In step S9, the workpiece attachment support program 6p causes the processor 5 to execute the following processing: the measurement composite image IS generated by superimposing the plurality of workpiece reference lines RL1 to RL3 on the measurement image such that the plurality of workpiece reference lines L1 to RL3 are respectively displayed at a plurality of positions identical to the plurality of workpiece reference line positions in the measurement composite image IS. Further, the workpiece mounting support program 6p causes the processor 5 to execute the following processing: the measurement synthetic image IS generated by superimposing the plurality of camera setting reference lines CRL1 to CRL2 on the measurement image such that the plurality of camera setting reference lines CRL1 to CRL2 are respectively displayed at a plurality of positions identical to the plurality of camera setting reference line positions in the measurement synthetic image IS. The workpiece installation support program 6p causes the processor 5 to execute the following processing: the display 8 IS caused to display the measurement composite image IS. Thus, the processor 5 generates the measurement composite image IS in which the plurality of workpiece reference lines RL1 to RL3 are superimposed on the measurement image such that the plurality of workpiece reference lines RL1 to RL3 are respectively indicated at the plurality of positions identical to the plurality of workpiece reference line positions in the measurement composite image IS. The processor 5 generates the measurement synthetic image IS in which the plurality of camera setting reference lines CRL1 to CRL2 are superimposed on the measurement image such that the plurality of camera setting reference lines CRL1 to CRL2 are respectively indicated at a plurality of positions identical to the plurality of camera setting reference line positions in the measurement synthetic image IS. The display 8 displays the measurement composite image IS.

Similarly, the workpiece mounting support program 6p causes the processor 5 to execute the following processing: the first additional measurement synthetic image ISA in which the plurality of first additional workpiece reference lines RL4 to RL5 are superimposed on the first additional measurement image is generated such that the plurality of first additional workpiece reference lines RL4 to RL5 are respectively indicated at the same plurality of positions as the plurality of first additional workpiece reference line positions in the first additional measurement synthetic image ISA. Further, the workpiece mounting support program 6p causes the processor 5 to execute the following processing: the first additional measurement synthetic image ISA in which the plurality of first additional camera setting reference lines CRL3 to CRL4 are superimposed on the first additional measurement image is generated such that the plurality of first additional camera setting reference lines CRL3 to CRL4 are respectively indicated at the same plurality of positions as the plurality of first additional camera setting reference line positions in the first additional measurement synthetic image ISA. The workpiece installation support program 6p causes the processor 5 to execute the following processing: the display 8 is caused to display the first additional measurement synthetic image ISA. Thus, the processor 5 generates the first additional measurement synthetic image ISA in which the plurality of first additional workpiece reference lines RL4 to RL5 are superimposed on the first additional measurement image in such a manner that the plurality of first additional workpiece reference lines RL4 to RL5 are respectively indicated at the same plurality of positions as the plurality of first additional workpiece reference line positions in the first additional measurement synthetic image ISA. The processor 5 generates the first additional measurement synthetic image ISA in which the plurality of first additional camera setting reference lines CRL3 to CRL4 are superimposed on the first additional measurement image ISA such that the plurality of first additional camera setting reference lines CRL3 to CRL4 are respectively indicated at the same plurality of positions as the plurality of first additional camera setting reference line positions in the first additional measurement synthetic image ISA. The display 8 displays the first additional measurement synthetic image ISA.

Further, the workpiece mounting support program 6p causes the processor 5 to execute the following processing: the second additional measurement synthetic image ISB in which the plurality of second additional workpiece reference lines RL6 to RL7 are superimposed on the second additional measurement image is generated such that the plurality of second additional workpiece reference lines RL6 to RL7 are respectively indicated at the same plurality of positions as the plurality of second additional workpiece reference line positions in the second additional measurement synthetic image ISB. The workpiece mounting support program 6p causes the processor 5 to execute the following processing: the second additional measurement synthetic image ISB in which the plurality of second additional camera setting reference lines CRL5 to CRL6 are superimposed on the second additional measurement image is generated such that the plurality of second additional camera setting reference lines CRL5 to CRL6 are respectively indicated at the same plurality of positions as the plurality of second additional camera setting reference line positions in the second additional measurement synthetic image ISB. The workpiece installation support program 6p causes the processor 5 to execute the following processing: the second additional measurement composite image ISB is displayed on the display 8. Thus, the processor 5 generates the second additional measurement synthetic image ISB in which the plurality of second additional workpiece reference lines RL6 to RL7 are superimposed on the second additional measurement image such that the plurality of second additional workpiece reference lines RL6 to RL7 are respectively indicated at the same plurality of positions as the plurality of second additional workpiece reference line positions in the second additional measurement synthetic image ISB. The processor 5 generates the second additional measurement synthetic image ISB in which the plurality of second additional camera setting reference lines CRL5 to CRL6 are superimposed on the second additional measurement image such that the plurality of second additional camera setting reference lines CRL5 to CRL6 are respectively indicated at the same plurality of positions as the plurality of second additional camera setting reference line positions in the second additional measurement synthetic image ISB. The display 8 displays the second additional measurement combined image ISB.

In step S10, the user confirms whether the camera 4 IS set to the target configuration according to whether the boundary of the fourth image area IS4 occupied by the stationary object in the measurement composite image IS displayed on the display 8 coincides with the plurality of camera setting reference lines CRL1 to CRL 2. The user confirms whether or not the first additional camera 4A is set to the first additional target arrangement based on whether or not the boundary of the image area BG3 occupied by the stationary object in the first additional measurement synthetic image ISA displayed on the display 8 overlaps with the plurality of first additional camera setting reference lines CRL3 to CRL 4. The user confirms whether or not the second additional camera 4B is set to the second additional target arrangement based on whether or not the boundaries of the image area BG4 occupied by the stationary object in the second additional measurement combined image ISB displayed on the display 8 overlap the plurality of second additional camera setting reference lines CRL5 to CRL 6. If the camera 4 IS not set to the target arrangement (no in step S10), in step S11, the user adjusts the viewpoint position and the line-of-sight direction of the camera 4 so that the boundary of the fourth image area IS4 occupied by the stationary object coincides with the plurality of camera setting reference lines CRL1 to CRL2 in the measurement composite image IS. By repeating the processing of steps S8 to S11, the camera 4 can be set to the target arrangement. Similarly, when the first additional camera 4A is not set to the first additional target arrangement (no in step S10), in step S11, the user adjusts the viewpoint position and the line-of-sight direction of the first additional camera 4A so that the boundary of the image area BG3 occupied by the stationary object overlaps the plurality of first additional camera setting reference lines CRL3 to CRL4 in the first additional measurement synthetic image ISA. By repeating the processing of steps S8 to S11, the first additional camera 4A can be set to the first additional target arrangement. If the second additional camera 4B is not set to the second additional target arrangement (no in step S10), in step S11, the user adjusts the viewpoint position and the line-of-sight direction of the second additional camera 4B so that the boundaries of the image area BG4 occupied by the still object in the second additional measurement synthetic image ISB overlap the plurality of second additional camera setting reference lines CRL5 to CRL 6. By repeating the processing of steps S8 to S11, the second additional camera 4B can be set to the second additional target arrangement.

When the camera 4 IS set to the target arrangement, the first additional camera 4A IS set to the first additional target arrangement, and the second additional camera 4B IS set to the second additional target arrangement (yes in step S10), the user confirms in step S12 whether the boundary of the second image area IR2 occupied by the workpiece W in the measurement composite image IS substantially parallel to or substantially coincides with the plurality of workpiece reference lines RL1 to RL 3. If the boundaries of the second image area IR2 do not coincide with and are not parallel to the plurality of workpiece reference lines RL1 to RL3 (no in step S12), in step S13, the user adjusts the posture of the workpiece W on the one of the stages so that the boundaries of the second image area IR2 occupied by the workpiece W in the measurement composite image IS are substantially parallel to or substantially coincide with the plurality of workpiece reference lines RL1 to RL 3. By repeating the processing of steps S8 to S10 and steps S12 and S13, the boundary of the second image area IR2 occupied by the workpiece W in the measurement composite image IS substantially parallel to or substantially coincides with the plurality of workpiece reference lines RL1 to RL 3. Similarly, in step S12, the user confirms whether the boundary of the seventh image region IR7 occupied by the workpiece W in the first additional measurement synthetic image ISA is substantially parallel to or substantially coincides with the plurality of first additional workpiece reference lines RL4 to RL 5. If the boundary of the seventh image area IR7 does not coincide with and is not parallel to the plurality of first additional workpiece reference lines RL4 to RL5 (no in step S12), in step S13, the user adjusts the posture of the workpiece W on the one of the above-described stages so that the boundary of the seventh image area IR7 occupied by the workpiece W in the first additional measurement synthetic image ISA becomes substantially parallel to or substantially coincides with the plurality of first additional workpiece reference lines RL4 to RL 5. By repeating the processing of steps S8 to S10 and steps S12 and S13, the boundary of the seventh image area IR7 occupied by the workpiece W in the first additional measurement synthetic image ISA is substantially parallel to or substantially coincides with the plurality of first additional workpiece reference lines RL4 to RL 5. Further, in step S12, the user confirms whether the boundary of the eighth image area IR8 occupied by the workpiece W in the second additional measurement combined image ISB is substantially parallel to or substantially coincides with the plurality of second additional workpiece reference lines RL6 to RL 7. If the boundary of the eighth image area IR8 does not coincide with and is not parallel to the plurality of second additional workpiece reference lines RL6 to RL7 (no in step S12), in step S13, the user adjusts the posture of the workpiece W on the one of the stages so that the boundary of the eighth image area IR8 occupied by the workpiece W in the second additional measurement composite image ISB is substantially parallel to or substantially coincides with the plurality of second additional workpiece reference lines RL6 to RL 7. By repeating the processing of steps S8 to S10 and steps S12 and S13, the boundary of the eighth image area IR8 occupied by the workpiece W in the second additional measurement combined image ISB becomes substantially parallel to or substantially coincides with the plurality of second additional workpiece reference lines RL6 to RL 7.

If the adjustment of the posture of the workpiece W is finished (yes in step S12), in step S14, the stage on which the workpiece W whose posture is adjusted is placed is moved to the machining position MP. In step S15, the position measuring sensor 9 of the machining device 10 measures the positions of the machining reference points CP1 to CPn, which are the plurality of feature points of the workpiece W having substantially the same shape and corresponding to the plurality of reference feature points BP1 to BPn, respectively. Although the machining reference points CP1 to CPn are shown in fig. 16, as shown in fig. 15 and 16, a plurality of machining reference points CP1 to CPn correspond to a plurality of reference feature points BP1 to BPn, respectively. The machining reference points CP1 to CPn are edges and corners whose positions can be easily recognized by the touch probe, and the position measurement sensor 9 can automatically search for the positions of the machining reference points CP1 to CPn in consideration of the fact that there is no large deviation in position. Then, the numerical control device 100 (electronic circuit 110) determines whether or not the positions of the plurality of machining reference points CP1 to CPn are within the allowable ranges. This determination method is the same as the determination method of determining whether or not the reference feature points BP1 to BPn are within the allowable range.

When the electronic circuit 110 determines that all the positions of the plurality of machining reference points CP1 to CPn are within the allowable range (yes in step S15), the machining device 10 outputs the determination result of the machining permission in step S17, and machines the workpiece W. When it is determined that the position of at least one of the plurality of machining reference points CP1 to CPn is not within the allowable range (no in step S15), in step S16, the machining device 10 outputs the determination result that machining is not permitted, and readjusts the machining position without returning the one of the tables on which the workpiece W is placed, which has been moved to the machining position MP, to the preparation position AP or to the preparation position.

< modification of the first embodiment >

In the above-described method, the user adjusts the posture of the workpiece W while visually checking the measurement composite image IS so that the boundary of the second image area IR2 occupied by the workpiece W in the measurement composite image IS becomes substantially parallel to or substantially coincides with the plurality of workpiece reference lines RL1 to RL3, and therefore it IS difficult to accurately perform the adjustment. In particular, it is not easy to accurately determine whether or not the boundary of the second image area IR2 is actually parallel to the plurality of workpiece reference lines RL1 to RL 3. Therefore, the determination can be made by machine learning.

Fig. 17 is a block diagram of a workpiece mounting support system 1a according to a modification of the first embodiment. In the present modification, the workpiece mounting support program 6p includes a machine learning program 6m, and the memory 6 further stores measurement image data DA1, image processing data DA2, reference line data DA3, camera parameters DA4, and completed learning parameters DA5, which are utilized by the machine learning program 6 m. The image processing apparatus 200a and the processing apparatus 10a have a second communication interface 204 and a communication interface 112, respectively, which are capable of communicating with each other. The second communication interface 204 and the communication interface 112 may be any interfaces as long as they are interfaces that can communicate with each other, such as an ethernet (registered trademark) interface and a wireless communication interface. The machining device 10a can transmit the determination result (the integrated determination result DA6) of determining whether or not all the positions of the plurality of machining reference points CP1 to CPn are within the allowable range, the result of determining whether or not the plurality of machining reference points CP1 to CPn are within the allowable range (the determination result DA7 for each machining reference point), and the deviation amounts of the plurality of machining reference points CP1 to CPn from the center values of the respective allowable ranges (the determination result DA7 for each machining reference point) to the image processing device 200a through the second communication interface 204 and the communication interface 112. The memory 6 may further store the integrated determination result DA6, the determination result DA7 distinguished by the machining reference point, and the deviation amount DA8 distinguished by the machining reference point. The machine learning program 6m, the measured image data DA1, the image processing data DA2, the reference line data DA3, the camera parameters DA4, the completed learning parameters DA5, the integrated determination result DA6, the determination result DA7 for each machining reference point, and the deviation amount DA8 for each machining reference point may be stored in an external server such as a cloud service, instead of being stored in the image processing apparatus 200a, and the machine learning program 6m may be executed in the external server, and only the execution result may be returned to the image processing apparatus 200 a.

The measurement image data DA1 is at least one of the measurement image captured by the camera 4, the first additional measurement image captured by the first additional camera 4A, and the second additional measurement image captured by the second additional camera 4B. The image processing data DA2 is at least one of a binary image obtained by performing edge detection processing on each of the measurement image, the first additional measurement image, and the second additional measurement image, and a binary image obtained by detecting the workpiece W by background difference or the like. The datum line data DA3 is at least one of the positions of the workpiece datum lines of the plurality of workpiece datum lines RL1 to RL3, the positions of the first additional workpiece datum lines of the plurality of first additional workpiece datum lines RL4 to RL5, and the positions of the second additional workpiece datum lines of the plurality of second additional workpiece datum lines RL6 to RL 7. The camera parameters DA4 include the respective image center positions and distortion parameters of the measurement image, the first additional measurement image, and the second additional measurement image; and the respective focal distances of the camera 4, the first additional camera 4A, and the second additional camera 4B. The integrated determination result DA6 is the determination result of the above-described step S15. The determination result DA7 for each machining reference point is a result of determining whether or not the absolute value of the difference between the position measured by the position measurement sensor 9 and the ideal position of the plurality of machining reference points CP1 to CPn obtained from the workpiece model is equal to or less than a threshold value for each of the plurality of machining reference points CP1 to CPn. The deviation amount DA8 for each machining reference point is a difference between the position measured by the position measuring sensor 9 of each of the plurality of machining reference points CP1 to CPn and the ideal position of the plurality of machining reference points CP1 to CPn obtained from the workpiece model.

The machine learning program 6m causes the processor 5 to execute the following processing: for example, a neural network, more preferably three or more layers of neural networks for deep learning, is used as a learning model, and the machine learning model is learned using training data having the measurement image data DA1 and the reference line data DA3 as inputs and at least one of the comprehensive determination result DA6, the determination result DA7 for each machining reference point, and the deviation amount DA8 for each machining reference point as an output. At least one of the image processing data DA2 and the camera parameters DA4 may be further input to learn the machine learning model. The completed learning parameter DA5 stores data such as the weight parameter between neurons of each layer of the neural network thus learned. Therefore, the workpiece attachment supporting program 6p uses a machine learning model that learns, as output training data, at least one of a determination result that takes the measurement image and the plurality of workpiece reference lines RL1 to RL3 as input and determines whether or not all the positions of the plurality of machining reference points CP1 to CPn are within the allowable range, a result that determines whether or not the plurality of machining reference points CP1 to CPn are within the allowable range, respectively, and an amount of deviation between the plurality of machining reference points CP1 to CPn and a center value of each of the allowable ranges (an ideal position of the plurality of machining reference points CP1 to CPn obtained from the workpiece model). That is, the processor 5 is configured to use a machine learning model that learns training data using, as inputs, the measurement image and the plurality of workpiece reference lines RL1 to RL3 and outputs at least one of the determination results, the result of determining whether or not each of the plurality of machining reference points CP1 to CPn is present within the allowable range, and the amounts of deviation of the plurality of machining reference points CP1 to CPn from the center values of the respective allowable ranges. The workpiece mounting method uses a machine learning model that uses training data that uses a measurement image and a plurality of workpiece reference lines RL1 to RL3 as inputs and that learns at least one of a determination result, a result of determining whether each of a plurality of machining reference points CP1 to CPn is present within an allowable range, and a deviation amount of a center value of each of the plurality of machining reference points CP1 to CPn from the allowable range as an output. The machine learning model is learned using training data that also takes as input at least one of edges detected from the measurement image, amounts of deviation from the image center of the plurality of workpiece reference line positions, focal length of a lens of the camera, and distortion parameters of the camera.

The machine learning program 6m causes the processor 5 to further execute the following processing: using the machine learning model having completed the learning in this way, at least one of the image of the camera 4 (third additional measurement image), the image of the first additional camera 4A (fifth additional measurement image), and the image of the second additional camera 4B (sixth additional measurement image) which have been captured of the first additional workpiece AW having the shape and size substantially the same as those of the reference workpiece RW, and the reference line data DA3 are input, and it is determined whether or not all the positions of the plurality of first additional processing reference points, which are the plurality of feature points of the first additional workpiece AW having the substantially same shape and correspond to the plurality of reference feature points BP1 to BPn when one of the tables on which the first additional workpiece AW is placed is moved to the processing position MP, are within the allowable range. At least one of a binary image obtained by performing edge detection processing on the third additional measurement image, the fifth additional measurement image, and the sixth additional measurement image, a binary image obtained by detecting the first additional workpiece AW by a background difference, and the camera parameter DA4 may be input to the machine learning program 6 m. Fig. 18 shows an example of the composite image ISC as a result of determination of the third additional measurement image. In the composite image ISC, the plurality of workpiece reference lines RL1 to RL3, the plurality of camera setting reference lines CRL1 to CRL2, and the determination result display window DW are superimposed on the third additional measurement image. Therefore, the workpiece mounting support program 6p causes the processor 5 to execute the following processing: based on the third additional measurement image showing the first additional workpiece AW having the shape and size substantially the same as those of the reference workpiece RW captured by the camera 4 set to the target arrangement, it is determined whether or not all of the positions of the plurality of first additional machining reference points, which are the plurality of feature points of the first additional workpiece AW having the substantially same shape and correspond to the plurality of reference feature points BP1 to BPn when one of the tables on which the first additional workpiece AW is placed moves to the machining position MP, are within the allowable range. That is, the processor 5 is configured to determine whether or not all of the positions of the plurality of first additional machining reference points, which are the plurality of feature points of the first additional workpiece AW having the substantially same shape and the substantially same shape as the plurality of reference feature points BP1 to BPn, respectively, exist within the allowable range when one of the tables on which the first additional workpiece AW is placed moves to the machining position MP, based on the third additional measurement image showing the first additional workpiece AW having the substantially same shape and the substantially same size as the reference workpiece RW captured by the camera 4 set to the target arrangement. The workpiece mounting method determines whether or not all of the positions of a plurality of first additional machining reference points, which are a plurality of feature points of the first additional workpiece AW having the substantially same shape and the substantially same size as the reference feature points BP1 to BPn when one of the tables on which the first additional workpiece AW is placed moves to the machining position MP, are within an allowable range, based on a third additional measurement image showing the first additional workpiece AW having the substantially same shape and size as the reference workpiece RW captured by the camera 4 set to the target arrangement. In this case, the machine learning program 6m may be executed in step S12 described above.

The workpiece W may be placed on both the placement base 2 and the additional placement base 2A. Therefore, in the above-described workpiece mounting method, the second additional workpiece BW having the shape and size substantially the same as those of the reference workpiece RW is placed on the other of the placing table 2 from which the reference workpiece RW has been removed and the additional placing table 2A disposed in place of the placing table 2, the fourth additional measurement image representing the second additional workpiece BW captured by the camera 4 disposed in the target arrangement is acquired, the fourth additional measurement composite image ISD in which the plurality of workpiece reference lines and the fourth additional measurement image are superimposed is generated by the processor 5, and the plurality of workpiece reference lines RL1 to RL3 are respectively displayed at the plurality of positions in the fourth additional measurement composite image ISD, which are the same as the positions of the plurality of workpiece reference lines. The user can adjust the posture of the second additional workpiece on the other table so that the boundary of the ninth image area IR9 occupied by the second additional workpiece BW and the plurality of workpiece reference lines RL1 to RL3 become substantially parallel or substantially coincide with each other in the fourth additional measurement composite image ISD. Fig. 19 shows an example of the fourth additional measurement composite image ISD.

In an environment where the positions and postures of the camera 4, the first additional camera 4A, and the second additional camera 4B are not changed, in the first embodiment, the setting of the plurality of camera setting reference lines CRL1 to CRL2 and the representation of the plurality of camera setting reference lines CRL1 to CRL2 in the measurement composite image IS may be omitted. Similarly, the setting of the plurality of first additional camera setting reference lines CRL3 to CRL4 and the representation of the plurality of first additional camera setting reference lines CRL3 to CRL4 in the first additional measurement synthetic image ISA may be omitted. The setting of the plurality of second additional camera setting reference lines CRL5 to CRL6 and the representation of the plurality of second additional camera setting reference lines CRL5 to CRL6 in the second additional measurement combined image ISB may be omitted. In this case, steps S3, S5, S10, S11 may be omitted in fig. 10.

The workpiece mounting support program 6p and the machine learning program 6m may be stored not only in the memory 6 incorporated in the

image processing apparatuses

200 and 200a but also in a storage medium that can be removed from the

image processing apparatuses

200 and 200a and read by the

image processing apparatuses

200 and 200a, such as a flexible disk, an optical disk, a cd rom, a magnetic disk, or the like, an SD card, a USB memory, or an external hard disk.

< action and Effect of the first embodiment >

The workpiece mounting

support systems

1 and 1a and the workpiece mounting method using the workpiece mounting

support systems

1 and 1a according to the first embodiment can adjust the posture of the workpiece W with reference to the plurality of reference line positions of the reference workpiece RW adjusted so that the posture thereof becomes the workpiece reference posture. Therefore, the posture of the workpiece can be efficiently adjusted.

< second embodiment >

The workpiece mounting method is not limited to the machining device 10 described above, and may be applied to other machining devices. Fig. 20 is a diagram showing a schematic configuration of a work mounting support system 11 according to a second embodiment. In the second embodiment, the machining device 10m is a machining device capable of performing milling and turning together. The placing table 2m fixes the reference workpiece RW and the workpiece W by the locking claws 21 to 24. The placing table 2m has a rotation center axis Ax and is rotatable around the rotation center axis Ax. In fig. 20, the machining position MP and the preparation position AP are shown as the same example, but the placing table 2m may be moved to the machining position MP where the reference workpiece RW and the workpiece W are machined and the preparation position AP apart from the machining position MP as in the first embodiment. The locking claws 21 to 24 are sequentially arranged clockwise at positions rotated by 90 degrees around the rotation center axis Ax. That is, when viewed in the axial direction of the rotation center axis Ax, an angle formed between a direction from the rotation center axis Ax toward the locking claw 21 and a direction from the rotation center axis Ax toward the locking claw 22 is 90 degrees. An angle formed between a direction from the rotation center axis Ax toward the locking claw 22 and a direction from the rotation center axis Ax toward the locking claw 23 is 90 degrees as viewed in the axial direction of the rotation center axis Ax. An angle formed between a direction from the rotation center axis Ax toward the locking claw 23 and a direction from the rotation center axis Ax toward the locking claw 24 is 90 degrees as viewed in the axial direction of the rotation center axis Ax. An angle formed between a direction from the rotation center axis Ax toward the locking claw 24 and a direction from the rotation center axis Ax toward the locking claw 21 is 90 degrees as viewed in the axial direction of the rotation center axis Ax. The optical axis of the camera 4 is arranged to face substantially parallel to the rotation center axis Ax. In the work mounting support system 11, the first additional camera 4A and the second additional camera 4B are omitted from the configuration of the work mounting support system 1. However, the processing apparatus 10m may include an additional placement table 2mA having the same shape and function as the placement table 2m, as in the first embodiment. Although not shown in fig. 20, the processing device 10m may have the position measurement sensor 9 shown in the first embodiment, and the image processing device 200 may have the function of the machine learning program 6 m. In the description of the second embodiment, the same components and processes as those of the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted. The configuration that is not described in the present embodiment is substantially the same as that of the first embodiment.

Fig. 21 and 22 are views for explaining the postures of the reference workpiece RW and the workpiece W according to the present embodiment. As shown in fig. 21, the reference workpiece RW and the workpiece W of the present embodiment have a central axis Cx and have shapes substantially line-symmetrical with respect to the central axis Cx. In the posture adjustment of the present embodiment, it is necessary to match both the center axes Cx of the reference workpiece RW and the workpiece W with the rotation center axis Ax as shown in fig. 21, and then match the phases (rotation angles around the rotation center axis Ax) of the reference workpiece RW and the workpiece W with a predetermined phase as shown in fig. 22. Therefore, the postures of the reference workpiece RW and the workpiece W are defined by the rotation center axis Ax of the placing table 2.

Fig. 23 is a flowchart illustrating a method of mounting a workpiece W according to a second embodiment. In this method, in step S1a, the user places the reference workpiece RW on the placing table 2, and adjusts the posture of the reference workpiece RW on the placing table 2 so that the posture of the reference workpiece RW becomes the workpiece reference posture. Fig. 24 is a flowchart of a specific process of step S1 a. First, in step S121, as shown in fig. 25, the long rod 30 is attached to the centering bar (an example of the position measuring sensor 9). The central axis of the elongated rod 30 is adjusted to be parallel to the rotation central axis Ax. The radius of the long bar 30 is the same with respect to the axial direction of the central axis of the long bar 30. Next, in step S122, the placing table 2 is turned to an angle of 0 degrees. This angle 0 degree defines a rotation angle at which the long bar 30 is positioned between the rotation center axis Ax and the locking claw 21 when viewed from the axial direction of the rotation center axis Ax. In step S123, the gap between the bar 30 and the first measuring surface WP1 is measured visually at an angle of 0 degrees. Here, the first measurement surface WP1 is a surface that is parallel to the rotation center axis Ax when the reference workpiece RW is posture-adjusted as shown in fig. 22 on the surface of the reference workpiece RW. Therefore, the posture of the reference workpiece RW can be adjusted so that the distance between the long rod 30 and the first measuring surface WP1 becomes uniform in the axial direction of the center axis of the long rod 30.

Next, in step S124, the placing table 2 is turned to an angle of 180 degrees. At this time, as shown in fig. 26, the reference workpiece RW and the placing table 2 are rotated about the rotation center axis Ax so that the long rod 30 is positioned between the rotation center axis Ax and the locking claw 23. However, the long bar 30 maintains the position and posture shown in fig. 25. At this time, the second measurement plane WP2, which is a plane opposite to the first measurement plane WP1 with respect to the rotation center axis Ax, faces the long rod 30. In step S125, the gap between the bar 30 and the second measuring plane WP2 is measured visually at an angle of 180 degrees. At this time, the second measurement surface WP2 is substantially parallel to the rotation center axis Ax. In step S126, it is determined whether the gap distances between the angle 0 degree and 180 degrees are equal. When the gap distances between the angle 0 degree and the angle 180 degree are not equal (no in step S126), since the center shift occurs as shown in fig. 21, the locking

claws

21 and 23 are adjusted so that the reference workpiece RW slides in the direction perpendicular to the rotation center axis Ax in step S127. Thereafter, until the gap distances at the angles 0 degree and 180 degree become equal (yes in step S126), the processing of step S127 and steps S122 to S126 is repeated.

If the distances of the gaps of the angle 0 degree and 180 degrees become equal (yes in step S126), the placing table 2 is turned to the angle 90 degrees in step S128. At this time, as shown in fig. 27, the reference workpiece RW and the placing table 2 are rotated about the rotation center axis Ax so that the long rod 30 is positioned between the rotation center axis Ax and the locking claw 22. However, the long bar 30 maintains the position and posture shown in fig. 25. In step S129, the gap between the bar 30 and the third measuring surface WP3 is measured visually at an angle of 90 degrees. The third measurement surface is a surface that is parallel to the rotation center axis Ax when the reference workpiece RW is attitude-adjusted as shown in fig. 22, among the surfaces of the reference workpiece RW, basically the same as the first measurement surface. Therefore, the posture of the reference workpiece RW can be adjusted so that the distance between the long rod 30 and the third measuring surface WP3 becomes uniform in the axial direction of the center axis of the long rod 30.

Next, in step S130, the placing table 2 is turned to an angle of 270 degrees. At this time, as shown in fig. 28, the reference workpiece RW and the placing table 2 are rotated about the rotation center axis Ax so that the long rod 30 is positioned between the rotation center axis Ax and the locking claw 24. However, the long bar 30 maintains the position and posture shown in fig. 25. At this time, the fourth measuring surface WP4, which is a surface opposite to the third measuring surface WP3 with respect to the rotation center axis Ax, faces the bar 30. In step S131, the gap between the bar 30 and the fourth measuring plane WP4 is measured visually at an angle of 270 degrees. At this time, the fourth measurement surface WP4 is substantially parallel to the rotation center axis Ax. In step S132, it is determined whether the distance between the gaps of the angle 90 degrees and 270 degrees is equal. When the gap distances between the angle 90 degrees and the angle 270 degrees are not equal (no in step S132), the center shift shown in fig. 21 occurs, and therefore, in step S133, the locking

claws

22 and 24 are operated so that the reference workpiece RW slides in the direction perpendicular to the rotation center axis Ax. Thereafter, until the gap distances at the angles of 90 degrees and 270 degrees become equal (yes in step S132), the processing of step S133 and steps S128 to S132 is repeated.

Returning to fig. 23, steps S2 to S5 perform the same processing as in the first embodiment, and the workpiece reference line is set in step S6 a. Fig. 29 is an example of a reference image IB according to the second embodiment. In step S6a, the user sets a plurality of workpiece reference lines RL1 to RL4 in the reference image IB by the input device 7 at the boundary of the first image area IR1 occupied by the reference workpiece RW while viewing the reference image IB as shown in fig. 29 displayed on the display 8. In addition, in this setting, it is preferable to set the work reference lines RL1 and RL4 which are not affected by the phase deviation and the work reference lines RL2 and RL3 which are affected by the phase deviation together. Thereafter, steps S7 to S11 are performed in the same manner as in the first embodiment, and the posture of the workpiece W is adjusted in steps S12a and S13 a.

Fig. 30 shows an example of the measurement composite image IS of the second embodiment. In fig. 30, the second image area IR2 occupied by the workpiece W IS represented by a dot pattern in the measurement composite image IS. In addition, the boundary of the fourth image region IR4 occupied by the stationary object corresponding to the third image region IR3 IS hatched in the measurement composite image IS. In the first embodiment, a case IS permissible in which the boundary of the second image region IR2 occupied by the workpiece W in the measurement composite image IS and the plurality of workpiece reference lines RL1 to RL3 become substantially parallel, but in the second embodiment, the user adjusts the posture of the workpiece W on the placing table 2 so that the boundary of the second image region IR2 occupied by the workpiece W and the plurality of workpiece reference lines RL1 to RL4 become substantially coincident in the measurement composite image IS. In fig. 30, since the center shift shown in fig. 21 and the phase shift shown in fig. 22 occur together, first, the attitude of the workpiece W is adjusted by operating the locking claws 21 to 24 so that the boundary lines BL1 and BL2 of the workpiece reference lines RL1 and RL4 and the corresponding second image region IR2, which are not affected by the phase shift, substantially coincide with each other. Fig. 31 shows a representative example of the measurement composite image IS subjected to such adjustment. Next, the workpiece W and the placing table 2 are rotated about the rotation center axis Ax so that the boundary lines BL1 and BL2 between the remaining workpiece reference lines RL2 and RL3 and the corresponding second image region IR2 substantially coincide with each other. Fig. 32 shows an example of a measurement composite image IS representing the workpiece W whose posture IS adjusted in this way. Fig. 29 to 31 show the case where all of the plurality of workpiece reference lines RL1 to RL4 are circles, but linear workpiece reference lines may be included.

Returning to fig. 23, in step S12a, it IS determined whether all of the plurality of workpiece reference lines RL1 to RL4 in the measurement composite image IS actually coincide with the boundary of the second image area IR2 occupied by the workpiece W. When the boundaries between all of the plurality of workpiece reference lines RL1 to RL4 and the second image area IR2 do not substantially match (no in step S12 a), in step S13a, the engagement claws 21 to 24 are operated or the workpiece W and the placement base 2 are rotated about the rotation center axis Ax. Thereafter, until the boundaries between all of the plurality of workpiece reference lines RL1 to RL4 and the second image area IR2 substantially match, the processing of step S13a and steps S8 to S13a is repeated. If all of the plurality of workpiece reference lines RL1 to RL4 actually match the boundaries of the second image area IR2 (yes in step S12 a), the processing in steps S15 to S17 is executed.

< action and Effect of the second embodiment >

The workpiece mounting support system 11 and the workpiece mounting method using the workpiece mounting support system 11 according to the second embodiment can reduce the mounting time by using the composite image for the troublesome mounting of the workpiece W such as mounting the workpiece W on the mounting table of the four grippers.

In the present invention, "include" and its derivatives are non-limiting terms that describe the presence of a component, and do not exclude the presence of other components not described. This also applies to "having," "including," and their derivatives.

Words such as "part", "element", "body", and "structure" may have a plurality of meanings such as a single part or a plurality of parts.

The ordinal numbers such as "first" and "second" are used merely to identify the components, and do not have other meanings (for example, a certain order). For example, the presence of the "first element" does not imply the presence of the "second element", and the presence of the "second element" does not imply the presence of the "first element".

Unless specifically stated otherwise in the embodiments, terms such as "substantially", "about" and "approximately" indicating a degree may mean a reasonable amount of deviation without significant change in the end result. All numerical values recited in the present invention can be interpreted as including terms such as "actually", "about" and "approximately".

In the present invention, the phrase "at least one of a and B" should be interpreted to include only a, only B, and both a and B.

It is apparent that various modifications and variations of the present invention are possible in light of the above disclosure. Therefore, the present invention can be carried out by a method different from the specific disclosure of the present invention without departing from the scope of the present invention.

Claims

1. A method of mounting a workpiece for a machining apparatus, comprising:

the reference workpiece is placed on the placing table,

adjusting the attitude of the reference workpiece on the placing table so that the attitude of the reference workpiece becomes a workpiece reference attitude,

setting a camera to a target arrangement in which a viewpoint position and a visual line direction of the camera for photographing the placing table are a reference viewpoint position and a reference visual line direction, respectively,

acquiring a reference image representing the reference workpiece whose posture is adjusted, the reference image being captured by the camera set to the target arrangement,

so that a plurality of workpiece reference lines are set at the boundary of a first image region occupied by the reference workpiece in the reference image,

storing a plurality of positions in the reference image where the plurality of workpiece reference lines are respectively located in a memory as a plurality of workpiece reference line positions,

placing a workpiece having substantially the same shape and size as those of the reference workpiece on one of the placing table from which the reference workpiece has been removed and an additional placing table different from the placing table and disposed in place of the placing table,

acquiring a measurement image representing the workpiece captured by the camera set to the target configuration,

generating, by a processor, a measurement composite image in which the plurality of workpiece reference lines are superimposed on the measurement image, the plurality of workpiece reference lines being respectively represented in the measurement composite image at a plurality of positions that are the same as the plurality of workpiece reference lines,

the attitude of the workpiece is adjusted on the one table so that the boundary of the second image region occupied by the workpiece and the plurality of workpiece reference lines are substantially parallel or substantially coincident in the measurement composite image.

2. The method of mounting a workpiece according to claim 1,

the placing table and the additional placing table are movable to a processing position where the workpiece is processed by the processing device and a preparation position away from the processing position,

the camera shoots any one of the reference workpiece and the workpiece placed on the one table moved to the preparation position,

adjusting the attitude of the workpiece on the placing table or the additional placing table moved to the preparation position.

3. The method of mounting a workpiece according to claim 2,

when the placing table on which the reference workpiece whose posture is adjusted so as to be the reference posture of the workpiece is placed is moved to the machining position, all of the positions of a plurality of reference feature points, which are a plurality of feature points of the shape of the reference workpiece, are within a predetermined allowable range.

4. The method of mounting a workpiece according to claim 3, further comprising:

moving the one side table on which the workpiece whose posture is adjusted is placed to the machining position,

measuring, by a position measuring sensor of the machining device, positions of a plurality of machining reference points, which are a plurality of feature points of the workpiece having the substantially same shape and corresponding to the plurality of reference feature points,

the machining device determines whether or not the positions of the plurality of machining reference points are within the allowable range.

5. The method of mounting a workpiece according to claim 4, further comprising:

outputting, by the processing device, a determination result of permission of processing and processing the workpiece when it is determined that all the positions of the plurality of processing reference points are within the allowable range,

when it is determined that the position of at least one of the plurality of machining reference points does not exist within the allowable range, a determination result that machining is not permitted is output, and the one of the tables on which the workpiece is placed, which has been moved to the machining position, is moved to the preparation position.

6. The method of mounting a workpiece according to any one of claims 1 to 5,

the attitude of the reference workpiece and the attitude of the workpiece are defined by a rotation angle about a coordinate axis of a coordinate system set for causing the machining device to execute a machining program or a rotation center axis of the placing table.

7. The method of mounting a workpiece according to any one of claims 1 to 6,

the workpiece reference line is an edge of a boundary of the first image region obtained by the image processing performed by the processor.

8. The method of mounting a workpiece according to any one of claims 1 to 6,

the workpiece reference line is a straight line or a circle.

9. The method of mounting a workpiece according to any one of claims 1 to 8,

further comprising setting a plurality of camera setting reference lines at a boundary of a third image area occupied by a stationary object in a background of the reference image,

further storing a plurality of positions within the reference image where the plurality of camera setting reference lines are respectively located in the memory as a plurality of camera setting reference line positions,

generating, by the processor, the measurement composite image in which the plurality of camera setting reference lines are superimposed on the measurement image such that the plurality of camera setting reference lines are respectively represented at a plurality of positions that are the same as the plurality of camera setting reference line positions in the measurement composite image,

setting the cameras to the target configuration by adjusting the viewpoint positions and the sight-line directions of the cameras in such a manner that boundaries of a fourth image area occupied by the stationary objects in the measurement composite image coincide with the plurality of camera setting reference lines.

10. The method of mounting a workpiece as recited in claims 1 to 9, further comprising:

a first additional camera is set to a first additional target arrangement in which a viewpoint position and a visual line direction of the first additional camera for imaging the placement base are set to a first additional reference position different from the reference viewpoint position and a first additional reference visual line direction non-parallel to the reference visual line direction, respectively,

setting a second additional camera such that a viewpoint position and a visual line direction of the second additional camera, which photographs the placing table, are respectively a second additional reference position different from the reference viewpoint position and the first additional reference position, and a second additional reference visual line direction which is not parallel to the reference visual line direction and the first additional reference visual line direction,

acquiring a first additional reference image representing the reference workpiece whose posture is adjusted, which is captured by the first additional camera set to the first additional target arrangement,

acquiring a second additional reference image representing the reference workpiece whose posture is adjusted, which is captured by the second additional camera set to the second additional target arrangement,

setting a plurality of first additional workpiece reference lines at a boundary of a fifth image region occupied by the reference workpiece in the first additional reference image,

setting a plurality of second additional workpiece reference lines at a boundary of a sixth image region occupied by the reference workpiece in the second additional reference image,

storing a plurality of positions in the first additional reference image where the plurality of first additional workpiece reference lines are respectively located in the memory as a plurality of first additional workpiece reference line positions,

storing a plurality of positions in the second additional reference image at which the plurality of second additional workpiece reference lines are respectively located in the memory as a plurality of second additional workpiece reference line positions,

acquiring a first additional measurement image representing the workpiece captured by the first additional camera set to the first additional target configuration,

acquiring a second additional measurement image representing the workpiece captured by the second additional camera set to the second additional target configuration,

generating, by the processor, a first additional measurement composite image in which the plurality of first additional workpiece reference lines and the first additional measurement image are superimposed such that the plurality of first additional workpiece reference lines are respectively represented at a plurality of positions in the first additional measurement composite image that are the same as the plurality of first additional workpiece reference line positions,

generating, by the processor, a second additional measurement composite image in which the plurality of second additional workpiece reference lines and the second additional measurement image are superimposed such that the plurality of second additional workpiece reference lines are respectively represented at a plurality of positions in the second additional measurement composite image that are the same as the plurality of second additional workpiece reference line positions,

adjusting the attitude of the workpiece on the one of the stages so that a boundary of a seventh image region occupied by the workpiece in the first additional measurement composite image becomes substantially parallel to or substantially coincident with the plurality of first additional workpiece reference lines,

the attitude of the workpiece is adjusted on the one table so that the boundary of the eighth image region occupied by the workpiece in the second additional measurement composite image is substantially parallel to or substantially coincides with the plurality of second additional workpiece reference lines.

11. The method of mounting a workpiece according to claim 5, further comprising: determining, using a machine learning model learned using training data, whether or not positions of all of the plurality of first additional machining reference points, which are a plurality of feature points of the first additional workpiece corresponding to the plurality of reference feature points respectively when the one table on which the first additional workpiece is placed is moved to the machining position, are within the allowable range based on a third additional measurement image representing a first additional workpiece having a shape and a size substantially identical to those of the reference workpiece captured by the camera set to the target arrangement, the training data having the measurement image and the plurality of workpiece reference lines as input, and having the determination result, a result of determining whether or not the plurality of machining reference points are within the allowable range, respectively, And at least one of deviation amounts of the plurality of machining reference points from the center value of the respective allowable ranges is output.

12. The method of mounting a workpiece according to claim 11,

learning the machine learning model using the training data that also takes as input at least one of edges detected from the measurement image, deviation amounts from an image center of the plurality of workpiece reference line positions, a focal point distance of a lens of the camera, a distortion parameter of the camera.

13. The method of mounting a workpiece according to any one of claims 1 to 12,

placing a second additional workpiece having substantially the same shape and size as the reference workpiece on the other of the placing table from which the reference workpiece has been removed and the additional placing table disposed in place of the placing table,

acquiring a fourth additional measurement image representing the second additional workpiece captured by the camera set to the target configuration,

generating, by the processor, a fourth additional measurement composite image in which the plurality of workpiece reference lines and the fourth additional measurement image are superimposed such that the plurality of workpiece reference lines are respectively represented at a plurality of positions in the fourth additional measurement composite image that are the same as the positions of the plurality of workpiece reference lines,

the second additional workpiece is adjusted in posture on the other stage so that the boundary of the ninth image region occupied by the second additional workpiece in the fourth additional measurement composite image is substantially parallel to or substantially coincident with the plurality of workpiece reference lines.

14. A work mounting support system characterized by comprising:

a placing table configured to place the reference workpiece or the workpiece alternatively in order to adjust a posture of the workpiece having a shape and a size substantially the same as a shape and a size of the reference workpiece and a posture of the reference workpiece;

a camera configured to photograph the reference workpiece and the workpiece on the placement table;

an input device configured to set a plurality of workpiece reference lines at a boundary of a first image region occupied by the reference workpiece in a reference image of the reference workpiece captured by the camera and indicating the reference workpiece adjusted so that the posture of the reference workpiece is a workpiece reference posture;

a memory configured to store a plurality of positions in the reference image at which the plurality of workpiece reference lines are respectively located as a plurality of workpiece reference line positions;

a processor configured to generate a measurement composite image in which the plurality of workpiece reference lines and a measurement image representing the workpiece captured by the camera when the posture of the workpiece is adjusted on the placing stage are superimposed so that the plurality of workpiece reference lines are represented in the measurement composite image at a plurality of positions that are the same as the positions of the plurality of workpiece reference lines, respectively; and

a display configured to display the measurement composite image when the measurement image is captured.

15. A workpiece mounting support program for causing a processor to execute:

acquiring a reference image showing a reference workpiece on a placing table adjusted so that the posture of the workpiece is a reference posture of the workpiece, the reference image being captured by a camera arranged to aim the viewpoint position and the visual line direction of the camera to be a reference viewpoint position and a reference visual line direction, respectively,

setting a plurality of workpiece reference lines at a boundary of a first image region occupied by the reference workpiece in the reference image,

obtaining a measurement image captured by the camera set to the target arrangement and indicating a workpiece having a shape and a size substantially identical to those of the reference workpiece placed on one of the placing table from which the reference workpiece is removed and an additional placing table different from the placing table and disposed in place of the placing table,

generating a measurement composite image in which the plurality of workpiece reference lines and the measurement image are superimposed such that the plurality of workpiece reference lines are respectively represented at a plurality of positions identical to the plurality of workpiece reference lines in the measurement composite image,

causing a display to display the measured composite image.