JP2020040130A - Processing device, control method of processing device, and control program of processing device - Google Patents

Abstract

To provide a processing device including measuring means capable of performing highly accurate measurement based on an image; and to provide a control method of the processing device.SOLUTION: A processing device includes at least two cameras including at least one mobile camera 101 and a camera 102 fixed inside the processing device, and a measuring part for measuring a three-dimensional shape of an imaging object by using images acquired by at least two cameras. The mobile camera is attached to a tool spindle. Measuring means measures at least either of the depth of a hole processed in the imaging object and the size of a step processed on the imaging object, as a three-dimensional shape.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a processing device, a control method for the processing device, and a control program for the processing device.

  In the above technical field, Patent Literature 1 discloses a technique in which a camera is arranged in a machine tool, and the inside of the machine is photographed and displayed using the camera.

JP 2018-94689 A

  However, with the technology described in the above-mentioned literature, a desired image could not be obtained, and therefore, highly accurate measurement based on the image could not be performed.

  An object of the present invention is to provide a technique for solving the above-mentioned problem.

In order to achieve the above object, a processing apparatus according to the present invention includes:
At least two cameras including at least one moving camera;
Using an image acquired by the at least two cameras, measuring means for measuring the three-dimensional shape of the imaging target,
With.

In order to achieve the above object, a method for controlling a processing apparatus according to the present invention includes:
A method for controlling a processing apparatus including at least two cameras including at least one moving camera,
An obtaining step of obtaining an image captured by the at least two cameras;
Using the acquired image, a measurement step of measuring the three-dimensional shape of the imaging target,
including.

In order to achieve the above object, a control program for a processing apparatus according to the present invention includes:
A control program for a processing apparatus provided with at least two cameras including at least one moving camera,
An obtaining step of obtaining an image captured by the at least two cameras;
Using the acquired image, a measurement step of measuring the three-dimensional shape of the imaging target,
On a computer.

  According to the present invention, highly accurate measurement based on an image can be performed.

It is a figure showing the composition of the processing device concerning a 1st embodiment of the present invention. It is a figure showing the outline of an example of the composition of the processing device concerning a 2nd embodiment of the present invention. It is a figure showing the outline of other examples of the composition of the processing device concerning a 2nd embodiment of the present invention. It is a block diagram showing the composition of the processing device concerning a 2nd embodiment of the present invention. It is a figure showing an example of a camera table with which a processing device concerning a 2nd embodiment of the present invention is provided. It is a block diagram showing the hardware constitutions of the processing device concerning a 2nd embodiment of the present invention. It is a flow chart explaining a processing procedure of a processing device concerning a 2nd embodiment of the present invention. It is a figure showing the outline of an example of the composition of the processing device concerning a 3rd embodiment of the present invention. It is a figure showing an example of a camera table with which a processing device concerning a 3rd embodiment of the present invention is provided. It is a flow chart explaining the processing procedure of the processing device concerning a 3rd embodiment of the present invention.

  Hereinafter, an embodiment for carrying out the present invention will be illustratively described in detail with reference to the drawings. However, the configurations, numerical values, processing flows, functional elements, and the like described in the following embodiments are merely examples, and modifications and changes are free, and the technical scope of the present invention is described in the following description. It is not intended to be limiting.

[First Embodiment]
A processing apparatus 100 according to a first embodiment of the present invention will be described with reference to FIG. The processing device 100 is a device that measures a three-dimensional shape from a camera image.

  As shown in FIG. 1, the processing apparatus 100 includes a moving camera 101, a camera 102, and a measuring unit 103. Further, the processing apparatus 100 has at least two cameras 101 and 102 including at least one moving camera 101. The measurement unit 103 measures the three-dimensional shape of the imaging target 110 using the images acquired by at least two cameras 101 and 102.

  According to the present embodiment, highly accurate measurement based on an image can be performed.

[Second embodiment]
Next, a processing apparatus according to a second embodiment of the present invention will be described with reference to FIGS. 2A to 6. FIG. 2A is a diagram illustrating an outline of an example of a configuration of a processing apparatus according to the present embodiment. The processing apparatus 200 performs processing on the imaging target 210 placed on the rotary stage 230 (processing table). The processing apparatus 200 has at least two cameras arranged in the apparatus.

  The processing apparatus 200 includes a moving camera 201 and a fixed camera 202. The moving camera 201 is attached to the tool spindle 220. A spindle is attached to the tool spindle 220, and the moving camera 201 is attached to the tool spindle 220 via, for example, a spindle. Then, the moving camera 201 moves in accordance with the movement of the tool spindle 220 or the spindle. That is, the mobile camera 201 is a mobile camera. Although the spindle rotates, the moving camera 201 may be mounted so as not to rotate in accordance with the rotation of the spindle.

  The fixed camera 202 is attached to the ceiling of the processing device 200. The fixed camera 202 is attached to the ceiling of the processing apparatus 200 and cannot be moved (the position does not change), so that it is a fixed camera.

  The processing apparatus 200 obtains a three-dimensional image as in the case of a so-called stereo camera by capturing images with the moving camera 201 and the fixed camera 202 and superimposing the obtained images. Using the obtained three-dimensional image, the processing apparatus 200 performs processing such as the three-dimensional shape of the imaging target 210 (work, processing target, etc.), the depth of a hole processed in the imaging target, the size of a step, and the like. Is measured. Further, by measuring the three-dimensional shape, the processing device 200 can avoid collision between the tool and the imaging target 210, collision between components of the processing device 200 and the imaging target 210, and the like.

  Further, one of the two cameras is attached to the tool spindle 220 to be the movable camera 201, and the other is fixed inside the processing apparatus 200 to be the fixed camera 202, so that a wider range of images can be obtained.

  In this way, if the mobile camera 201 and the fixed camera 202 are attached to the processing apparatus 200, a stereoscopic image in the apparatus can be obtained, so that there is no need to provide a glass window for checking the inside of the processing apparatus 200 from outside. . Therefore, even if an accident such as breakage of a tool or destruction of the imaging target 210 occurs inside the processing apparatus 200, a window is broken by broken pieces of the tool or the like, and damage is caused outside the processing apparatus 200. Will not reach.

  Next, a method for determining the shooting posture and the installation position of the mobile camera 201 and the fixed camera 202 will be described. The accuracy of the three-dimensional shape of the imaging target 210 measured using the images acquired by the moving camera 201 and the fixed camera 202 increases as the base angle increases. Here, the base angle is the angle of a triangle formed by the mobile camera 201, the fixed camera 202, and the photographing target 210. As described above, the accuracy of the three-dimensional shape increases as the base angle increases, but on the other hand, the feature points must be captured by the two cameras (the moving camera 201 and the fixed camera 202).

  Further, the position of the mobile camera 201 in the Z direction can be changed (determined) by determining whether or not the imaging target 210 falls within the field of view of the camera in order to change the field of view (FOV) of the camera. The position of the mobile camera 201 in the X direction may be any position as long as the photographing target 210 comes to the center of the visual field. The position of the mobile camera 201 in the Y direction only needs to be a position where the base angle is maximum and the surface of the imaging target 210 can be observed.

  The photographing attitude and the installation positions of the mobile camera 201 and the fixed camera 202 can be specifically determined according to, for example, the following procedure. (1) An attention area is set from a model of the imaging target 210 (work). (2) Match the angle between the center of the attention area and the fixed camera 202 with the angle of the SP axis, and match the epipolar line with the pixel direction of the camera. (3) The X coordinate of the moving camera 201 is determined so that the center of the attention area matches the center of the visual field. (4) The Z position is determined from the size of the attention area and the angle of view of the moving camera 201. (5) The distribution of the Y component of the normal vector of each facet of the imaging target 210 is obtained. (6) The shooting direction of the mobile camera 201 is set so that the angle between the vector directed from the mobile camera 201 to the shooting target 210 and the normal vector of (5) is equal to or less than a predetermined value (for example, 70 °). Decide. (7) The Y coordinate is obtained from the angle obtained in (6) and the Z coordinate obtained in (4).

  FIG. 2B is a diagram illustrating an outline of another example of the configuration of the processing apparatus according to the present embodiment. The processing apparatus 240 has at least two cameras disposed in the apparatus as in the processing apparatus 200. In the processing device 240, the rotary stage 230 slides and moves in the Y-axis direction (vertical direction). In the processing device 240, two moving cameras 201) are attached to the rotary stage 230. The disk-shaped pallet 250 has a photographing target 210 such as a processing target (work) previously set thereon, and is conveyed onto the rotary stage 230 by a transfer robot or the like (not shown).

  Using the images acquired by the two moving cameras 201), the imaging target 210 and the like inside the processing device 240 can be captured as a three-dimensional shape, so that the control of the transfer robot can be closed-loop control. Further, since the processing device 240 measures the three-dimensional shape, it determines whether or not the NC (Numerical Control) program responds to the imaging target 210 while the imaging target 210 such as a work or a processing target is being conveyed, and makes a response. You can select a program.

  In addition, since the processing device 240 can measure the three-dimensional shape of a part of the spindle attached to the tool spindle 220, it can check itself after a collision, check the winding of chips, and the like. If the processing device 240 includes an acceleration sensor or the like and can detect an abnormality such as a collision, the abnormal state can be easily verified by storing the spindle itself as a three-dimensional shape after the abnormality is detected.

    The number of mobile cameras 201 disposed inside the processing apparatuses 200 and 240 may be one or more, and all of the plurality of cameras inside the processing apparatuses are mobile cameras 201. You may. When the number of mobile cameras is one, at least one fixed camera 202 is required to be disposed inside the processing apparatuses 200 and 240. However, a plurality of fixed cameras 202 may be provided.

  FIG. 3 is a block diagram illustrating a configuration of the processing apparatus according to the present embodiment. The processing apparatus 200 includes a moving camera 201, a fixed camera 202, a measuring unit 301, and a detecting unit 302. The moving camera 201 is movable in the processing device 200. Further, the mobile camera 201 captures an image of the inside of the processing apparatus 200 by capturing an image. In addition, the moving camera 201 may capture an image while moving, and acquire an image. Alternatively, when capturing, the mobile camera 201 may capture a still image to acquire an image.

  The fixed camera 202 is a camera attached to a predetermined position inside the processing device 200. That is, the fixed camera 202 is a camera that does not move. The fixed camera 202 photographs the inside of the processing apparatus 200 and acquires an image.

  The mobile camera 201 and the fixed camera 202 are objects that exist inside the processing apparatus 200 and are to be photographed. In FIG. 3, the object 210 to be photographed by the mobile camera 201 and the fixed camera 202 is a processing object mounted on a rotary stage 230 (processing table). However, the imaging target 210 may be, for example, a processing target (work) to be processed by the processing apparatus 200, a processing table on which the processing target is placed, a tool spindle, a tool holder, a chuck, a cutting tool, or the like. It is not limited to these as long as it exists inside the processing device 200. Further, the fixed camera 202 may be photographed by the moving camera 201 or the moving camera 201 may be photographed by the fixed camera 202.

  The measurement unit 301 measures the three-dimensional shape of the photographing target 210 by using the images captured by the moving camera 201 and the fixed camera 202 and using the acquired images. The measurement unit 301 obtains information in the depth direction of the imaging target 210 by using images obtained by simultaneously capturing the imaging target 210 from a plurality of different directions, that is, images acquired by the moving camera 201 and the fixed camera 202. Can be Accordingly, the measurement unit 301 can measure the three-dimensional shape of the imaging target 210.

  Further, the detection unit 302 detects the position of the mobile camera 201 using the fixed camera 202, and executes calibration. First, the measurement unit 301 acquires an image of the mobile camera 201 captured by the fixed camera 202. Then, the detection unit 302 obtains parameters such as internal parameters, external parameters, and distortion coefficients of the mobile camera 201 and the fixed camera 202, and performs calibration. If the above parameters are obtained by executing the calibration, the corresponding point search, the rectification, and the like can be efficiently performed under the epipolar geometric constraint condition. Note that the detection unit 302 performs calibration at a timing such as when the processing apparatus 200 is started. The timing at which calibration is performed by the detection unit 302 is not limited to this, and may be timing during processing, after processing is completed, or the like. Incidentally, the three-dimensional coordinates may be directly calculated by the 8-point algorithm without performing the calibration.

  Further, the measurement unit 301 measures at least one of the depth of a hole formed in the imaging target 210 and the size of a step formed in the imaging target 210, as the three-dimensional shape of the imaging target 210. Thus, since the processing state and the progress of the processing of the imaging target 210 can be known, the processing apparatus 200 can change or control the processing conditions and the like. Note that the three-dimensional shape measured by the measurement unit 301 is not limited to the depth of the hole or the size of the step, and may be, for example, the diameter of the hole, the depth and the length of the groove, and the like.

  FIG. 4 is a diagram illustrating an example of a camera table included in the processing apparatus according to the present embodiment. The camera table 401 stores a move / fix 412, a position 413, a specification 414, and a captured image 415 in association with a camera ID (Identifier) 411. The camera ID 411 is an identifier for identifying a camera. The move / fix 412 indicates whether the camera is a mobile camera or a fixed camera. The position 413 indicates the position of the camera (the mobile camera 201 and the fixed camera 202). The specification 414 indicates the performance of the camera, and includes, but is not limited to, an angle of view, an F value, a focal length, the number of pixels, a magnification, and the like. The captured image 415 is an image captured by a camera (the moving camera 201 and the fixed camera 202). The processing apparatus 200 measures, for example, the three-dimensional shape of the imaging target with reference to the camera table 401. Further, the processing device 200 detects the position of the mobile camera 201 with reference to the camera table 401.

  FIG. 5 is a block diagram illustrating a hardware configuration of the processing apparatus according to the present embodiment. A CPU (Central Processing Unit) 510 is a processor for arithmetic control, and realizes the functional components of the processing apparatus 200 in FIG. 3 by executing a program. The CPU 510 includes a plurality of processors, and may execute different programs, modules, tasks, threads, and the like in parallel. A ROM (Read Only Memory) 520 stores fixed data such as initial data and programs, and other programs. In addition, the network interface 530 communicates with other devices via a network. Note that the number of CPUs 510 is not limited to one, and may include a plurality of CPUs or a GPU (Graphics Processing Unit) for image processing. Further, it is desirable that the network interface 530 has a CPU independent of the CPU 510 and writes or reads transmission / reception data in an area of a RAM (Random Access Memory) 540. It is desirable to provide a DMAC (Direct Memory Access Controller) for transferring data between the RAM 540 and the storage 550 (not shown). Further, CPU 510 recognizes that the data has been received or transferred to RAM 540 and processes the data. The CPU 510 prepares the processing result in the RAM 540, and leaves the subsequent transmission or transfer to the network interface 530 or the DMAC.

  RAM 540 is a random access memory used by CPU 510 as a work area for temporary storage. In the RAM 540, an area for storing data necessary for realizing the present embodiment is secured. The position 541 is data of the position of the mobile camera 201 and the fixed camera 202 inside the processing device 200. The specification 542 is data relating to the performance of the mobile camera 201 and the fixed camera 202 and the like. The captured image 543 is data of an image captured by the mobile camera 201 and the fixed camera 202. The parameter 544 is an internal parameter, an external parameter, a distortion coefficient, and the like between the mobile camera 201 and the fixed camera 202, which are obtained by the calibration.

  The transmission / reception data 545 is data transmitted / received via the network interface 530. Further, the RAM 540 has an application execution area 546 for executing various application modules.

  The storage 550 stores a database, various parameters, or the following data or programs necessary for realizing the present embodiment. The storage 550 stores the camera table 401. The camera table 401 is a table for managing the relationship between the camera ID 411 and the captured image 415 shown in FIG.

  The storage 550 further stores a measurement module 551 and a detection module 552. The measurement module 551 is a module that measures the three-dimensional shape of the photographing target 210 using images acquired by photographing with the moving camera 201 and the fixed camera 202. The measurement module 551 is a module that measures, as a three-dimensional shape, at least one of the depth of a hole formed in the imaging target 210 and the size of a step formed in the imaging target 210. The detection module 552 is a module that detects the position of the mobile camera 201 using the fixed camera 202. These modules 551 to 552 are read into the application execution area 546 of the RAM 540 by the CPU 510 and executed. The control program 553 is a program for controlling the entire processing apparatus 200.

  The input / output interface 560 interfaces input / output data with input / output devices. The display unit 561 and the operation unit 562 are connected to the input / output interface 560. Further, a storage medium 564 may be connected to the input / output interface 560. Furthermore, a speaker 563 as an audio output unit, a microphone (not shown) as an audio input unit, or a GPS position determination unit may be connected. Note that the RAM 540 and the storage 550 shown in FIG. 5 do not show programs and data relating to general-purpose functions and other realizable functions of the processing apparatus 200.

  FIG. 6 is a flowchart illustrating a processing procedure of the processing apparatus according to the present embodiment. This flowchart is executed by the CPU 510 in FIG. 5 using the RAM 540, and implements the functional components of the processing apparatus 200 in FIG.

  In step S601, the processing device 200 executes a start operation of the device. In step S603, the processing device 200 determines whether calibration is necessary. If calibration is not necessary (NO in step S603), the processing apparatus 200 proceeds to step S609. If calibration is necessary (YES in step S603), the processing apparatus 200 proceeds to step S605.

  In step S605, the processing apparatus 200 detects the position of the mobile camera 201 using the fixed camera 202. In step S607, the processing apparatus 200 executes calibration to obtain parameters such as internal parameters, external parameters, and distortion coefficients of the mobile camera 201 and the fixed camera 202.

  In step S609, the processing apparatus 200 acquires an image captured by the moving camera 201 and the fixed camera 202, and acquires the acquired image. In step S611, the processing apparatus 200 measures the three-dimensional shape of the imaging target 210 using the images acquired by the moving camera 201 and the fixed camera 202. The processing apparatus 200 measures, for example, at least one of a depth of a hole processed in the imaging target 210 as a three-dimensional shape and a size of a step processed in the imaging target 210. In step S613, the processing device 200 determines whether to end the measurement. If it is determined that the measurement is not to be ended (NO in step S613), the processing device 200 returns to step S609. If it is determined that the measurement is to be ended (YES in step S613), the processing device 200 ends the processing.

  According to the present embodiment, highly accurate measurement based on an image can be performed. Further, since the position of the mobile camera is detected using the fixed camera, the camera can be calibrated. Further, it is possible to measure the depth of a hole formed in the object to be photographed as a three-dimensional shape and the size of a step formed in the object to be photographed.

[Third embodiment]
Next, a processing apparatus according to a third embodiment of the present invention will be described with reference to FIGS. FIG. 7 is a diagram illustrating an outline of an example of a configuration of a processing apparatus according to the present embodiment. The processing apparatus according to the present embodiment is different from the above-described second embodiment in that it has a tool magazine and a camera exchange unit. Other configurations and operations are the same as those of the second embodiment, and thus the same configurations and operations are denoted by the same reference numerals and detailed description thereof will be omitted.

  The processing device 700 includes a tool magazine 701 and a camera exchange unit 702. The tool magazine 701 houses a plurality of types of cameras together with a plurality of tools (tools). The tool magazine 701 is, for example, an ATC (Automatic Tool Changer) magazine and has a role of storing a tool, selecting a tool, and exchanging a tool. The tool magazine 701 may be either a turret type or a magazine storage type.

  The camera exchange unit 702 takes out the selected camera from the tool magazine 701 and attaches the selected camera to a predetermined position inside the processing device 700. The camera exchange unit 702 is, for example, a robot arm, and can take out the camera from the tool magazine 701 and attach it to a predetermined position inside the processing device 700. Note that the camera exchange unit 702 may be any device as long as it can exchange cameras.

  FIG. 8 is a diagram illustrating an example of a camera table included in the processing apparatus according to the present embodiment. The camera table 801 stores the accommodation position 811 in association with the camera ID 411. The storage position 811 indicates a position where the camera is stored in the tool magazine 701. The processing device 700 refers to the camera table 801 to select and exchange a camera according to the purpose.

  FIG. 9 is a flowchart illustrating a processing procedure of the processing apparatus according to the present embodiment. This flowchart is executed by the CPU 510 in FIG. 5 using the RAM 540, and implements the functional components of the processing apparatus 700 in FIG. In step S901, the processing device 700 determines whether the camera needs to be replaced. If it is determined that the camera does not need to be replaced (NO in step S901), the processing device 700 proceeds to step S603. If it is determined that the camera needs to be replaced (YES in step S901), the processing device 700 proceeds to step S903. In step S903, the processing apparatus 700 selects a camera according to the application from among a plurality of types of cameras, and replaces the camera.

  According to the present embodiment, the camera can be selectively replaced, so that measurement suitable for various uses can be performed.

[Other embodiments]
Although the present invention has been described with reference to the exemplary embodiments, the present invention is not limited to the above exemplary embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. In addition, a system or an apparatus in which different features included in each embodiment are combined in any way is also included in the scope of the present invention.

In order to achieve the above object, a processing apparatus according to the present invention includes:
A tool spindle for mounting a tool used for machining,
At least two cameras including at least one moving camera;
Using an image acquired by the at least two cameras, measuring means for measuring the three-dimensional shape of the imaging target,
A tool magazine accommodating the moving camera together with a tool,
Camera mounting means for mounting the moving camera housed in the tool magazine to the tool spindle,
With.

In order to achieve the above object, a method for controlling a processing apparatus according to the present invention includes:
A tool spindle for mounting a tool used for machining,
At least two cameras including at least one moving camera;
Using an image acquired by the at least two cameras, measuring means for measuring the three-dimensional shape of the imaging target,
A tool magazine accommodating the moving camera together with a tool,
Camera mounting means for mounting the moving camera housed in the tool magazine to the tool spindle,
A method for controlling a processing apparatus comprising:
A camera mounting step of mounting the moving camera accommodated in the tool magazine to the tool spindle,
An acquisition step of acquiring an image taken by the moving camera,
Using the acquired image, a measurement step of measuring the three-dimensional shape of the imaging target,
including.

In order to achieve the above object, a control program for a processing apparatus according to the present invention
A tool spindle for mounting a tool used for machining,
At least two cameras including at least one moving camera;
Using an image acquired by the at least two cameras, measuring means for measuring the three-dimensional shape of the imaging target,
A tool magazine accommodating the moving camera together with a tool,
Camera mounting means for mounting the moving camera housed in the tool magazine to the tool spindle,
A control program for a processing apparatus comprising:
A camera mounting step of mounting the moving camera accommodated in the tool magazine to the tool spindle,
An acquisition step of acquiring an image taken by the moving camera,
Using the acquired image, a measurement step of measuring the three-dimensional shape of the imaging target,
On a computer.

Claims (7)

At least two cameras including at least one moving camera;
Using an image acquired by the at least two cameras, measuring means for measuring the three-dimensional shape of the imaging target,
Processing equipment equipped with The at least two cameras include a fixed camera mounted inside the processing device;
The processing apparatus according to claim 1, further comprising a detection unit configured to detect a position of the mobile camera using the fixed camera.   The processing apparatus according to claim 1, wherein the movable camera is attached to a tool spindle.   4. The apparatus according to claim 1, wherein the measuring unit measures at least one of a depth of a hole processed in the imaging target and a size of a step processed in the imaging target as the three-dimensional shape. 5. The processing apparatus according to claim 1. A tool magazine that houses multiple types of cameras with multiple tools,
Camera exchange means for selectively exchanging the plurality of types of stored cameras,
The processing apparatus according to any one of claims 1 to 4, further comprising: A method for controlling a processing apparatus including at least two cameras including at least one moving camera,
An obtaining step of obtaining an image captured by the at least two cameras;
Using the acquired image, a measurement step of measuring the three-dimensional shape of the imaging target,
A method for controlling a processing apparatus including: A control program for a processing apparatus provided with at least two cameras including at least one moving camera,
An obtaining step of obtaining an image captured by the at least two cameras;
Using the acquired image, a measurement step of measuring the three-dimensional shape of the imaging target,
Control program for a processing apparatus that causes a computer to execute the processing.

Images