WO2023032876A1

WO2023032876A1 - Information processing device, and machine tool

Info

Publication number: WO2023032876A1
Application number: PCT/JP2022/032317
Authority: WO
Inventors: 絢一郎奥野
Original assignee: Dmg森精機株式会社
Priority date: 2021-09-06
Filing date: 2022-08-29
Publication date: 2023-03-09

Abstract

An information processing device according to one embodiment of the present invention comprises: an acquisition unit that acquires a captured image of an image capture area, captured by an image capturing unit which is disposed on a machine tool; a grid setting unit that divides the captured image into a grid large enough to contain multiple pixels; and a display control unit that, taking the arranged rows and columns of the grid as units, causes a reference screen of a display unit to display an extraction region, which is a portion of the captured image that has been extracted by cutting out the outer peripheral portion of the captured image. The display control unit causes the display unit to display the extraction region, which is smaller than the frame of the reference screen as a result of cutting out the outer peripheral portion from the captured image, so that the contour of the extraction region is enlarged toward the frame of the reference screen.

Description

Information processing equipment and machine tools

The present invention relates to an information processing device that processes captured images used in machine tools.

Chips are generated when workpieces are processed by machine tools. If a large amount of chips accumulate in the processing chamber, it becomes difficult to continue processing. For this reason, conventionally, the operator has taken measures such as periodically stopping the operation of the machine tool and removing the chips using an air blower or the like. However, such manual chip removal reduces the operating efficiency of the machine tool.

Therefore, in recent years, a system has been proposed in which chips are automatically removed by installing a camera in the processing chamber and analyzing the captured image (see Patent Document 1). In this system, the scattering position of chips is specified based on the analysis result, and control (hereinafter also referred to as "cleaning control") is performed to target the scattering position and discharge coolant. A technology has also been proposed in which the chip scattering range is superimposed on a captured image displayed on a monitor screen, and the operator can specify the coolant emission range on the screen. The software for cleaning control is preset with a corresponding relationship between the coordinates of the area captured by the camera and the coordinates used for the coolant control command.

Japanese Patent No. 6887033

By the way, when introducing a new system like this, it is assumed that it will be retrofitted to existing machine tools. In that case, there is a possibility that the captured image will not be displayed at the angle at which it should be seen due to mounting errors or individual differences of the retrofitted camera. As a countermeasure, for example, correcting the display angle of the captured image may be considered.

An aspect of the present invention is an information processing device. This information processing apparatus includes an acquisition unit that acquires a captured image of an imaging area by an imaging unit installed in a machine tool, a grid setting unit that divides the captured image into grids each having a size including a plurality of pixels, and a grid arrangement. a display control unit for displaying an extraction region, which is a part of the captured image, extracted by cutting out the outer peripheral portion of the captured image in units of rows and columns, on a reference screen of the display unit. The display control unit causes the display unit to display the extraction area, which is smaller than the frame of the reference screen by cutting the outer periphery from the captured image, so that the contour of the extraction area expands toward the frame of the reference screen.

Another aspect of the present invention is a machine tool. This machine tool includes an imaging unit that captures an image of an imaging area set in a processing chamber, a display unit that displays an image captured by the imaging unit, and a grid setting unit that divides the captured image into grids each having a size including a plurality of pixels. and a display control unit for displaying, on a reference screen of a display unit, an extraction region which is a part of the captured image extracted by cutting out the outer peripheral portion of the captured image in units of rows and columns of the grid arrangement. The display control unit causes the display unit to display the extraction area, which is smaller than the frame of the reference screen by cutting the outer periphery from the captured image, so that the contour of the extraction area expands toward the frame of the reference screen.

According to the present invention, it is possible to display the captured image of the imaging unit installed in the machine tool so that the operator does not feel uncomfortable.

It is a perspective view showing the appearance of the machine tool concerning an embodiment. It is a hardware block diagram of a machine tool. It is a perspective view showing the structure in a processing chamber. It is a figure which shows the structure of a camera typically. 3 is a functional block diagram of an information processing device; FIG. 3 is a block diagram showing the configuration of a recognition unit; FIG. It is a figure showing the correction method of a screen display. It is a figure showing the correction method of a screen display. It is a figure showing the correction method of a screen display. It is a figure showing the correction method of a screen display. It is a figure showing an example of the screen which an operator operates. 4 is a flowchart showing the flow of correction processing; 4 is a flowchart schematically showing the flow of cleaning control; It is a figure showing correction processing when rotation angle theta of a captured image is set to 3.3 degrees. It is a figure showing the structure of the information processing system which concerns on a modification. It is a figure showing the image processing method concerning a modification.

An embodiment of the present invention will be described below with reference to the drawings.
FIG. 1 is a perspective view showing the appearance of the machine tool according to the embodiment.
The machine tool 1 is configured as a multitasking machine that processes a workpiece into a desired shape while appropriately exchanging tools. A machine tool 1 is provided with a processing chamber 2 inside a device housing. The processing chamber 2 is provided with a processing device for processing a work. An operation panel 4 for operating the processing apparatus is provided on the front surface of the apparatus housing.

FIG. 2 is a hardware configuration diagram of the machine tool 1. As shown in FIG.
The machine tool 1 includes an information processing device 100 , a machining control device 102 , a machining device 104 , a tool changing section 106 , a tool storage section 108 and an imaging section 110 . The machining control device 102 functions as a numerical control unit and outputs control signals to the machining device 104 according to a machining program (NC program). The processing device 104 processes a workpiece by moving a tool spindle (not shown; hereinafter simply referred to as “spindle”) according to instructions from the machining control device 102 .

The processing device 104 includes a mechanism for driving the spindle, a liquid reservoir 112 for storing coolant, and a liquid injection section 114 for injecting coolant. The coolant is used as cutting oil for removing heat and lubricating the tools and workpieces during machining, and is also used as a cleaning liquid for removing chips scattered in the machining chamber 2 . The liquid injection section 114 includes a pump that draws up coolant from the liquid injection section 114, a nozzle that injects the coolant, and an actuator that drives the nozzle.

The information processing device 100 includes an operation panel 4, and outputs control commands to the processing control device 102 based on operator input. The information processing apparatus 100 also controls the screen displayed on the monitor of the operation panel 4 according to the operator's operation input. The tool storage section 108 stores tools. The tool changer 106 corresponds to a so-called ATC (Automatic Tool Changer), takes out a tool from the tool storage part 108 according to a change instruction from the machining control device 102, and replaces the tool on the spindle with the taken out tool.

The imaging unit 110 is, for example, a camera equipped with an imaging device such as a CCD or CMOS, and images an imaging area set in the processing chamber 2 . As the "imaging area", an area in which chips generated by machining the workpiece are assumed to exist is set in advance. The angle of view of the camera is set so that the distribution and accumulation of chips can be grasped over a wide range in the processing chamber 2 . The imaging unit 110 outputs the captured image to the information processing device 100 .

FIG. 3 is a perspective view showing the configuration inside the processing chamber 2. As shown in FIG. FIG. 3(A) shows a state seen obliquely from above, and FIG. 3(B) shows a state seen obliquely from below.
As shown in FIG. 3(A), the processing chamber 2 is surrounded by four side surfaces, and a main shaft 10 is provided on one side surface so as to be vertically and horizontally movable. The main shaft 10 has a horizontal rotating shaft, and a tool T is coaxially attached to its tip. A side surface facing the main shaft 10 in the axial direction has a revolving door 12 . A support plate 14 extends horizontally from the revolving door 12 . The revolving door 12 is a door that can rotate around a vertical axis.

A table 16 is provided below the support plate 14 . A pallet 18 is detachably attached to the table 16 , and a work is placed and fixed on the pallet 18 . By preparing a plurality of pallets 18 on which works are fixed, it is possible to change the work by changing the pallet 18, and it is possible to improve the efficiency of time.

The table 16 is movable in the axial direction of the main shaft 10 and rotatable in the horizontal plane. By rotating the table 16, the work on the pallet 18 can be rotated. The workpiece approaches or separates from the tool T by linearly driving the table 16 . That is, by controlling the rotation and movement of the table 16 and the movement of the spindle 10, the workpiece can be processed into a desired shape.

The support plate 14 is fitted with the pallet 18 at the position where the table 16 is farthest from the spindle 10 . By rotating the revolving door 12 in this state, the support plate 14 separates the pallet 18 from the table 16 and rotates together with the pallet 18 . As a result, the pallet 18 on which the work has been processed can be carried out from the processing chamber 2 and the pallet 18 to which the work to be processed next is fixed can be carried into the processing chamber 2 .

A chip conveyor 20 is provided below the table 16 and the spindle 10 for conveying chips to the outside of the processing chamber 2 . The table 16 moves above the chip conveyor 20 . A shooter 22 is provided below the table 16 . The chute 22 guides the chips flowing from above due to cleaning onto the chip conveyor 20 .

In the machining chamber 2, the bottoms located on both sides of the table 16 are slopes 24, which are inclined downward toward the shooter 22 so that chips scattered during machining can easily flow to the shooter 22.

As shown in FIG. 3(B), the upper part of the processing chamber 2 is shielded by a cover 26 (ceiling). A cover 26 is provided with a plurality of nozzles 28 for supplying coolant. In this embodiment, the nozzle 28 is configured to be replaceable together with the cover 26 . The nozzle 28 constitutes a liquid injection section 114 and is connected to the liquid storage section 112 via piping, valves, pumps and the like (not shown) (see FIG. 2). The nozzle 28 is configured to be three-dimensionally rotatable. By rotating the nozzle 28, the injection direction of the coolant can be controlled. By specifying the direction of the nozzle 28 and driving the pump, the coolant can be injected toward the target in the processing chamber 2 . Chips generated by machining the workpiece are washed away by the coolant and carried out of the machining chamber 2 by the chip conveyor 20 . Although two nozzles 28 are installed in this embodiment, the number can be set as appropriate.

A plurality of cameras 30 for capturing images of the inside of the processing chamber 2 from above are also installed in the upper part of the processing chamber 2 . In this embodiment, two cameras 30 are attached to the side wall slightly below the cover 26 in the processing chamber 2 . In this embodiment, the camera 30 cannot be integrated with the cover 26 due to wiring reasons. The camera 30 constitutes an image capturing unit 110, and captures an image of the machining status of the workpiece by the tool T, and an image of chips generated by the machining (see FIG. 2). Since two cameras 30 are provided, an area that cannot be captured by one camera 30 can be captured by the other camera 30 . The imaging unit 110 outputs the captured image to the information processing device 100 .

FIG. 4 is a diagram schematically showing the structure of the camera 30. As shown in FIG.
The camera 30 of this embodiment has a structure in which rotation about its optical axis L is restricted. That is, the camera 30 is rotatable around the yaw axis L1 orthogonal to the optical axis L and around the pitch axis L2 orthogonal to the optical axis L and the yaw axis L1, but cannot be rotated around the optical axis L. . Therefore, if the coordinates of the imaging area displayed on the screen deviate from the coordinates of the imaging area recognized by the software due to an installation error of the camera 30 or the like, manual adjustment becomes difficult. Therefore, the information processing apparatus 100 executes correction processing for eliminating the deviation of the coordinates when the operator adjusts the angle of the camera.

FIG. 5 is a functional block diagram of the information processing device 100. As shown in FIG.
Each component of the information processing apparatus 100 includes computing units such as a CPU (Central Processing Unit) and various computer processors, storage devices such as memory and storage, hardware including wired or wireless communication lines connecting them, and storage devices. , and implemented by software that supplies processing instructions to the computing unit. A computer program may consist of a device driver, an operating system, various application programs located in their higher layers, and a library that provides common functions to these programs. Each block described below represents a functional block rather than a hardware configuration.

The information processing device 100 includes a user interface processing unit 130 , a data processing unit 132 and a data storage unit 134 . The user interface processing unit 130 receives an operation input from an operator, and is in charge of user interface processing such as image display. The data processing unit 132 executes various processes based on data acquired by the user interface processing unit 130 and data stored in the data storage unit 134 . Data processing unit 132 also functions as an interface for user interface processing unit 130 and data storage unit 134 . The data storage unit 134 stores various programs and setting data.

The user interface processing section 130 includes an input section 140 and an output section 142 . The input unit 140 receives an operation input from an operator via the touch panel of the monitor on the operation panel 4 or the like. The output unit 142 includes a display unit 144 that displays an image or the like on the screen of the monitor. The output unit 142 provides various types of information to the operator through its display. A display unit 144 displays an image captured by the imaging unit 110 .

Data processing unit 132 includes acquisition unit 150 , grid setting unit 152 , detection unit 154 , recognition unit 156 , correction unit 158 , display control unit 160 and ejection control unit 162 .
Acquisition unit 150 acquires an image captured by imaging unit 110 . The grid setting unit 152 divides (divides) the captured image into a plurality of grids in order to determine the presence of a predetermined physical quantity (chip) in the imaging area. The grid has a specific shape (square in this embodiment, but any geometric shape is acceptable) (details will be described later). Hereinafter, an area divided by a plurality of grids in a captured image will also be referred to as a "grid area". Also, an image composed of a plurality of grids is also called a "grid image".

The detection unit 154 detects an operator's operation input via the input unit 140 . Although the details will be described later, the operator can grasp the accumulation state of chips in the processing chamber 2 by referring to the captured image displayed on the display unit 144 . The operator can specify the cleaning range (spray range) of the coolant by specifying the area of the captured image via the touch panel. The detection unit 154 detects the instruction input by the operator as the indicated position in the captured image. The detection unit 154 may detect the pointing position based on the grid area created by the grid setting unit 152 .

The recognition unit 156 automatically recognizes chips based on the grid area set in the captured image, determines whether chips exist in the grid area, and determines the amount of chips present. . These determinations are also referred to as "chip determinations". If the recognition unit 156 determines that chips are present in the grid area, the recognition unit 156 recognizes the position on the captured image corresponding to the grid area as the accumulation position of the chips. When the chip stacking position is recognized, an automatic detection signal is output to the injection control section 162 . The automatic detection signal includes at least information regarding the predetermined location where debris has been recognized in the captured image.

Prior to execution of cleaning control, the correction unit 158 executes correction processing for matching the coordinates of the imaging area by the camera 30 with the coordinates used for the coolant control command based on the operator's operation. Since this correction process is based on the premise that there is an installation error in the camera 30 , it is performed when the camera 30 is installed in the processing chamber 2 . The details will be described later.

The display control unit 160 causes the display unit 144 to display the image captured by the camera 30 . When correction is performed by the correction unit 158, the display control unit 160 displays an operation screen for the correction and displays the captured image after correction. An example of a display screen related to this correction will be described in detail later.

The injection control unit 162 outputs a coolant injection command toward the target position to the machining control device 102 when cleaning control is performed. This injection command includes information specifying the position to inject the coolant (information specifying the injection route, etc.). The machining control device 102 receives this injection command, drives the liquid injection unit 114, and controls injection of the coolant.

This cleaning control is to wash away chips in the processing chamber 2 with coolant (that is, to wash the inside of the processing chamber 2), and in this embodiment, an automatic cleaning mode or a manual cleaning mode can be selected. In the automatic cleaning mode, the injection control unit 162 automatically sets the coolant injection position based on the automatic detection signal output by the recognition unit 156, and outputs an injection command. Further, in the manual cleaning mode, the position for injecting the coolant is set based on the operator's area specification detected by the detection unit 154, and an injection command is output.

The data storage unit 134 stores various programs including a correction processing program for executing the correction processing described above and a cleaning control program for executing cleaning control, as well as various data necessary for these processing. Data storage unit 134 includes correction information storage unit 146 and reference information storage unit 148 . The correction information storage unit 146 stores the captured image acquired by the acquisition unit 150 during the correction process, the grid area (grid image) created by the grid setting unit 152, and the predetermined Position information, information on the amount of chips, position information detected by the detection unit 154, and the like are stored. The correction information storage unit 146 also stores correction information (calibration data) acquired in a correction process, which will be described later. The reference information storage unit 148 stores an image for guiding the operator's operation input in the correction process. The data storage unit 134 also functions as a work area when arithmetic processing is performed.

Here, an overview of chip recognition by the recognition unit 156 will be described.
FIG. 6 is a block diagram showing the configuration of the recognition unit 156. As shown in FIG.
The recognition unit 156 includes a model learning unit 41 , a calculation unit 43 and a determination unit 44 .

The model learning unit 41 creates a learning model. When one of the grid areas created by the grid setting unit 152 is input as an input, the learning model is a model that can calculate and output the probability of which one of the predetermined items related to chips in the grid area corresponds. is. This learning model can be created by, for example, using pairs of input data and output data as teacher data and inputting them into a CNN (convolutional neural network) in advance for learning. In this embodiment, the input data can be the grid area, and the output data can be information about the presence and amount of chips in the grid area.

The data storage unit 134 includes a model storage unit 42. The model storage unit 42 stores learning models for automatically determining the presence or absence of chips. The learning model is read into the calculator 43 as needed.

The calculation unit 43 calculates the matching probability of a predetermined item related to chips on a grid-by-grid basis based on the captured image. Specifically, the calculation unit 43 uses the learning model learned by the model learning unit 41 to determine whether there are many chips (class 2), few chips (class 1), or no chips for the grid area. (Class 0)”, the probabilities relating to which of the three items it corresponds to are calculated. These classes indicate the degree of presence of material (such as chips) recognized in the imaging area.

The determination unit 44 determines to which of classes 0 to 2 the chips in the grid area belong, based on the probability calculated by the calculation unit 43 for the input grid area. When the determination unit 44 determines that there is debris in the grid area (that is, determines that it is class 2 or class 1), automatic detection including position information on the captured image corresponding to the position of the grid area in the grid image A signal is output to display control unit 160 and injection control unit 162 .

In this manner, the recognizing unit 156 automatically recognizes chips based on the captured image captured by the imaging unit 110 during or after the processing of the workpiece, and can perform automatic cleaning by injecting coolant. can. The automatic cleaning may be performed periodically, or may be performed by giving some instructions such as instructions from the operator.

Next, the correction method in this embodiment will be described in detail.
As already described, when the above-described cleaning control system is introduced, there is a possibility that the coordinates of the imaging area displayed on the screen and the coordinates of the imaging area recognized by the software may deviate. Therefore, in the present embodiment, prior to using the machine tool 1, the worker (operator) adjusts the angle of the camera 30 to eliminate the deviation. For convenience of explanation, the state in which the two coordinates match is also expressed as "screen matching state".

As described with reference to FIG. 4, the camera 30 of the present embodiment is restricted in rotation around the optical axis, so there is a limit to its adjustment. Therefore, if left as it is, there is a possibility of causing a setting error in the target of the coolant injection control. Therefore, in the present embodiment, the correction unit 158 rotates the image captured by the camera 30 to correct the target setting error. The details will be described below.

7 to 10 are diagrams showing the method of correcting the screen display.
In this embodiment, since a plurality of cameras 30 are provided, each camera 30 is corrected. Since the other cameras 30 are the same, description thereof is omitted.

The camera 30 is installed in the processing chamber 2 so that the predetermined imaging area is included in the angle of view. The imaging area includes areas where swarf scattering and deposition is expected. Here, it is assumed that the captured image P0 shown in FIG. 7 is displayed on the screen (reference screen 170) of the display unit 144 when the camera 30 is installed in the processing chamber 2. FIG. When the operation input by the operator shifts to a maintenance mode (described later) for correcting the screen display, the display control unit 160 superimposes the baseline BL (thick line) and the auxiliary line AL (chain-dotted line) on the captured image P0. display.

The baseline BL is a line set to follow the outline of the specific area SA in the processing chamber 2 when the screen is in the aligned state, and is stored in the reference information storage unit 148 in advance. In this embodiment, the specific area SA is an area surrounded by a boundary line formed by the slope 24 and the edge of the pallet 18 . The auxiliary line AL is a cross-shaped auxiliary line passing through the center C of the captured image P0 in this embodiment. The processing chamber 2 is provided with a marker M on which the center C overlaps when the screens are aligned. The marker M is attached to the center setting position preset in the imaging area.

In the illustrated state, the center C of the captured image P0 is shifted from the marker M. Also, the outline of the specific area SA is shifted from the baseline BL. Therefore, the screen is not in alignment. Therefore, the operator first manually adjusts the angle of the camera 30 within a possible range to bring the screens closer to matching.

That is, the operator rotates the camera 30 around the yaw axis L1 and the pitch axis L2 so that the center C of the captured image P0 overlaps the marker M as shown in FIG. 8(A). Up to this point, it can be easily performed manually. However, in the illustrated example, the outline of the specific area SA is shifted from the baseline BL even at this stage. Therefore, if the camera 30 can be rotated around the optical axis L, the deviation can be eliminated. However, in this embodiment, the rotation of the camera 30 around the optical axis L is restricted as described above.

Therefore, as shown in FIG. 8B, the captured image P0 is rotated around the center C (that is, in the rotation direction of the optical axis L of the camera 30) so that the correction unit 158 aligns the contour of the specific area SA with the baseline BL. ) rotate. Thereby, the screen is brought into a matching state, and the target setting error in the coolant injection control is corrected. However, as a result of the captured image P0 being tilted with respect to the reference screen 170 at this time, there is a possibility that the screen display will be uncomfortable.

As shown in FIG. 9A, the grid area GA is set based on an ideal state in which there is no installation error of the camera 30. In this ideal state, the aspect ratio (screen ratio) of the captured image P0 (see dotted line) is 4:3, and the number of grids is 28×21. More specifically, each area surrounded by intersections of a plurality of grid lines arranged in parallel in the vertical direction and a plurality of grid lines arranged in parallel in the horizontal direction constitutes one section of the grid. Each grid line is displayed parallel to the frame of the reference screen 170 (display screen). Alternatively, the reference screen 170 (display screen) may be partitioned so that the frame and the grid lines overlap, but as shown in FIG. A form in which lines are displayed may be used. Since the aspect ratio of the frame of reference screen 170 is also 4:3, the outer frame of captured image P0 contacts the frame of reference screen 170 in this ideal state.

The captured image P0 of this embodiment is an image having 3584×2688 pixels. It is divided into square grids that are easy to understand visually. Then, in this embodiment, it can be divided into 28×21 grids, and 128×128 pixels are present in each grid.
Each grid on the reference screen 170 includes a plurality of pixels forming the captured image P0. In the above ideal state, the size of the grid area GA and the size of the captured image P0 are the same. Therefore, the number of pixels of the grid area GA and the number of pixels of the captured image P0 match, and the grid area GA includes 3584×2688 pixels.

In this embodiment, both the reference screen 170 and the captured image P0 are rectangular (rectangular), and the aspect ratio is 4:3. The grid area GA is positioned so that the edges of the grid are in contact with the inside of the frame of the reference screen 170 . Note that the number of pixels per grid can be appropriately set within a range of, for example, 50×50 to 250×250. One grid is preferably set to a size that allows accumulated chips to be identified by image analysis.

By the way, in such a configuration, if the captured image P0 is rotated around the optical axis, the excess or deficiency of the grid may occur in the peripheral portion of the captured image P0. If the image does not correspond to each grid, it is not possible to accurately determine the degree of chip accumulation.

Therefore, in the grid area GA, the area (see the two-dot chain line) in which the image is definitely included is set as the determination area JA, and only the determination area JA is used for chip determination. That is, as shown in FIG. 9B, the display control unit 160 extracts a region of a specific shape by cutting out the peripheral portion of the captured image P0 after rotating the angle of view. Hereinafter, the region of the image extracted at this time will be referred to as "extracted region P1". The extraction area P1 is an image corresponding to the determination area JA and is part of the captured image P0. In this embodiment, the "specific shape" is a rectangular shape having the same aspect ratio (4:3) as the captured image P0.

Specifically, outside the grid area GA in the upper right area of the captured image P0, the first to fourth grids from the right have an occupation area of 50% or less, and the fifth to tenth grids have an occupation area larger than 50%. 80% or less. The extraction area P1 is set in consideration of the area occupied by each grid. The outer perimeter cut in the captured image P0 is a grid row or grid column including a grid in which the captured image P0 occupies 50% or less of the grid section.

Note that the “cutting” referred to here may be performed by erasing the outer peripheral portion of the captured image P0 and leaving the image portion of the judgment area JA as the data of the extraction region P1, or by removing the judgment area JA without erasing the outer peripheral portion. may be extracted as the data of the extraction region P1.

Since the image processing for shavings determination is performed in units of grids, the display control unit 160 cuts out the outer peripheral portion from the captured image P0 in units of at least one of rows and columns of the grid. In the illustrated example, the rotation angle θ of the captured image P0 is set to 3 degrees. The portion where the minute grid image overlaps is clipped. As a result, the number of grids in the grid area GA (extraction area P1) is 26×19. Since the number of pixels per grid remains unchanged at 128×128, the number of pixels in the grid area GA is 3328×2432.

However, if the extraction area P1, which is a part of the captured image P0, is displayed as it is, it will become smaller than the reference screen 170, and a margin may be formed around it, which may cause discomfort. Moreover, if the captured image P0 is displayed in a form that includes the extraction region P1, the divisions of the grid do not correspond to the reference screen 170, which may cause the operator to feel uncomfortable. Therefore, as shown in FIG. 10A, the extraction area P1 is enlarged to a preset display size and displayed on the reference screen 170. FIG. Here, the image is enlarged to the same scale as the original captured image P0. That is, the extraction region P1, which is smaller than the frame of the reference screen 170 by cutting the outer periphery from the captured image P0, is enlarged and displayed so that the outer frame approaches the frame of the reference screen 170. FIG. At this time, the extraction area P1 is enlarged while maintaining the number of pixels. Therefore, the number of pixels of the image displayed within the frame of the reference screen 170 is reduced from 3584×2688 of the captured image P0 to 3328×2432 of the extraction region P1.

As a result, as shown in FIG. 10B, the extraction area P1 has the same size as the reference screen 170, and is displayed so that the sides of the rectangle are at the same positions as the sides of the rectangle of the original captured image P0. That is, the extraction area P<b>1 is enlarged so that the end of the grid in the grid area GA (that is, the grid area after the outer periphery is cut) approaches the frame of the reference screen 170 .

It should be noted that the recognition and determination of substances (chips) by the recognition unit 156 are performed in the initially set grid area (the number of grids: 26×19). The screen display by the display control unit 160 may be a part of the cutout of the captured image P0.

By the correction process described above, the coordinates of the imaging area displayed on the reference screen 170 can be matched with the coordinates of the imaging area recognized by the software, and the coolant can be jetted toward the target position for correct confirmation. . The position of the grid set by the grid setting unit 152 and the position of the grid recognized by the operator from the image are matched. The position indicated by the operator via the touch panel corresponds to one of the grids, and the area of the image where the grids overlap is set as the coolant injection target (target position).

Even if the outer circumference of the grid area is cut off and displayed as described above, the position of the grid being displayed and the actual position of the imaging area correspond accurately. Therefore, coolant injection control can be maintained with high accuracy.

FIG. 11 is a diagram showing an example of a screen operated by an operator.
A screen 200 shown in FIG. 11A is displayed on the monitor of the operation panel 4 . The screen 200 includes the above-described reference screen 170 and operation screen 202 side by side. The operation screen 202 is provided with an automatic cleaning button 210 , a manual cleaning button 212 , a cleaning path adjustment button 214 and a detail setting button 216 .

The automatic cleaning button 210 is selected when executing the automatic cleaning mode. A manual wash button 212 is selected when executing the manual wash mode. The cleaning path adjustment button 214 is selected when adjusting the cleaning path with coolant. When each button is selected, a pull-down menu is displayed, and an operation item in each cleaning mode can be selected, but detailed description thereof will be omitted.

When the detail setting button 216 is selected, a maintenance button 220, an auxiliary line display button 222, and an image rotation operation section 224 are displayed as shown. When the maintenance button 220 is selected by the operator, the maintenance mode is entered, and the baseline BL is displayed superimposed on the captured image P0. Also, when the auxiliary line display button 222 is turned on, the auxiliary line AL is displayed.

While looking at the reference screen 170, the operator adjusts the yaw angle and pitch angle of the camera 30 so that the center C of the captured image P0 overlaps the marker M as described above. After that, by touching the + button or - button of the image rotation operation section 224, the rotation angle θ of the captured image P0 can be adjusted. The correction unit 158 rotates the captured image P0 clockwise by 0.1 degrees each time the + button is touched by the operator, and rotates the captured image P0 counterclockwise by 0.1 degrees each time the - button is touched. rotate each. As shown in FIG. 11B, when the auxiliary line display button 222 is turned off, the auxiliary line AL is hidden.

FIG. 12 is a flowchart showing the flow of correction processing.
This process is executed when the operator selects the maintenance button 220 .
The display control unit 160 displays the captured image P0 on the reference screen 170 as a maintenance screen (S10), and displays the baseline BL so as to overlap it (S12).

At this time, when the operator turns on the auxiliary line display button 222 (Y in S14), the display control unit 160 displays the auxiliary line AL (S16). If the auxiliary line display button 222 is off (N of S14), the auxiliary line AL is hidden (S18). When an image rotation instruction is issued by operating the image rotation operation unit 224 (Y of S20), the display control unit 160 rotates the captured image P0 (S22).

When the contour of the specific area SA thus overlaps the baseline BL and the operator selects an unillustrated confirm button (Y in S24), the correction unit 158 calculates the determination area JA (S26). That is, an area in which the image is surely included in the grid area GA is set as the judgment area JA according to the inclination of the captured image P0. This determination area JA functions as an "AI inference area" in which chip determination is performed on a grid-by-grid basis.

Based on the determination area JA, the display control unit 160 extracts the extraction area P1 by cutting out the outer periphery from the captured image P0 (S28), adjusts (enlarges) the extraction area P1, and displays it on the reference screen 170. (S30). The correction unit 158 stores this series of correction information as calibration data in the correction information storage unit 146 (S32). This correction information includes the set angle (rotational angle θ) of the captured image P0, the setting of the determination area JA, the enlargement ratio (set magnification) of the extraction area P1, and the like. If the confirmation button is not selected (N of S24), the processing of S26-S32 is skipped.

Then, when a predetermined maintenance termination condition is met, such as when the operator selects another button (Y in S34), the display control unit 160 terminates the display of the maintenance screen (S36). If the maintenance end condition is not satisfied (N of S34), the process returns to S14.

FIG. 13 is a flow chart schematically showing the flow of cleaning control.
When cleaning control is started in either the automatic cleaning mode or the manual cleaning mode, the acquisition unit 150 acquires the captured image P0 (S40). Subsequently, the correction unit 158 reads the correction information (calibration data) stored in the correction information storage unit 146 (S42), and internally reflects the correction process described above on the captured image P0.

That is, the correction unit 158 rotates the captured image P0 at a set angle (S44), sets a determination region (S46), and extracts an extraction image (S48). Subsequently, the extracted image is enlarged by the set magnification (S50) and displayed on the screen as a captured image (S52). Then, when the operator gives an instruction to display the chip accumulation state (Y in S54), the display control unit 160 displays the grid image superimposed on the captured image (S56), and further displays the chip accumulation state. (S58). This chip accumulation state is displayed by means of color coding or the like according to the class determined by the recognition unit 156 .

When the operator instructs the coolant injection position (target) based on the displayed image (Y in S60), the detection unit 154 detects this. The injection control unit 162 sets a coolant injection route based on the injection position (S62), and outputs an injection command to the machining control device 102 to inject the coolant based on the injection route (S64).

Then, when a predetermined condition for ending the cleaning mode is met, such as when the operator selects another button (Y in S66), the display control unit 160 terminates the display of the cleaning operation screen (S68). If the cleaning mode end condition is not satisfied (N of S66), the process returns to S40.

The machine tool has been described above based on the embodiment.
In this embodiment, if there is an installation error in the installation of the camera 30 in the processing chamber 2, the operator (operator) can finely adjust the angle (yaw angle, pitch angle) of the camera 30 while checking the captured image P0. . Although the camera 30 cannot adjust the rotation (roll angle) around the optical axis due to its structure, it can be corrected by rotating the captured image P0. That is, according to the present embodiment, even if there are restrictions on the angle adjustment of the camera 30 in the machine tool 1, it is possible to match the imaging area displayed on the screen with the imaging area recognized by the software. Therefore, even when the operator gives an instruction to wash chips with coolant based on the captured image displayed on the screen, the washing instruction position and the coolant injection position match. That is, it is possible to maintain high accuracy of cleaning control.

In addition, in the present embodiment, when correcting the captured image P0, the extracted region P1 is extracted by cutting out the outer peripheral portion from the captured image P0 tilted by rotation, and is enlarged to a preset display size and displayed. bottom. Therefore, the frame shape of the captured image displayed on the screen can be matched to the screen while maintaining the rectangular frame shape. Therefore, the presence or absence of correction (that is, due to individual differences in installed cameras) does not significantly change the appearance of the captured image, and the corrected image does not cause the operator to feel uncomfortable.

[Modification]
In the above embodiment, the machine tool 1 is described as a multitasking machine, but it may be a turning center or a machining center. It may also be an additional processing machine that processes a material (for example, metal powder) while melting it with a laser. In that case, the material that scatters during processing becomes the "physical quantity" recognized by the substance recognition unit.

In the above embodiment, the coolant is exemplified as the fluid that is injected for chip removal. In a modification, liquid (cleaning liquid) other than coolant or gas such as air may be used. In that case, a gas injection section for injecting gas is provided instead of the liquid injection section.

In the above embodiment, an example was shown in which the rotation angle θ of the captured image P0 was 3 degrees in the correction process (FIG. 8B). stomach. When the aspect ratio of the extraction region P1 is 4:3, for example, when θ=1°, the number of grids is 27×20, when θ=2°, the number of grids is 26×20, and when θ=4°, When the number of grids is 25×19 and θ=5°, the number of grids can be 25×18.

FIG. 14 is a diagram showing correction processing when the rotation angle θ of the captured image P0 is 3.3 degrees.
In this example, the captured image P0 has a shape slightly larger than the frame of the reference screen 170 when the rotation angle θ=0°, and its aspect ratio is the same as that of the reference screen 170 (4:3 in this example). . The number of pixels of the captured image P0 is 3584×2688. The number of pixels of the reference screen 170 (display screen) is 3337×2500, which is smaller than the number of pixels of the captured image P0. As shown in FIG. 14 , when the rotation angle θ of the captured image P0 with respect to the reference screen 170 is 3.3°, the entire captured image P0 cannot fit within the reference screen 170 . Therefore, the peripheral portion of the captured image P0 is cut off, and a partial region of the captured image P0 that fits within the reference screen 170 as the maximum rectangle is extracted as an extraction region P1. In other words, the extraction region P1 is a residual partial image obtained by cutting the outer peripheral portion of the captured image P0 after rotation.

The outline of the extraction area P1 overlaps the outline of the grid area GA. That is, of the grids displayed on the reference screen 170 and superimposed on the captured image P0, the grid area GA is an area that includes the ends of the grid, that is, the area that includes the entire grid area. In other words, the grid area GA is left in units of rows and columns of the grid arrangement. In this case, the number of grids is 26×19. This grid area GA is an area in which predetermined determination processing (processing such as chip determination using a learning model) is performed in grid units. In this modified example, this grid area GA is defined as an extraction area P1. Since the extraction area P1 is inside the reference screen 170 (reference screen), the number of pixels of the extraction area P1 is smaller than the number of pixels of the reference screen 170, which is 3328×2432.

The extraction area P1, which is smaller than the frame of the reference screen 170 by cutting the outer periphery of the captured image P0 in this way, is displayed so that the outline of the extraction area P1 expands toward the frame of the reference screen 170 ( See Figure 10). The display control unit 160 stops enlarging the extraction region P1 when the upper outline of the extraction region P1 moves to a position where it overlaps or touches the frame of the reference screen 170 . In this modification, the extraction region P1 is enlarged so that one of the two pairs of parallel sides forming the outline of the extraction region P1 is in contact with the frame of the reference screen 170 and the other is positioned inside the frame of the reference screen 170. be done. Visually, if the upper frame is in place, even if the bottom and left and right are blank, the sense of incongruity can be reduced somewhat. At this time, since the extraction area P1 is enlarged while maintaining the number of pixels thereof, the number of pixels of the image displayed within the frame of the reference screen 170 is reduced.

Note that the “extraction and enlargement of the extraction region P1 by cutting out the outer periphery of the captured image P0” referred to here may be a form in which the outer periphery is hidden and then enlarged. may not be displayed on the reference screen 170 as a result of enlarging the .

In the above embodiment, the information processing device 100 is used as the internal computer of the machine tool 1, and an example is shown in which it is configured integrally with the operation panel 4, but it may be configured independently of the operation panel 4. In that case, the information processing apparatus may use the monitor of the control panel as a remote desktop to function as a display unit. Alternatively, the information processing device may be an external computer connected to the machine tool. In that case, the information processing device may be a general laptop PC (Personal Computer) or a tablet computer.

FIG. 15 is a diagram showing the configuration of an information processing system according to a modification. In addition, the same code|symbol is attached|subjected about the structure similar to the said embodiment, and the detailed description is abbreviate|omitted.
In this modification, the information processing device is installed outside the machine tool. That is, the information processing system includes a machine tool 301 , a data processing device 310 and a data storage device 312 . The data processing device 310 functions as an "information processing device".

The machine tool 301 includes a machining control device 102 , a machining device 104 , a tool changer 106 , a tool storage 108 , a data processor 330 , an operation panel 304 and an imaging unit 110 . Data processing unit 330 includes communication unit 332 , grid setting unit 152 , detection unit 154 , recognition unit 156 and injection control unit 162 . Communication unit 332 has a receiving unit and a transmitting unit, and is in charge of communication with external devices including data processing device 310 and data storage device 312 .

The operation panel 304 includes a user interface processing section 130 and a data processing section 334 . The data processing unit 334 outputs a control command to the processing control device 102 based on the operation input by the operator. Further, the screen displayed on the display unit 144 is controlled according to the operation input by the operator.

The data processing device 310 includes a communication section 320 , a correction section 158 and a display control section 160 . The communication unit 320 has a receiving unit and a transmitting unit and takes charge of communication with the machine tool 301 . Data storage device 312 includes correction information storage section 146 and reference information storage section 148 . The data storage device 312 is wired to the data processing device 310 in this modification, but may be wirelessly connected. In other variations, data storage device 312 may be incorporated as part of data processing device 310 .

In this modification, the functions of the information processing apparatus 100 in the above embodiment are realized by dividing them into the inside and the outside of the machine tool. With such a configuration, it is possible to obtain the same effects as those of the above-described embodiment.

Although not described in the above embodiment, the size and shape of the grid may be configured to be changeable as needed. Also in this case, it is preferable to obtain an extracted image by cutting out the outer peripheral portion of the captured image in units of grids.

In the above embodiment, an example was shown in which a worker (operator) manually adjusts the angles around the yaw axis and the pitch axis of the camera 30 in order to eliminate the mounting error of the camera 30 . In a modification, regarding the angle control of the camera, a drive control section may be provided that controls the first rotation about the yaw axis and the second rotation about the pitch axis. The drive control unit controls the first rotation and the second rotation so that the optical axis of the camera is positioned at a center setting position (marker M) preset in the imaging area prior to correction by the correction unit. .

In the above-described embodiment, the configuration in which the camera 30 cannot rotate around the optical axis is exemplified as the restricted state around the optical axis of the camera 30 . In a modification, the camera 30 may be rotated around the optical axis by a predetermined amount, but the angle of rotation may be limited. Alternatively, the camera 30 itself may have a rotatable structure, but its rotation may be restricted to avoid interference with surrounding structures.

In the above embodiment, as shown in FIGS. 7 and 8, the configuration in which the center C of the captured image P0 and the optical axis L of the camera 30 are aligned and the captured image P0 is rotated around the optical axis L is illustrated. . In a modified example, the captured image may be rotated independently of the optical axis. Also in this case, the correction unit rotates the captured image so that the contour of the specific area is aligned with the baseline. Specifically, it is preferable to rotate around an orthogonal axis that is orthogonal to the plane of the captured image and passes through the center of the captured image.

FIG. 16 is a diagram showing an image processing method according to a modification.
In this modified example, the captured image shown in the upper part of FIG. 16 can be corrected as shown in the lower part of FIG. That is, the grid setting unit 152 performs processing (grid line division processing) to superimpose grid lines on preset positions of the captured image received from the imaging unit 110 . When such processing is performed, the overlapping relationship between the grid lines and the captured image P10 may be as shown in the upper diagram of FIG. In the data obtained by superimposing the grid lines and the captured image P10, there is a grid area where grid lines do not appear on the four sides of the outermost grid overlapping the captured image P10. In such a case, the display control unit 160 ignores the grid area whose four sides do not overlap the captured image, and displays only the grid area whose four sides overlap the captured image P10 according to the size of the reference screen. Processing can be performed (bottom of FIG. 16). In addition, the grid setting unit 152 detects that there is a grid area whose four sides do not overlap the captured image P10 (or a grid area whose four sides do not appear), and the grid is formed in a mesh shape so that the four sides overlap the captured image P10. A process of adjusting by moving the entire grid line up, down, left, or right may be performed.

In the above embodiment, chips present in the imaging area were exemplified as an object to be imaged by the imaging unit, but there is no particular limitation as long as it is a physical quantity such as a material or contaminant generated along with processing.

In the above embodiment, after rotating the captured image, a part of the captured image is extracted by cutting the outer periphery. Then, an example is shown in which the extraction area is displayed such that its outline expands toward the frame of the reference screen. In the modified example, a part of the captured image is extracted by cutting off the outer periphery without rotating the captured image, and the extraction area is displayed so that the contour expands toward the frame of the reference screen. good too. For example, when the physical quantity to be imaged exists near the center of the captured image, that is, when it exists in the outer peripheral portion to a negligible extent, the outer peripheral portion may be cut off. The extracted area may be enlarged and displayed in the same manner so that the operator does not feel uncomfortable due to the clipping.

The above-described embodiment and this modification are based on the premise that processing is performed to extract a portion of the captured image in such a manner that the peripheral portion of the captured image is cut off. Therefore, it is possible to prevent the operator from feeling uncomfortable. That is, it is possible to solve the problem of not giving the sense of discomfort. For this reason, a process of "displaying the extraction area so that its outline expands toward the frame of the reference screen" is performed. Since the factor that gives discomfort is "cutting off the outer peripheral part of the captured image", it may not be essential whether the captured image is obtained for removing chips, which is one of the application examples of the present invention. can be one.

In addition, by not displaying an incomplete area such as "the peripheral portion includes a grid in which the captured image occupies 50% or less of the grid section", the size of the grid section is constant, Since the target area is also the same, the operator can operate while viewing the screen without feeling discomfort. Moreover, if the size of the grid is the same, the internal structure of the machine tool can also be seen in the same size, so the operator does not feel uncomfortable. When used for the purpose of recognizing the state of accumulation of chips, the effect becomes even more pronounced.

It should be noted that even if the object of recognition based on the captured image is color or temperature, the operator will feel uncomfortable if the color or temperature is displayed in an incomplete area. In other words, the above problems can be solved even when colors, temperatures, and the like are displayed. In addition, the accumulation of chips varies depending on the internal structure of the machine tool, but even if it is a photographed image of the internal structure of the machine tool, if the size displayed on the screen changes, the operator will feel uncomfortable. Conceivable. Therefore, according to the above embodiments and modifications, the above problems can be solved.

It should be noted that the present invention is not limited to the above embodiments and modifications, and can be embodied by modifying the constituent elements without departing from the scope of the invention. Various inventions may be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments and modifications. Also, some constituent elements may be deleted from all the constituent elements shown in the above embodiments and modifications.

This patent application claims priority to Japanese Patent Application No. 2021-144425 (filed on September 6, 2021) and Japanese Patent Application No. 2022-072847 (filed on April 27, 2022), the entirety of which is incorporated herein by reference. shall be incorporated into this document.

Claims

an acquisition unit that acquires an image of an imaging area captured by an imaging unit installed in a machine tool;
a grid setting unit that divides the captured image into grids each having a size that includes a plurality of pixels;
a display control unit for displaying, on a reference screen of a display unit, an extraction region that is a part of the captured image, which is extracted by cutting out the outer peripheral portion of the captured image in units of rows and columns of the grid arrangement;
with
The display control unit displays the extraction area, which has become smaller than the frame of the reference screen by cutting the outer peripheral portion from the captured image, such that the outline of the extraction area expands toward the frame of the reference screen. Information processing device to display on the part.
The information processing apparatus according to claim 1, further comprising a recognition unit that recognizes a predetermined physical quantity in the imaging area in units of the grid based on the extraction region.
The information processing apparatus according to claim 2, wherein the display control unit causes a display indicating the physical quantity recognized within the imaging area to be superimposed on the extraction region.
the frame of the reference screen is rectangular and the extraction area is also rectangular;
2. The display control unit, when displaying the enlarged extraction area, causes the sides of the rectangle of the enlarged extraction area to be displayed at the same positions as the sides of the rectangle of the frame of the reference screen. 1. The information processing device according to 1.
a reference information storage unit that stores in advance a line along the contour of a specific area of the machine tool in the display image;
a correction unit that rotates the captured image in a rotational direction of the optical axis of the imaging unit so that the outline of the specific area displayed in the captured image is aligned with the line;
further comprising
2. The information processing apparatus according to claim 1, wherein said display control unit enlarges and displays said extraction region extracted by cutting out an outer peripheral portion of the captured image after correction by said correction unit.
The information processing apparatus according to claim 1, wherein the outer peripheral portion includes a grid in which a portion of the section of the grid occupied by the captured image is 50% or less.
The information processing apparatus according to claim 1, wherein the outer peripheral portion is a grid row or a grid column including a grid in which the portion occupied by the captured image in the division of the grid is 50% or less.
The information processing apparatus according to claim 1, wherein the display control unit stops enlarging the extraction area when the upper outline of the extraction area moves to a position where it overlaps or touches the frame of the reference screen.
an imaging unit that images an imaging area set in the processing chamber;
a display unit for displaying an image captured by the imaging unit;
a grid setting unit that divides the captured image into grids each having a size including a plurality of pixels;
a display control unit for displaying, on a reference screen of a display unit, an extraction region that is a part of the captured image, which is extracted by cutting out the outer peripheral portion of the captured image in units of rows and columns of the grid arrangement;
A machine tool comprising
The display control unit displays the extraction area, which has become smaller than the frame of the reference screen by cutting the outer peripheral portion from the captured image, such that the outline of the extraction area expands toward the frame of the reference screen. A machine tool displayed in the department.