JP2022120667A - Device identification system, device identification method, and device identification program - Google Patents

Abstract

To allow for configuring auxiliary devices, such as workpiece turnover devices, for assisting work on workpieces loaded in a loading area in a freely placeable manner, and accurately identifying placement positions and orientations thereof.SOLUTION: A device identification system provided herein comprises: freely placeable, portable auxiliary devices for assisting work on workpieces loaded in a loading area; an image capturing device capable of capturing an image of the loading area; and a control device configured to identify placement positions and orientations of the auxiliary devices in the loading area using the captured image captured by the image capturing device. Each auxiliary device comprises a device main body, a movable unit capable of moving relative to the device main body and contacting a workpiece in the loading area, and indicators provided at least on the movable unit so as to be present in the captured image and designed to enable identification of a position and orientation of the auxiliary device, where the control device is configured to detect the indicators in the captured image and identify the placement position and orientation.SELECTED DRAWING: Figure 1

Description

The present invention relates to a device identification system, a device identification method, and a device identification program.

2. Description of the Related Art Conventionally, workpieces such as a plurality of plate-like members loaded in a loading area are supplied to a processing machine such as a press brake for bending by a loading device such as a supply robot. When loading, for example, a work such as a magnetic floater that floats a work by magnetic force and separates it from other works, or an air separator (for example, see Patent Document 1) that separates a work from other works by air injection pressure. A turning device (single workpiece picking device) is used.

JP 2017-149496 A

Usually, the conventional work turning device including the air separator described in Patent Document 1 is provided in a state of being fixed to a setting table installed at a position adjacent to the loading area. For this reason, a plurality of works loaded on the loading area are placed at one place, and one work turning device is arranged on the installation table so as to correspond to the works.

However, since the work turning device is fixedly arranged on the installation table, it cannot actually be installed at an arbitrary position with respect to the work. It is done. For this reason, when a sufficiently large work is to be loaded in the loading area, it may be inconvenient due to the space problem of the loading area in order to load the work in correspondence with the installation location of the work turning device. Therefore, there was a demand to install the work turning device at an arbitrary location instead of fixing it.

However, in the case of loading a work that is sufficiently large for the loading area as described above, even if the work turning device can be installed at any location, it can be installed at any location around the loading area. There is a need to. For this reason, even if an attempt is made to detect the work turning device by imaging the stacking area with an imaging device, the work turning device cannot be captured completely within the imaging range, making it difficult to properly detect the device itself. It was very difficult to determine the position and orientation of the placement.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned circumstances, and an auxiliary device such as a work-turning device for assisting work on a work loaded in a loading area can be arranged freely, and the arrangement position and arrangement direction can be changed. It is an object of the present invention to provide a device discrimination system, a device discrimination method, and a device discrimination program capable of making an accurate discrimination.

A device discrimination system according to the present invention includes an auxiliary device having portability that can be arranged freely to assist work on a work loaded in a loading area, an imaging device capable of imaging the loading area, and an imaging device imaging by the imaging device. and a control device that determines the arrangement position and arrangement direction of the auxiliary device with respect to the loading area based on the captured image, wherein the auxiliary device moves with respect to the device main body and the device main body. a movable portion provided to be able to contact the work in the loading area, and an index capable of identifying the position and direction of the auxiliary device provided at least on the movable portion appearing in the captured image. Then, the control device detects the index included in the captured image, and determines the placement position and the placement direction.

In one embodiment of the present invention, the index is provided on the upper surface side near the tip of the movable portion.

In another embodiment of the present invention, the control device compares the detected index with an index indicated by a pre-stored index model image to determine the arrangement position and orientation of the arrangement.

In still another embodiment of the present invention, the loading area is formed in a rectangular shape in a plan view, the auxiliary device is arranged so that the device main body is positioned outside the loading area, and the control device comprises the From the captured image, the index included in a pair of intersecting long-side and short-side outer peripheral inner rectangular areas in the stacking area is detected, and the placement position and the placement direction are determined.

In still another embodiment of the present invention, a plurality of the auxiliary devices are provided, and the control device determines the arrangement position and arrangement of each auxiliary device with respect to the loading area determined based on the detected index. Determine placement conditions, including orientation.

In still another embodiment of the present invention, the auxiliary device has a top surface portion on the device main body, and the indicator is provided on the top surface portion of the device main body.

In still another embodiment of the present invention, the auxiliary device has a slant portion on the device main body that inclines from the top surface portion, and the indicator is provided on the slant portion of the device main body.

In still another embodiment of the present invention, the indicator provided on the slope portion is provided at a location most distant from the top surface portion.

In still another embodiment of the present invention, the indicator provided on the top surface is larger than the indicators provided on the movable section and the slope section, and the indicator provided on the slope section is larger than the indicator provided on the slope section. It is smaller than the indicators provided on the movable portion and the top surface portion.

In still another embodiment of the present invention, the auxiliary device has a pair of side surface portions connected to the top surface portion on the device main body, and the indicators are provided on the pair of side surface portions of the device main body. .

In still another embodiment of the present invention, the index is light transmissive, and the auxiliary device has light sources inside the movable part and the device main body, respectively.

In still another embodiment of the present invention, the control device detects the auxiliary device placed in the loading area based on the captured image, and determines the placement position and the placement direction based on the detected index. discriminate.

In still another embodiment of the present invention, the control device detects the work in the loading area based on the captured image, and operates an auxiliary device on the work based on the result of determination of the placement situation. identify.

In still another embodiment of the present invention, a detection means capable of detecting the auxiliary device stored in a storage area provided at a location separated from the loading area is provided, and the control device receives a signal from the detection means. Based on the detection result, the storage status of the auxiliary device in the storage area is determined.

In still another embodiment of the present invention, an informing means capable of informing a predetermined alarm is provided, and the control device controls the informing means based on the determination result of the arrangement status and/or the storage status. The predetermined warning is notified.

The device identification method according to the present invention includes a step of capturing an image of the loading area around which at least an auxiliary device having portability that can be freely arranged for assisting work on the work loaded in the loading area is arranged, by an imaging device. and determining the arrangement position and arrangement direction of the auxiliary device with respect to the loading area by the control device based on the captured image captured by the imaging device. The auxiliary device having a movable portion provided so as to be able to move with respect to the apparatus main body and contact the work in the loading area, wherein the auxiliary device is provided at least in the movable portion shown in the captured image. An index that can identify the position and direction is detected from the captured image, and the arrangement position and the orientation of the arrangement are determined.

In one embodiment of the present invention, in the step of determining, from the imaged image, in a pair of intersecting long side and short side outer peripheral inner rectangular regions in the stacking area formed in a rectangular shape in a plan view, The indices included are detected to determine the placement position and the placement orientation.

In another embodiment of the present invention, in the step of capturing an image, the step of capturing an image of the loading area in which a plurality of the auxiliary devices are arranged such that the device body is positioned outside the loading area, and in the determining step, A placement situation including the placement position and the placement direction of each auxiliary device with respect to the loading area determined based on the detected index is determined.

In still another embodiment of the present invention, in the imaging step, the loading area in which a plurality of the auxiliary devices are further arranged is imaged, and in the determining step, based on the detected index. An arrangement state including an arrangement position and an arrangement direction of each auxiliary device in the determined loading area is determined.

In still another embodiment of the present invention, an auxiliary device that detects the work in the loading area based on the captured image by the control device and operates on the work based on the determination result of the placement situation is provided. including the identifying step.

In still another embodiment of the present invention, detecting means for detecting the auxiliary device stored in a storage area provided at a location separated from the loading area; and determining, by the control device, the storage status of the auxiliary device in the storage area.

In still another embodiment of the present invention, based on the determination result of the arrangement status and/or the storage status, the control device controls an informing means capable of informing a predetermined alarm to issue the predetermined alarm. including the step of allowing

A device determination program according to the present invention causes a computer to detect, by an imaging device, a loading area in which at least an auxiliary device having a freely arrangable portability for assisting work on a work loaded in the loading area is arranged around the loading area. A device determination program for executing a step of acquiring a captured image obtained by imaging, and a step of determining the arrangement position and arrangement direction of the auxiliary device with respect to the loading area based on the acquired captured image. wherein, in the determining step, the apparatus main body and the auxiliary device having a movable portion provided so as to be able to move relative to the apparatus main body and come into contact with the work in the loading area are reflected in the captured image. An index capable of identifying the position and direction of the auxiliary device, which is provided at least on the movable portion, is detected from the captured image to determine the arrangement position and the orientation of the arrangement.

According to the present invention, an auxiliary device, such as a work turnover device, for assisting work on a work loaded in a loading area can be arranged freely, and its arrangement position and orientation can be determined accurately.

1 is a perspective view schematically showing a configuration example of a device discrimination system according to one embodiment of the present invention; FIG. It is a perspective view which shows roughly the auxiliary device in the same apparatus discrimination system. It is a figure which shows an example of the index provided in the same auxiliary device. It is a figure which shows the other example of the same index. It is a block diagram which shows roughly the functional structure of the control apparatus in the same apparatus discrimination system. It is a figure which shows the image of the index model image memorize|stored in the memory|storage part of the same control apparatus. It is a block diagram which shows roughly the hardware constitutions of the same control apparatus. FIG. 4 is a plan view showing an image of an index detected from a captured image captured by an imaging device in the same control device; FIG. 11 is a perspective view schematically showing another configuration example of the same device discrimination system; FIG. 4 is a plan view showing an image of an index detected from a captured image in the same device discrimination system; It is a figure which shows the example of some setting screens displayed on the display of the same control apparatus. 4 is a flow chart showing an example of operations associated with determination processing of the same control device; 9 is a flow chart showing another example of operations associated with determination processing of the same control device;

DESCRIPTION OF THE PREFERRED EMBODIMENTS Device identification systems, device identification methods, and device identification programs according to embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, the following embodiments do not limit the invention according to each claim, and not all combinations of features described in the embodiments are essential to the solution of the invention. . In addition, in this embodiment, the scale and dimensions of each component may be exaggerated, and some components may be omitted.

FIG. 1 is a perspective view schematically showing a configuration example of a device discrimination system 100 according to one embodiment of the present invention. As shown in FIG. 1, the apparatus determination system 100 according to the present embodiment is an auxiliary system that assists work (for example, loading to a bending machine or the like) on the work W loaded in the loading area TA on the pallet P. A workpiece turning device 10 as a device and a camera 20 as an imaging device capable of imaging the loading area TA are provided.

In addition, the device determination system 100 has functions such as an NC device that controls the entire device determination system 100, and performs overall image processing such as image analysis and image correction based on the captured image captured by the camera 20. A control device 30 that has the function of a processing device and determines the placement position and orientation of the work turning device 10 with respect to the loading area TA is provided.

In this embodiment, the control device 30 includes a display 69 as a display device and an input section 35a including an input device such as a keyboard and a mouse. 35a can be replaced with equivalent functions (for example, display means or input means that can be used remotely), and is not limited to this.

Further, in the present embodiment, for example, processing machines such as bending machines and loading devices such as workpiece holding robots are not shown, but these processing machines, loading devices, etc. are controlled by the control device 30. It may optionally be included in the device identification system 100 .

In this embodiment, a storage area 50 is provided at a location separated from the loading area TA of the pallet P for storing the required number of work turning devices 10 corresponding to the works W loaded on the loading area TA. ing. The storage area 50 is provided with a storage sensor 51 which is a detecting means for individually detecting the stored work turning devices 10 . The control device 30 can determine the storage status of the work turning device 10 in the storage area 50 based on the detection signal from the storage sensor 51 .

Further, in the present embodiment, the work turning device 10 will be described as an example of an auxiliary device, but the auxiliary device is limited to this as long as it can assist various operations on the work W. is not. Furthermore, although illustration is omitted, the workpiece turning devices 10, the camera 20, and the control device 30 are connected by wire or wirelessly so that they can transmit and receive data and signals to each other.

The work-turning device 10 is freely arranged and portable, and can be freely arranged inside or outside the loading area TA so as to correspond to the works W arranged in the loading area TA on the pallet P. In this embodiment, the work turning device 10 can be arranged to correspond to the work W from outside the loading area TA on a one-to-one basis.

More specifically, the work turning device 10 of this embodiment is arranged as follows, for example. In general, another auxiliary device (not shown) such as a magnet floater is installed on one long side of the loading area TA, which is rectangular in plan view. An end face bar (not shown) for abutting the work W is installed on one of the short sides. The work turning device 10 is arranged on the long side or the short side of the loading area TA together with other auxiliary devices and end face bars so that at least the turning unit 10B can be present in the loading area TA.

That is, in FIG. 1, the work turning device 10 is arranged on one long side of the loading area TA where the large-sized works W are loaded so that the device body 10A is positioned outside the loading area TA. may be similarly arranged on one short side, for example, as indicated by a two-dot chain line.

FIG. 2 is a perspective view schematically showing the workpiece turning device 10 in the device discrimination system 100. As shown in FIG. As shown in FIG. 2, the work turning device 10 includes a device main body 10A having a rectangular outer shape when viewed from above, and a device main body 10A which is movable relative to the device main body 10A and capable of contacting and holding a work W in a stacking area TA. A turning unit 10B, which is a movable part provided in a so-called cantilevered manner with respect to the apparatus main body 10A, is provided.

The turning unit 10B of the work turning device 10 lifts the uppermost work W among the plurality of plate-like works W loaded in the loading area TA on the pallet P by, for example, air jet pressure and adsorption force. , has a function of separating from other works W by turning over them. As described above, the work turning device 10 can be used as a one-sheet picking device for picking up one work W by the turning unit 10B. omitted. In the work turning device 10 of this embodiment, the side on which the turning unit 10B is arranged is defined as the front side of the device.

A device main body 10A of the work turning device 10 includes a top surface portion 11 provided on the upper side of the device, a slope portion 12 inclined at two left and right points on the front side of the device from the top surface portion 11, the top surface portion 11 (and the slope 12) and a pair of side portions 13 connected on the left and right sides of the device. Markers 14, 15, and 16 are provided at predetermined locations on the top surface 11, slope 12, and pair of side surfaces 13 of the apparatus main body 10A, and on the turning unit 10B as indicators capable of identifying the position and direction of the workpiece turning apparatus 10, respectively. , 17 are provided.

Specifically, a plurality of markers 14 and 16 are provided at predetermined locations on the top surface portion 11 and the pair of side surface portions 13 of the apparatus main body 10A, and a plurality of markers 15 are provided at predetermined locations on the slope portion 12 of the apparatus main body 10A. ing. Also, the marker 17 is provided near the tip of the upper surface portion 10a of the turning unit 10B.

More specifically, the markers 14 are provided on the top surface 11 of the apparatus main body 10A at two locations on the left and right near the rear side of the apparatus, and the markers 16 are provided on the pair of side surfaces 13 of the apparatus main body 10A above the apparatus. They are located near each other. That is, the marker 16 is provided to the side of the marker 14 and directly below it. In addition, the markers 15 are provided in the vicinity of both end portions in the inclination direction of the two slope portions 12 of the apparatus main body 10A.

FIG. 3 is a diagram showing an example of the markers 14-17 provided in the work turning device 10, and FIG. 4 is a diagram showing another example of the markers 14-17. The markers 14, 15, 16 provided on the device main body 10A of the work turning device 10 and the marker 17 provided on the turning unit 10B are composed of, for example, AR (augmented reality) markers including known ArUco markers, ChArUco markers, and the like. can do.

As shown in FIG. 3, the marker 17 (14 to 16) is a film-like member or plate-like member having at least an overall area 18 representing the entire marker and an identification area 19 for identifying the position and orientation of the marker. It is composed of members. 1 and 2, the identification regions 19 of the markers 14 to 17 are omitted.

The shape of the identification area 19 of the markers 14 to 17 is, for example, a shape obtained by combining simple rectangles in the example shown in FIG. 3, but as shown in FIGS. Any shape that can identify the position and direction of the workpiece turning device 10 at least by itself, such as combining squares or changing the position of the identification area 19 within the entire area 18, is limited to those shown in the drawings. Instead, various shapes, such as more complex shapes and simpler shapes, can be employed.

Further, the dimensions of the markers 14 to 17 (the size and shape of the entire area 18 and the identification area 19, etc.) can be set according to the number of pixels of the camera 20 and imaging performance such as imaging resolution. It is sufficient that the position and direction of arrangement of the workpiece turning device 10 can be sufficiently determined from the image taken from the image pickup device 10 .

In this embodiment, at least one marker 17 should be provided in the turning unit 10B. This is because only the turning unit 10B appears in the captured image when the work turning device 10 is normally arranged around the stacking area TA. If the workpiece turning device 10 is arranged within the stacking area TA, at least one marker 14 out of the markers 14 to 16 should be provided on the top surface 11 of the device main body 10A. This is because, in this case, the top surface 11 of the apparatus main body 10A is most likely to appear in the captured image.

When markers 14 to 16 are provided on device main body 10A, markers 14 and 15 are provided in pairs at a plurality of locations on top surface 11 and slope portion 12, respectively, so that markers 14 and 15 can be detected from captured images. Accuracy and recognition accuracy can be further improved, but the present invention is not limited to this.

Further, from the viewpoint of further improving the detection accuracy, etc., the marker 17 is provided at a location farthest from the device main body 10A, that is, at a location near the tip of the turning unit 10B as described above. The marker 15 provided on the portion 12 may be provided at a location farthest from the top surface portion 11, that is, at a location closest to the front end of the device in the inclination direction. As a result, the markers 17 of the turning unit 10B, the markers 14 of the top surface 11, and the markers 15 of the slanted surface 12 can be arranged as far apart as possible, thereby reducing erroneous detection (or detection error). can.

In order to make the marker 14 provided on the top surface 11 easier to detect, both the overall area 18 and the identification area 19 are larger than the markers 17 and 15 provided on the turning unit 10B and the slope 12. It should be something. Thus, if the marker 14 on the top surface 11 is large, it can be easily detected even if the imaging performance of the camera 20 is slightly lower than that generally used in this field. As shown in FIG. 2 and the like, the marker 15 provided on the slope portion 12 is arranged so that both the entire area 18 and the identification area 19 are smaller than the markers 17 and 14 provided on the turning unit 10B and the top surface portion 11. It is configured.

Note that the markers 16 provided on the pair of side surfaces 13 of the device main body 10A are not displayed even if the work turning device 10 falls due to some accident (for example, it collides with a robot arm of a loading device (not shown), etc.). are provided to enable identification of the position and direction of the work turning device 10 (in this case, the placement position and placement direction when the marker 16 is overturned so that the marker 16 appears in the loading area TA).

On the other hand, the reason why the markers 16 are not provided on the front and back portions of the device main body 10A connected to the pair of side portions 13 is that even if the work turning device 10 falls over, the outer shape of the device main body 10A is This is because there is almost no possibility that the front part and the back part will stand still in the upward direction. Needless to say, the markers 16 may be provided on the front surface and rear surface of the apparatus main body 10A depending on the external shape of the apparatus main body 10A.

Further, the markers 14 to 17 are configured to have optical transparency so that the entire area 18 and the identification area 19 can be well recognized even in a captured image captured in a dark environment, for example. Also good. When the markers 14 to 17 are light transmissive, the work turning device 10 further includes light sources (LEDs, etc.) (not shown) inside the turning unit 10B and the device main body 10A (in this embodiment, at least inside the turning unit 10B). ).

In this way, even if lighting equipment, which will be described later, is turned off in a dark environment such as at night, if the light sources in the turning unit 10B and the apparatus main body 10A are turned on, the markers 14 to 17 can be displayed in the captured image. Since it can be clearly recognizable, at least the marker 17 (and the markers 14 to 16) of the turning unit 10B of the work turning device 10 in the loading area TA can be seen in the dark.

On the other hand, the camera 20 is configured by, for example, an inexpensive and highly versatile monocular camera (that is, one camera), and is configured to capture the entire loading area TA as an imaging range. It is arranged directly above the center via a supporting member such as a camera stand 21 . Although not shown, lighting equipment having a plurality of light-emitting diodes (LEDs) or the like is provided at a predetermined location around the pallet P to illuminate the works W loaded in the loading area TA. there is

When the camera 20 picks up an image of the loading area TA, by irradiating the work W with irradiation light from this lighting equipment, the brightness of the picked-up image can be appropriately adjusted with proper exposure. Note that if the markers 14 to 17 having the light transmissivity and the light source described above are used, for example, the illumination equipment can be provided at a position where the illumination light from the illumination equipment does not reach the top surface portion 11 of the device main body 10A. Even in this case, the markers 14 to 17 in the captured image can be highlighted in a sufficiently recognizable manner.

FIG. 5 is a block diagram schematically showing the functional configuration of the control device 30 in the device discrimination system 100, and FIG. be. 7 is a block diagram schematically showing the hardware configuration of the control device 30, and FIG. 8 is a plan view showing an image of the marker 17 detected from the captured image captured by the camera 20 in the control device 30. is.

As shown in FIG. 5 , image data of a captured image captured by camera 20 is input to control device 30 . The control device 30 functionally includes an image acquisition unit 31 , an image processing unit 32 , a calculation unit 33 , a storage unit 34 and an operation unit 35 . Determination of the placement position and placement direction of the workpiece turning device 10 with respect to the stacking area TA (such processing may be hereinafter referred to as “determination processing”) is mainly performed by the image processing section 32 of the control device 30. done by

Here, the image acquisition unit 31 acquires image data of the captured image output from the camera 20 . The image processing unit 32 includes, for example, a marker detection unit 32a and a matching processing unit 32b, and performs edge detection processing (edge detection processing) using binarization processing or the Sobel method on the image data acquired by the image acquisition unit 31. ), morphology processing, approximation processing using Newton's method, various image conversion and image analysis processing including template matching processing (such processing is hereinafter collectively referred to as "image processing"), and the above discrimination processing to run.

Based on the determination result from the image processing unit 32, the computation unit 33 performs various computation processes for controlling the entire device determination system 100 including the workpiece turning device 10, and outputs a control output to each unit. The storage unit 34 stores various types of data used by the control device 30, such as CAD data representing the workpiece W and the workpiece turning device 10, image data of model (marker model) images serving as references for the markers 14 to 17, and various program data. Stores data in a readable and writable manner.

Note that, as shown in FIG. 6, the marker model images 17c ₁ to 17c _n are corresponding images (for example, they appear to be tilted) when the height positions and degree of rotation of the markers 14 to 17 in the workpiece turning device 10 are taken into consideration. A plurality of types of template images are stored in the database (DB) of the storage unit 34, for example, as template images.

Therefore, by comparing and matching the markers 17 (14 to 16) detected from the captured image with the marker model images 17c ₁ to 17c _n , the positions, heights and heights of the workpiece turning device 10 with respect to the stacking area TA can actually be determined. It is possible to determine the posture such as orientation. The operation unit 35 receives an operation input by the user of the control device 30 via the input unit 35a.

As shown in FIG. 7, the control device 30 includes, for example, a CPU 61, a RAM 62, a ROM 63, a HDD (Hard Disk Drive) 64, and an SSD (Solid State Drive) 65 as a hardware configuration. The control device 30 also includes an input I/F (interface) 66 , an output I/F (interface) 67 , and a communication I/F (interface) 68 . Each component 61 to 68 is interconnected by a bus 60, respectively.

The CPU 61 controls the entire device determination system 100 including the control device 30 by executing various programs stored in the RAM 62, ROM 63, HDD 64, SSD 65, and the like. It realizes the functions of the processing unit 32 and the calculation unit 33 .

RAM 62 can be used as a work area for CPU 61 . The ROM 63 stores at least the above various programs in a readable manner. The HDD 64 and SSD 65 store the above-described various data in a readable and writable manner, and implement the function of the storage section 34 together with the RAM 62 and ROM 63 .

The camera 20 is connected to the input I/F 66 to acquire a captured image. Therefore, the input I/F 66 implements the function of the image acquisition section 31 described above. Along with this, the input I/F 66 is connected to a touch panel 69a functioning as the input section 35a of the operation section 35, and receives information accompanying an operation input from the user. The input I/F 66 may also be connected to input means such as a keyboard and a mouse (not shown).

The output I/F 67 is connected to, for example, a display 69 (see FIG. 1) of the control device 30 having a built-in touch panel 69a, and outputs various information to be displayed on the monitor. Note that the control device 30 can be connected to a network such as the Internet (not shown), an external device, or the like via the communication I/F 68 .

Specifically, the marker detection unit 32a of the image processing unit 32 performs edge detection processing or the like to determine the loading area TA as shown in FIG. The marker 17 provided on the upper surface portion 10a of the turning unit 10B of the work turning device 10 is detected from the captured image 22 that has been imaged and image-processed.

When detecting the marker 17, the marker 17 is not detected from the entire captured image 22 in order to reduce the processing load. Focus on the long side and the short side. That is, the marker 17 is detected from the captured image from the detection ranges D1 and D2, which are a pair of intersecting rectangular areas inside the outer periphery in the stacking area TA.

Then, the image processing unit 32 records, in the storage unit 34, position information (pixel positions) of, for example, four corner pixels (pixels) of the marker 17 detected by the marker detection unit 32a. Here, a pixel (picture element) refers to a minimum unit or minimum element having color information (color tone, gradation, etc.) when a captured image is handled by the control device 30 .

Next, for the marker 17 in the captured image 22, the image processing unit 32 obtains dimensional information such as the CAD data stored in the storage unit 34 for each part of the device and the stacking area TA, and the marker model images 17c ₁ to 17c _n . A position model representing the actual position of the marker 17 in the workpiece turning device 10 is obtained based on the image data.

Then, the image processing unit 32 causes the matching processing unit 32b to apply (1) approximation processing such as Newton's method to the position model and the pixel position to determine the physical position (x, y, z position) of the marker 17 within the loading area TA. ) and the rotation state (rz), (2) generate a marker image reflecting the physical position (x, y, z position) and the rotation state (rz) based on the calculation result, and (3) generate the marker The marker 17 indicated by the image is searched with various marker model images 17c ₁ to 17c _n pre-stored in the storage unit 34, for example, by performing template matching processing, and (4) the marker indicated by the searched marker model image is searched. Based on the matching information, the pixel position of the marker 17 stored in the storage unit 34 is updated, that is, by repeating the above processes (1) to (4) a plurality of times, the discrimination accuracy is improved. , the actual position and orientation of the work-turning device 10 with respect to the loading area TA (that is, the placement position and placement direction) are determined.

As described above, when the markers 17 exist within the detection ranges D1 and D2, the detected markers 17 are stored in the storage unit 34 during the processes (1) to (4) in the matching processing unit 32b. Among the marker model images 17c ₁ to 17c _n in the DB, two types of marker model images with an image angle of 0° or 90° are searched for preferential matching, and then the details of the work turning device 10 By specifying the correct orientation (for example, with a rotation angle of several degrees) and determining the arrangement position and arrangement orientation, it is possible to further reduce the processing load of the matching processing unit 32b.

As a result, for example, it is possible to accurately grasp the arrangement position and arrangement direction of the work turning device 10 that is freely arranged with respect to the loading area TA, and to judge the arrangement state. Then, it is possible to reliably perform various operation controls and the like according to the work turning device 10 whose arrangement state has been determined.

As shown in FIGS. 9 and 10, even when a plurality of works W are arranged in the stacking area TA and a plurality of work turning devices 10 are arranged with respect to the works W in a one-to-one relationship, the captured image Instead of detecting the marker 17 from the whole of 22, it is sufficient to focus on the detection ranges D1 and D2 and detect the markers 17 included in each of the detection ranges D1 and D2 and perform discrimination processing. In this way, while reducing the processing load, the placement position and placement direction of each workpiece turning device 10 can be accurately grasped to determine the placement situation, and various operation controls and the like can be performed for each device accordingly. It is also possible to do

The determination process described above in the present embodiment is merely an example, and is not limited to this. The marker 17 or the like is detected by image recognition to identify the position and direction of the work turning device 10, and the position and direction of the work turning device 10 are identified. Various methods can be applied as long as the method can determine the arrangement position and arrangement direction. Further, when the work turning device 10 is also arranged in the stacking area TA depending on the stacking mode of the works W, the above markers 14 to 16 are detected based on the captured image of the stacking area TA, and the determination is made in the same manner. processing should be done.

Next, a description will be given of an operation example when, for example, a plurality of work turning devices 10 are actually arranged inside and outside the loading area TA and operated. In this operation, the control device 30 is provided with an alarm device (not shown) as an informing means capable of informing a predetermined alarm. The alarm device is configured to be able to issue a predetermined alarm by turning on a warning light or outputting the alarm by voice.

FIG. 11 is a diagram showing an example of part of a setting screen displayed on the display 69 of the control device 30, and FIG. Here, in the device identification system 100, a case is described in which a maximum of two work turning devices 10 can be arranged and stored in either the inside or the outside of the loading area TA or the storage area 50. FIG.

As shown in FIG. 11, a setting screen 70 displayed on the display 69 of the control device 30 includes, for example, an operation input area 71 having numeric keys and cursor keys, and various parameters relating to the operation of the workpiece turning device 10. and a parameter setting area 72 for setting. Here, when the work turning device 10 is installed inside or outside the loading area TA, the user operates, for example, the operation input area 71 to set the setting value of "turning device" in the parameter setting area 72 to "ON". do. When the work turning device 10 is not installed in the loading area TA, the setting value is set to "OFF".

That is, in this operation, as shown in FIG. 12, the control device 30 first determines whether or not the arrangement of the work turning device 10 inside and outside the loading area TA is ON. (step S100). If it is determined that the placement of the device is not ON, that is, is OFF (the workpiece turning device 10 is not placed) ("No" in step S100), the processing of this flow chart ends. If it is determined that it is ON (that is, the work turning device 10 is arranged) ("Yes" in step S100), the following processing is performed.

That is, first, an image obtained by imaging the loading area TA with the camera 20 is obtained, and the image data is image-analyzed as described above (step S101). It is determined whether or not (step S102). If it is determined that the markers 17 are not within the detection ranges D1 and D2 ("No" in step S102), it is determined that the workpiece turning device 10 is not placed outside the stacking area TA, and the markers 14 to 14 are detected from the captured image. 16 is detected to perform a device detection process capable of determining the arrangement position and orientation of each of the plurality of work turning devices 10 in the loading area TA (step S103). When it is determined that the markers 17 are within the detection ranges D1 and D2 ("Yes" in step S102), the arrangement position and arrangement direction of the workpiece turning device 10 outside the loading area TA are determined as described above. device detection processing is performed (step S106).

In the device detection process of step S103, if the device is not detected ("No" in step S104), it is determined that the placement of the device in the loading area TA is ON. If there is a problem that the device was not detected (for example, the device was forgotten to be installed), the alarm device is controlled to give an alarm (step S105), and the processing according to this flowchart is terminated.

On the other hand, if the device is detected ("Yes" in step S104), and after the placement status of the work turning device 10 outside the loading area TA is determined in step S106, the Outside the TA, it is determined whether the number of detected workpiece turning devices 10 is one or two (step S107). When it is determined that the number of the detected work turning devices 10 is one ("1" in step S107), for example, based on the captured image, the contour of the work W in the stacking area TA is extracted as a line image. image processing is performed to detect the work W in the stacking area TA (step S108).

Interference calculation for determining whether or not there is interference with the detected workpiece W by the workpiece turning device 10, the robot arm, or the like whose arrangement position and arrangement direction have been determined in step S103 or step S106. is performed (step S109), and the processing according to this flowchart is terminated.

Further, when it is determined that the number of detected workpiece turning devices 10 is two ("2" in step S107), similar to step S108, the workpiece W in the loading area TA is detected (step S110). ), based on the placement status including the placement position and placement direction of each work turning device 10 within the loading area TA or outside the loading area TA determined in step S103 or step S1062, for example, the same It is determined whether or not two work turning devices 10 are installed for the work W (step S111).

If it is determined that two work turning devices 10 are not installed for the same work W ("No" in step S111), interference calculation for determining the presence or absence of interference is performed in the same manner as in step S109. (step S112), and the processing according to this flowchart is terminated.

On the other hand, if it is determined that two work turnover devices 10 are installed for the same work W ("Yes" in step S111), then for one work W within the loading area TA or outside the loading area TA, In spite of the fact that the work turning devices 10 must be arranged one-on-one, if there is a problem in the arrangement mode (overlapping installation of devices, etc.), the alarm device is controlled and an alarm is issued in the same manner as in step S105. Notification is made (step S113), and the processing according to this flowchart is terminated. As described above, according to this operation, when the work turning device 10 is arranged inside and outside the loading area TA, it is possible to prevent the loading of the work W and the like from being performed while the above-mentioned problems are still occurring. It is possible to prevent

FIG. 13 is a flow chart showing another example of operations associated with the determination processing of the control device 30. FIG. It should be noted that, hereinafter, descriptions of parts that overlap with the contents already described will be omitted. Here, an operation based on the number of detected and stored workpiece turning devices 10 inside and outside the loading area TA and in the storage area 50 will be described.

As shown in FIG. 13, in this operation, the control device 30 analyzes the captured image obtained by capturing the loading area TA with the camera 20 (step S120) and detects It is determined whether or not there is a marker 17 within the ranges D1 and D2 (step S121). If it is determined that the markers 17 are not within the detection ranges D1 and D2 (“No” in step S121), it is assumed that the workpiece turning device 10 is not placed outside the stacking area TA, and the markers 14 to 14 are detected from the captured image. 16 is detected, and device detection processing capable of determining the arrangement position and arrangement orientation of each device in the loading area TA is performed (step S122) in the same manner as in step S103.

Then, if the device is detected in the device detection process of step S122 ("Yes" in step S123), the work W in the stacking area TA is detected (step S125), as in step S110 and the like. Similar to step S111, it is determined whether or not two work turning devices 10 are installed for the same work W (step S126).

If it is determined that two work turning devices 10 are installed for the same work W ("Yes" in step S126), there is a problem in the arrangement of the work turning devices 10 as described above (duplicated installation of devices). etc.), the alarm device is controlled to issue an alarm (step S127) in the same manner as in step S113 and the like, and the process according to this flowchart ends.

On the other hand, if it is determined that two work turning devices 10 are not installed for the same work W ("No" in step S126), interference calculation is performed (step S128) in the same manner as in step S112 and the like. ), if it is determined that there is interference ("Yes" in step S129), the process proceeds to step S127, and the subsequent processes are repeated.

Further, when it is determined that there is no interference (“No” in step S129), the number of workpiece turning devices 10 detected within the loading area TA or outside the loading area TA is 1, as in step S107. It is determined whether there are 1 or 2 (step S130). Then, for example, when it is determined that the number of detected workpiece turning devices 10 is one ("1" in step S130), based on the detection signal from the storage sensor 51, the workpiece is stored in the storage area 50. It is determined whether the number of stored work turning devices 10 is between 0 and 2 (step S131).

Here, when it is determined that the stored number is 1 ("1" in step S131), the number of workpiece turning devices 10 detected inside or outside the loading area TA and It can be determined that there is no abnormality in the number of stored work turning devices 10 . Therefore, assuming that the work turning device 10 is properly installed and managed, the process shifts to the loading process of the work W (step S132), and the process according to this flowchart ends.

On the other hand, if it is determined that the number of stored work is 0 or 2 ("0" or "2" in step S131), the number of work turning devices 10 inside the loading area TA or outside the loading area TA and stored It can be determined that there is an abnormality in the number of devices in the area 50 (for example, an installation error, a storage failure, or the like). For this reason, it is determined that the workpiece turning device 10 is not properly installed and managed, and the alarm device is controlled to give an alarm (step S135) in the same manner as in step S127 and the like, and the processing according to this flow chart ends.

Further, when it is determined that the number of workpiece turning devices 10 detected within the loading area TA or outside the loading area TA is two (“2” in step S130), the storage area 50 It is determined whether the number of work-turning devices 10 stored therein is from 0 to 2 (step S134).

Here, when it is determined that the number of work turning devices 10 stored is 0 ("0" in step S134), it is determined that there is no abnormality in the number of work turning devices 10 within the loading area TA, outside the loading area TA, and within the storage area 50. Since it can be determined, it is assumed that the work turning device 10 is properly installed and managed.

On the other hand, if it is determined that the number of stored devices is 1 or 2 ("1" or "2" in step S134), the workpiece turning devices 10 inside or outside the loading area TA and inside the storage area 50 Therefore, it is determined that the work turning device 10 is not properly installed and managed. end the processing by

If the device is not detected in the device detection process of step S122 (“No” in step S123), based on the detection signal from the storage sensor 51, the workpieces stored in the storage area 50 are turned over. It is determined whether the number of stored devices 10 is 0 or 2 (step S133).

When it is determined that the number of work turning devices 10 stored is two ("2" in step S133), it is determined that there is no abnormality in the number of work turning devices 10 within the loading area TA, outside the loading area TA, and within the storage area 50. Therefore, it is assumed that the work turning device 10 is properly installed and managed, and the process goes to step S132 to perform loading, and the processing according to this flowchart ends.

On the other hand, when it is determined that the stored number is 0 or 1 ("0" or "1" in step S133), even though the work turning device 10 is not arranged inside or outside the loading area TA, Since it can be determined that there is an abnormality in the number of work turning devices 10 in the storage area 50, it is determined that the work turning devices 10 are not properly installed and managed in the same manner as described above, and the process proceeds to step S135. The alarm device is controlled to issue an alarm, and the processing according to this flowchart is terminated.

As described above, according to this operation, it is possible to accurately determine the arrangement and storage conditions of a plurality of work turnover devices 10 arranged in the storage area 50 as well as inside and outside the loading area TA. is possible, and it is possible to reliably prevent the loading of the workpiece W and the like from being performed under the occurrence of a problem.

Although the embodiment of the present invention has been described above, this embodiment is presented as an example and is not intended to limit the scope of the invention. This novel embodiment can be embodied in various other forms, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the scope of the invention described in the claims and equivalents thereof.

10 work turning device (auxiliary device)
10A Apparatus main body 10B Turning unit (movable part)
10a upper surface portion 11 top surface portion 12 slope portion 13 side portion 14, 15, 16, 17 marker (indicator)
18 entire area 19 identification area 20 camera (imaging device)
21 Camera stand 22 Captured image 30 Control device 50 Storage area 51 Storage sensor 100 Device discrimination system D1, D2 Detection range P Pallet TA Loading area W Work

Claims (23)

Auxiliary equipment that can be freely arranged and has portability to assist the work on the work loaded in the loading area;
an imaging device capable of imaging the loading area;
a control device that determines the arrangement position and arrangement direction of the auxiliary device with respect to the loading area based on the captured image captured by the imaging device;
The auxiliary device has an apparatus main body and a movable section that is provided so as to be able to move relative to the apparatus main body and come into contact with the work in the loading area. provided with an indicator capable of identifying the position and direction of the auxiliary device;
A device discrimination system, wherein the control device detects the index included in the captured image and discriminates the arrangement position and the orientation of the arrangement. 2. The device identification system according to claim 1, wherein the indicator is provided on the upper surface side near the tip of the movable portion. 3. The apparatus determination system according to claim 1, wherein the control device compares the detected index with an index indicated by a pre-stored index model image to determine the arrangement position and orientation of the arrangement. The loading area is formed in a rectangular shape in plan view,
The auxiliary device is arranged such that the device main body is positioned outside the loading area,
The control device detects the index included in a set of crossing long-side and short-side outer peripheral rectangular areas in the stacking area from the captured image, and determines the placement position and the placement direction. The device discrimination system according to any one of claims 1 to 3, which discriminates. A plurality of the auxiliary devices are provided,
5. The control device according to any one of claims 1 to 4, wherein the control device determines a placement situation including the placement position and the placement direction of each auxiliary device with respect to the loading area determined based on the detected index. Device discrimination system. The auxiliary device has a top surface portion on the device main body,
The device identification system according to any one of claims 1 to 5, wherein the indicator is provided on the top surface portion of the device main body. The auxiliary device has a slanted surface portion inclined from the top surface portion on the device main body,
7. The device identification system according to claim 6, wherein the indicator is provided on the slope portion of the device main body. 8. The device identification system according to claim 7, wherein the index provided on the slope portion is provided at a location most distant from the top surface portion. the indicator provided on the top surface portion is larger than the indicators provided on the movable portion and the slope portion;
9. The device identification system according to claim 7, wherein the indicator provided on the slope portion is smaller than the indicators provided on the movable portion and the top surface portion. The auxiliary device has a pair of side surface portions connected to the top surface portion on the device main body,
10. The apparatus identification system according to claim 6, wherein the indicators are provided on the pair of side surfaces of the apparatus main body. The indicator has optical transparency,
The device discrimination system according to any one of claims 1 to 10, wherein the auxiliary device includes a light source inside each of the movable portion and the device main body. 12. The controller according to any one of claims 6 to 11, wherein the control device detects the auxiliary device arranged in the loading area based on the captured image, and determines the arrangement position and orientation of the arrangement based on the detected index. 2. Device discrimination system according to claim 1. 13. The control device detects the work in the loading area based on the captured image, and specifies an auxiliary device to operate on the work based on the determination result of the placement situation. 1. The apparatus discrimination system according to 1. A detection means capable of detecting the auxiliary device stored in a storage area provided at a location separated from the loading area,
14. The apparatus determination system according to any one of claims 5 to 13, wherein the control device determines the storage status of the auxiliary device in the storage area based on the detection result from the detection means. Equipped with a notification means capable of notifying a predetermined alarm,
15. The device determination system according to claim 14, wherein the control device controls the notification means to notify the predetermined warning based on the determination result of the arrangement status and/or the storage status. a step of capturing an image of the loading area around which at least an auxiliary device having portability that can be freely arranged for assisting work on the workpieces loaded in the loading area is arranged;
determining the arrangement position and arrangement direction of the auxiliary device with respect to the loading area by a control device based on the captured image captured by the imaging device;
In the discriminating step, the movable portion of the auxiliary device having a movable portion provided so as to be able to move relative to the apparatus main body and the apparatus main body and come into contact with the workpiece in the loading area is Detecting from the captured image at least an index that is provided and capable of identifying the position and direction of the auxiliary device, and determining the arrangement position and the orientation of the arrangement. In the discriminating step, the indicators included in a set of intersecting long side and short side outer peripheral inner rectangular regions in the stacking area formed in a rectangular shape in a plan view from the captured image are detected; 17. The device determination method according to claim 16, wherein the placement position and the placement orientation are determined. In the step of capturing an image, an image is captured of the loading area in which a plurality of the auxiliary devices are arranged such that the device main body is positioned outside the loading area;
18. The device determination method according to claim 16, wherein in the determining step, an arrangement state including the arrangement position and the arrangement direction of each auxiliary device with respect to the loading area determined based on the detected index is determined. In the imaging step, the loading area in which a plurality of the auxiliary devices are further arranged is imaged;
19. The device determination method according to claim 18, wherein in the determination step, the placement status including the placement position and placement direction of each auxiliary device in the loading area determined based on the detected index is determined. 20. The step of detecting the work in the loading area based on the captured image by the control device, and specifying an auxiliary device to be operated with respect to the work based on the determination result of the placement situation. Apparatus identification method described. a step of detecting, by a detecting means, the auxiliary device stored in a storage area provided at a location separated from the loading area;
The device determination method according to any one of claims 18 to 20, further comprising the step of determining the storage status of said auxiliary device in said storage area by said control device based on the detection result from said detection means. 22. The apparatus according to claim 21, further comprising a step of controlling an informing means capable of informing a predetermined alarm by said control device to issue said predetermined alarm based on the determination result of said arrangement status and/or said storage status. Discrimination method. to the computer,
A step of obtaining a captured image obtained by capturing an image of the loading area, in which at least an auxiliary device having portability that can be freely arranged to assist the work on the work loaded in the loading area, is arranged around the loading area with an imaging device. When,
A device discrimination program for executing a step of discriminating the arrangement position and arrangement direction of the auxiliary device with respect to the loading area based on the acquired captured image,
In the discriminating step, the movable portion of the auxiliary device having a movable portion provided so as to be able to move relative to the apparatus main body and the apparatus main body and come into contact with the work in the loading area, is displayed in the captured image. A device determination program for determining the arrangement position and the orientation of the arrangement by detecting from the captured image at least an index that is provided and capable of identifying the position and direction of the auxiliary device.

Images