JP2019193974A - Robot system and robot control method - Google Patents

Abstract

To provide a robot system that can recognize an object changing a shape of a human or a robot or the like.SOLUTION: A robot system 70 includes: an object recognition device; a robot 10 operating in a predetermined work area within a monitoring area; and a robot control unit 16 for controlling the robot. The object recognition device includes: an imaging unit 40 for acquiring a three-dimensional image and a two-dimensional image of the predetermined monitoring area; an object extraction unit for extracting an area containing pixel values of a predetermined range in the acquired three-dimensional image; an image search unit for searching a preregistered reference image according to a type of the object in the two-dimensional image; and a determination unit for determining a type of the object according to whether or not the reference image searched by the image search unit exists within the area extracted by the object extraction unit. The object recognition device detects intrusion of an object to the work area, when the object is recognized to be a safety monitoring target, and when intrusion of the object to the work area is detected by the object recognition device, the robot control unit stops the robot.SELECTED DRAWING: Figure 10

Description

  The present invention relates to a robot system and a robot control method.

Conventionally, a technique for specifying the position of a three-dimensional object by combining a range sensor that acquires three-dimensional information of the object and a camera that acquires a two-dimensional image is known (see, for example, Patent Document 1). .
In the technique of Patent Document 1, a two-dimensional image acquired by a camera is processed to detect an object image in the two-dimensional image to acquire two-dimensional position information of the object, and based on the acquired two-dimensional position information. Thus, the three-dimensional position and orientation of the object are calculated from the three-dimensional information.

JP 2013-101045 A

  However, the technique disclosed in Patent Document 1 has a disadvantage that it can accurately recognize a stationary three-dimensional object, but cannot identify an object whose shape changes, such as a human being or a robot.

  Therefore, a robot system and a robot control method that can accurately recognize an object whose shape changes, such as a human or a robot, are desired.

  According to a first aspect of the present invention, an imaging unit that captures a predetermined monitoring area and acquires a three-dimensional image and a two-dimensional image, and the three-dimensional image acquired by the imaging unit has a pixel value in a predetermined range. An object extraction unit that extracts a region, an image search unit that searches for a reference image registered in advance according to the type of the object in the two-dimensional image acquired by the imaging unit, and the image search unit An object recognition apparatus comprising: a determination unit that determines a type of the object based on whether or not the reference image exists in the area extracted by the object extraction unit; and a work in a predetermined work area in the monitoring area And a robot control unit for controlling the robot, and the object recognition device detects that the object enters the work area when the object is recognized as a safety monitoring target. , The robot control unit, penetration of the object into the work area by the object recognition device is a robotic system to stop the robot when it is detected.

  According to a second aspect of the present invention, an imaging unit that captures a predetermined monitoring region and acquires a three-dimensional image and a two-dimensional image, and the three-dimensional image acquired by the imaging unit has a pixel value in a predetermined range. An object extraction unit that extracts a region, an image search unit that searches for a reference image registered in advance according to the type of the object in the two-dimensional image acquired by the imaging unit, and the image search unit An object recognition apparatus comprising: a determination unit that determines a type of the object based on whether or not the reference image exists in the area extracted by the object extraction unit; and a work in a predetermined work area in the monitoring area And a robot controller that controls the robot, and the object recognition device allows the object to enter the work area when the object is recognized as a safety monitoring target. On the other hand, when the object is recognized as a non-target object that is not recognized as the safety monitoring target, the object is detected to enter the work area, and the robot control unit detects the object recognition device. The robot system stops the robot when the non-target object has entered the work area.

  According to a third aspect of the present invention, there is an imaging step of capturing a predetermined monitoring area and acquiring a three-dimensional image and a two-dimensional image, and the three-dimensional image acquired by the imaging step has a pixel value in a predetermined range. An object extraction step for extracting a region, an image search step for searching for a reference image registered in advance according to the type of the object in the two-dimensional image acquired by the imaging step, and a search by the image search step A determination step of determining a type of the object based on whether or not the reference image exists in the region extracted by the object extraction step; and when the object is identified as a safety monitoring target by the determination step, A detection step for detecting the entry of the object into a predetermined work area in the monitoring area where the robot works; and The entry of the object into the work area is a robot control method comprising a robot stop step of stopping said robot when it is detected.

According to a fourth aspect of the present invention, an imaging step of capturing a predetermined monitoring area and acquiring a three-dimensional image and a two-dimensional image, and the three-dimensional image acquired by the imaging step have a pixel value in a predetermined range. An object extraction step for extracting a region, an image search step for searching for a reference image registered in advance according to the type of the object in the two-dimensional image acquired by the imaging step, and a search by the image search step A determination step of determining the type of the object based on whether or not the reference image exists in the region extracted by the object extraction step; and when the object is identified as a safety monitoring target by the determination step, While allowing the object to enter a predetermined work area within the monitoring area where the robot works, the object is the safety monitoring target. A non-recognized non-target object is detected, the detection step detects that the object enters the work area, and the detection step detects the non-target object intrusion into the work area. The robot control method includes a robot stop step for stopping the robot in the event of failure.

  One aspect of the invention as a reference example of the present invention is an imaging unit that captures a predetermined monitoring region to acquire a three-dimensional image and a two-dimensional image, and a predetermined range in the three-dimensional image acquired by the imaging unit. An object extraction unit that extracts a region having a pixel value of the image, an image search unit that searches for a reference image registered in advance according to the type of the object in the two-dimensional image acquired by the imaging unit, and the image search And a determination unit that determines a type of the object based on whether or not the reference image searched for by the unit exists in the region extracted by the object extraction unit. .

  According to this aspect, when a predetermined monitoring area is imaged by the imaging unit and a two-dimensional image and a three-dimensional image are acquired, an area having a pixel value in a predetermined range in the acquired three-dimensional image is acquired by the object extraction unit. The reference image is searched by the image search unit in the extracted and acquired two-dimensional image, and the type of the object is determined by determining whether or not the reference image exists in the extracted region. The Since the reference image registered in advance is present at the position of the object extracted in the three-dimensional image, not the shape of the object in the three-dimensional image, the type of the object in the monitoring area is recognized. It is also possible to accurately recognize an object whose shape changes.

In the said aspect, you may provide the search area | region setting part which sets the search area | region by the said image search part so that the said area | region extracted by the said object extraction part may be included.
In this way, instead of using the shape of the region representing the object extracted by the object extracting unit, it is possible to set a simplified search region so as to include the region, The search calculation by can be facilitated.

Moreover, in the said aspect, the said image search part changes the scale of the said reference image based on the distance from the said imaging part in each part of the said three-dimensional image acquired by the said imaging part, and performs a search. Also good.
A mark attached to an object arranged at a position close to the imaging unit is greatly reflected in the two-dimensional image, and a mark attached to an object arranged at a position far from the imaging unit is It appears small in a two-dimensional image. In this way, by changing the scale of the reference image in accordance with the distance from the imaging unit, the reference image can be brought close to the size of the mark to be searched, and the search can be facilitated.

In the above aspect, the image search unit performs a search by modifying the reference image based on an inclination angle of each part of the three-dimensional image acquired by the imaging unit with respect to the optical axis of the imaging unit. May be.
If the surface of the object to which the mark is attached is inclined with respect to the optical axis of the imaging unit, the mark is distorted. In this way, by deforming the reference image according to the inclination angle of the surface of the object, the reference image can be brought close to the shape of the mark to be searched, and the search can be facilitated.

  According to another aspect of the invention as a reference example of the present invention, an imaging step of capturing a predetermined monitoring area to acquire a three-dimensional image and a two-dimensional image, and the three-dimensional image acquired by the imaging step An object extraction step of extracting a region having a pixel value in a predetermined range; and an image search step of searching for a reference image registered in advance according to the type of the object in the two-dimensional image acquired by the imaging step; And a determination step of determining the type of the object based on whether or not the reference image searched by the image search step is present in the region extracted by the object extraction step. .

  According to the present invention, it is possible to accurately recognize an object such as a human being or a robot whose shape changes.

1 is an overall configuration diagram showing an object recognition device according to an embodiment of the present invention. It is a figure which shows an example of the three-dimensional image acquired by the object recognition apparatus of FIG. It is a figure which shows an example of the binarized image of the object extracted from the three-dimensional image of FIG. It is a figure which shows an example of the search area | region containing the object extracted in FIG. It is a figure which shows an example of the search area | region in the two-dimensional image corresponding to the search area | region set in FIG. It is a flowchart explaining the object recognition method using the object recognition apparatus of FIG. It is a figure which shows an example of the three-dimensional inclination in the three-dimensional image acquired by the object recognition apparatus of FIG. It is a figure which shows an example of the two-dimensional image of the object extracted from the three-dimensional image of FIG. It is a top view which shows the robot system which is an example of the application of the object recognition apparatus of FIG. FIG. 10 is a front view of the robot system of FIG. 9.

An object recognition apparatus 1 and an object recognition method according to an embodiment of the present invention will be described below with reference to the drawings.
As shown in FIG. 1, the object recognition apparatus 1 according to the present embodiment includes an imaging unit 2 that captures a predetermined monitoring area A, and a two-dimensional image G2 and a three-dimensional image G1 acquired by the imaging unit 2. And an image processing unit 3 that recognizes the type of the object based on the image processing unit 3.

  The imaging unit 2 includes a two-dimensional camera 4 that captures a region including the monitoring region A and acquires a two-dimensional image G2 of the monitoring region A, and a three-dimensional camera 5 that acquires a three-dimensional image G1 of the monitoring region A. ing. The relative positions of the two-dimensional camera 4 and the three-dimensional camera 5 are set in advance, and each pixel position of the two-dimensional image G2 acquired by the two-dimensional camera 4 and the three-dimensional image G1 acquired by the three-dimensional camera 5 are set. Each pixel position is associated with each other precisely. Note that the two-dimensional camera 4 and the three-dimensional camera 5 may be integrated instead of separate.

  The three-dimensional camera 5 is arranged vertically downward, for example, above the monitoring area A. In the example shown in FIG. 1, the two-dimensional camera 4 is arranged at a slight angle with respect to the three-dimensional camera 5, thereby arranging the same monitoring area A as the three-dimensional camera 5 in the field of view, A viewed two-dimensional image G2 is acquired. On the other hand, the three-dimensional camera 5 acquires a three-dimensional image G1 showing the height distribution of each part in the monitoring area A.

  The image processing unit 3 includes an object extraction unit 6 that extracts an object B in the monitoring area A in the three-dimensional image G1 acquired by the three-dimensional camera 5, and a search that includes the object B extracted by the object extraction unit 6. A search area setting unit 7 for setting the area C1 and an image correspondence calculation unit 8 for converting the search area C1 set in the search area setting unit 7 into a search area C2 in the two-dimensional image G2 acquired by the two-dimensional camera 4. And an image search unit 9 for searching for the same image as the reference image M in the search region C2 of the two-dimensional image G2, and whether or not the same image as the reference image M is found in the search region C2. And a determination unit 100 that determines the type.

  In the present embodiment, as shown in FIG. 2, the object B is exemplified by a human head having a square mark M1 on a head, for example, a helmet, and a work table having a triangular mark M2 on a top board. ing.

  In the extraction of the object B by the object extraction unit 6, pixels arranged in a predetermined height range in the monitoring area A in the three-dimensional image G1 are extracted as an area where the object B exists. Specifically, as shown in FIG. 3, by binarizing the pixel value of the three-dimensional image G1 with a predetermined value, an area where the height from the ground surface is equal to or higher than the predetermined height is defined as an object B. It comes to extract. Then, the object extraction unit 6 extracts a region where pixels extracted by binarization are continuous as a region representing a single object B.

  The search area setting unit 7 includes an area representing the object B extracted by the object extraction unit 6 and includes the extracted area and has a simple shape, for example, a rectangular shape as shown in FIG. (Hereinafter referred to as the search area) C1 is replaced.

  The image correspondence calculation unit 8 stores data indicating the corresponding positional relationship between the two-dimensional image G2 and the three-dimensional image G1, and the three-dimensional image G1 set by the search region setting unit 7 as shown in FIG. The search area C1 is converted into a search area C2 in the two-dimensional image G2 and sent to the image search unit 9 as shown in FIG.

  The image search unit 9 stores a reference image to be searched, for example, a reference image M having the same shape as a square mark M1 attached to a person and a triangular mark M2 attached to a workbench. That is, the quadrangle reference image M is associated with a human and the triangle reference image M is associated with a work table and stored.

The image search unit 9 searches for the same image as each reference image M in the search region C2 of the two-dimensional image G2 sent from the image correspondence calculation unit 8 by known image processing such as pattern matching. It has become.
If the same image as the reference image M is found in the search area C2 of the two-dimensional image G2, the determination unit 100 determines that the type of the object B is a type associated with the reference image M. If it is recognized and not found, the type of the object B is recognized as a foreign object.

An object recognition method using the object recognition apparatus 1 according to this embodiment configured as described above will be described below.
As shown in FIG. 6, the object recognition method according to the present embodiment includes an imaging step S1 in which the imaging unit 2 acquires a two-dimensional image G2 and a three-dimensional image G1 of the monitoring area A, and an object B from the three-dimensional image G1. Extraction step S2 for extracting the region where the object exists, step S3 for determining whether or not the object B is extracted, and the reference image M stored in the two-dimensional image G2 when the object B is extracted The image search step S4 for searching for the same mark as the reference image, the step S5 for determining whether or not the same mark as the reference image M is found in the search area C2, and the type of the object B is determined based on whether or not it is found. Determination steps S6 and S7.

If it is determined in step S3 that the object B has not been extracted, the processes from step S1 are repeated.
If no mark is found in step S5, it is determined whether or not the search has ended (step S8). If the mark has ended, the object B is recognized as a foreign object in determination step S7. It is like that. If the search has not ended, the processes from step S4 are repeated.

  If it is determined in step S5 that the mark has been found in the search area C2, the object B is recognized as an object corresponding to the reference image M in determination step S6, and the search is completed. (Step S9), and if not finished, the process from step S4 is repeated.

As described above, according to the object recognition device 1 and the object recognition method according to the present embodiment, the presence or absence of the object B is recognized from the three-dimensional image G1, and the type of the object B is recognized from the two-dimensional image G2. There is an advantage that the type of the object B can be recognized more surely even for the object B whose shape changes like a robot.
In the present embodiment, by adding the mark M1 to all humans, the object B can be recognized as a human if the mark M1 is detected in the search areas C1 and C2, but only for a specific human being. By providing the mark M1, there is also an advantage that it can be recognized that the object B is a specific person.

In this embodiment, since a continuous region having a predetermined range of height information is extracted as a single object B in the three-dimensional image G1 obtained by the three-dimensional camera 5, not only humans but also humans The possessed article can also be recognized as a part of the object B.
In addition, since simple regions such as rectangles are defined as the search regions C1 and C2 including the object B extracted from the three-dimensional image G1, it is easy to determine whether or not the reference image M exists in the search region C2. Can be determined.

  Moreover, in this embodiment, the image search part 9 is based on the height information (distance information from the three-dimensional camera 5) of each part of the three-dimensional image G1 acquired by the three-dimensional camera 5. The scale may be adjusted. Since the size of the marks M1 and M2 attached to the object B in the two-dimensional image G2 changes according to the height of the object B, the marks M1 and M2 are attached at positions where the height is high. In this case, if the size of the reference image M is large and the marks M1 and M2 are attached at positions where the height is low, the size of the reference image M is adjusted to be small. Thereby, it is possible to easily search for the same image as the reference image M in the two-dimensional image G2.

  Further, the image search unit 9 uses the height information of each part of the three-dimensional image G1 acquired by the three-dimensional camera 5 so that the marks M1 and M2 attached to the object B are on the optical axis of the three-dimensional camera 5. If the reference image M is attached to the inclined surface, the reference image M may be modified based on the inclination angle with respect to the optical axis of the three-dimensional camera 5 to perform the search. That is, as shown in FIG. 7, the normal vector V of each part can be obtained based on the height information h of each part in the three-dimensional image G1, and a rectangular shape is obtained according to the direction of the obtained normal vector V. The reference image M may be transformed into a trapezoidal shape.

  As shown in FIG. 8, since the marks M1 and M2 shown in the two-dimensional image G2 have a trapezoidal shape that gradually decreases in the direction of lowering the height, the mark M1 is marked using the trapezoidal reference image M. There is an advantage that the positions of M1 and M2 can be easily searched.

Next, the robot system 70 and the robot control method using the object recognition apparatus 1 and the object recognition method according to the present embodiment will be described.
As shown in FIGS. 9 and 10, the robot system 70 includes a robot 10 that performs a predetermined operation, a work table 20 on which the robot 10 operates, and a safety fence 30 that surrounds the robot 10 and the work table 20. ing.

  In this robot system 70, when the worker S exchanges or adjusts jigs, molds, workpieces, etc. on the work table 20, the worker S may place his hand or head above the work table 20. . An opening 31 for the operator S to access the work table 20 is provided in a part of the safety fence 30 in a portion located on the opposite side of the work table 20 from the robot 10.

  The robot system 70 is provided above the work table 20 and is shown in FIGS. 9 and 10 so that at least a part of the work area of the robot 10 and at least a part of the area where the operator S can enter are copied. An imaging unit 40 that acquires a two-dimensional image G2 and a three-dimensional image G1 at a predetermined frame rate with a visual field W indicated by a two-dot chain line, and a display device 50 supported by the safety fence 30 Is further provided. The display device 50 has a liquid crystal screen and is disposed at a position where it can be seen by the worker S near the opening 31.

The robot 10 is, for example, a six-axis articulated robot fixed to the floor surface, and is configured to move a workpiece (not shown) on the work table 20, process the workpiece, and the like. The form of the robot 10 may be arbitrary.
The robot 10 is connected to a robot control unit 16, and the operation of each axis is controlled by the robot control unit 16.

  9 and 10, the imaging unit 40 is displayed by a single camera. However, as in FIG. 1, for example, a stereo camera, a TOF (Time of Flight) type three-dimensional camera 5, and a CCD or A two-dimensional camera 4 such as a CMOS image sensor may be provided separately.

  In the robot system 70, the image processing unit 3 identifies the object B based on the two-dimensional image G2 and the three-dimensional image G1 acquired by the imaging unit 40, and displays the identification result on the display device 50 and the robot control unit. 16 controls the robot 10. The imaging unit 40 and the image processing unit 3 constitute the object recognition device 1 according to the present embodiment.

  In the robot system 70, in the three-dimensional image G1 acquired in time series, as shown in FIG. 9 and FIG. 10, the worker recognition area A1 surrounded by a one-dot chain line and the common work area indicated by diagonal lines A2 is set. The common work area A2 is an area that the worker S can access and the robot 10 can also access.

  In the robot system 70, the worker-side boundary B1 and the robot-side boundary B2 shown in FIGS. 9 and 10 are set in the three-dimensional image G1 acquired in time series. The worker-side boundary B1 is a boundary surface that is exceeded when the worker S accesses the common work area A2, and the robot-side boundary B2 is a boundary surface that is exceeded when the robot 10 accesses the common work area A2.

  Using the worker recognition area A1 and the common work area A2 as monitoring areas, the object B in the monitoring area is extracted from the three-dimensional image G1 acquired in time series to set the search area C1, and simultaneously acquired in time series. It is determined whether or not a reference image, for example, a mark M as shown in FIG. 9 exists in the corresponding search area C2 of the two-dimensional image G2. When the mark M is present, the detected object B is recognized as the worker S as a safety monitoring target, and when the mark M is not present, it is recognized as a non-target object. The

  In the example shown in FIG. 9, the mark M is composed of three black square colored portions attached to the upper surface of the helmet or hat of the worker S. The three square coloring portions are arranged so as to be aligned in the width direction (left-right direction) of the worker S, and the central square coloring portion is arranged in front of the worker S with respect to the other two. Thereby, when looking at the mark M, it is possible to determine or estimate which direction the worker S is facing.

  According to the present embodiment, the worker S is recognized as a safety monitoring target in the two-dimensional image G2 and the three-dimensional image G1 acquired in time series, and is tracked in the two-dimensional image G2 and the three-dimensional image G1. Therefore, it is possible to monitor the worker S without losing sight of the worker S in the two-dimensional image G2 and the three-dimensional image G1. Thereby, it can be reliably detected that the operator S has entered the boundary (robot side boundary B2) that is not allowed to enter. Then, the robot 10 can be stopped by detecting the intrusion, or the robot 10 can perform a danger avoiding operation.

  In the three-dimensional image G1, a worker-side boundary B1 that is exceeded to access the common work area A2 from the worker S side and a robot-side boundary B2 that exists on the robot 10 side in the common work area A2 are set. When it is recognized as a monitoring target, it can access the common work area A2 beyond the worker-side boundary B1, but a non-target object that is not recognized as a safety monitoring target exceeds the worker-side boundary to access the common work area A2. Intrusion into B1 is detected. For this reason, for example, when a person who is not the worker S and does not have sufficient knowledge of the safe work (a person who is not attached with the mark M) moves toward the common work area A2 and enters the worker-side boundary B1. Since the movement is detected, for example, by stopping the robot 10 or causing the robot 10 to perform the danger avoiding operation, it is avoided that a person who does not have enough knowledge of the safety work contacts the robot 10. .

  Moreover, since the robot 10 is a non-target object, when the robot 10 passes through the common work area A2 and an abnormality such as exceeding the worker-side boundary B1 occurs, the abnormality can be detected. Or, an abnormality such as a jig or workpiece in the common work area A2 unintentionally moving to the worker S side, or a jig or workpiece held by the worker S is separated from the hand of the worker S and is common. When there are abnormalities such as entering the work area A2, these abnormalities can be detected.

In addition, when tracking the worker S in the three-dimensional image G1, the processing time is shortened and erroneous detection is made by trying to collate the mark M in the vicinity where the mark M is recognized one frame before. It is possible to reduce the possibility of
Further, there may be a plurality of workers S recognized as safety monitoring targets. In this case, the same processing as described above is performed for each of the plurality of workers S. That is, the above-described safety monitoring process is performed until all the workers S are not recognized.

In the present embodiment, since the robot 10 is stopped when an intrusion by a non-target object is detected, it is possible to ensure the safety of the worker S and prevent the robot 10 from being broken.
Further, since not only the worker S but also an article transported by the worker S is a target for safety monitoring, it is possible to prevent contact between the article and the robot 10 and the safety of the worker S is ensured. And failure of the robot 10 can be prevented.
Further, even when an intrusion by a non-target object is detected, the robot 10 may continue working outside the common work area A2. In this case, the safety of the worker S is ensured without lowering the operation rate of the robot 10.

  In the present embodiment, the operator S can check whether or not he / she is recognized as a safety monitoring target by looking at the display device 50, and the common work area A2 is not recognized as a safety monitoring target. Thus, it is possible to prevent a standby time from waiting without knowing whether or not the work is performed and whether the object is recognized as a safety monitoring target.

  The reference image M may be associated with information related to the height and physique of the worker S. Since the three-dimensional image G1 includes the height information of the head and shoulders of the worker S in the three-dimensional image G1, the height of the head and shoulders of the worker S permitted to work on the reference image M. By associating the information, it is possible to determine whether or not the person who attached the mark M to the helmet is the worker S who is permitted to work.

  In this case, when it is determined that the helmet M is attached to the helmet but the work is not permitted, the person is recognized as a non-target object, and when entering the worker-side boundary B1, the robot 10 will be stopped.

  Further, a robot mark may be attached to the robot 10 and another reference image for recognizing the robot 10 by searching for the robot mark attached to the robot 10 in the two-dimensional image G2 may be stored. In this case, similarly to the worker S, the robot 10 can be recognized as a safety monitoring target, and it can be configured to detect that an object other than the safety monitoring target has entered the robot-side boundary B2.

  As a result, the position of the robot 10 can also be tracked, and the worker S is allowed to cross the common work area A2, but the robot 10 exceeds the worker-side boundary B1 that is not allowed to cross. In addition, when the robot 10 moves differently from normal, these events can be known immediately, which is advantageous in improving safety.

  In the present embodiment, the worker recognition area A1, the common work area A2, the worker side boundary B1, and the robot side boundary B2 are set in the three-dimensional image G1. Instead, the worker recognition area A1, the common work area A2, the worker side boundary B1, and the robot side boundary B2 are provided on the floor surface with tape or the like, and these areas and boundaries appearing in the two-dimensional image G2 are provided. It can also be used as the worker recognition area A1, the common work area A2, the worker side boundary B1, and the robot side boundary B2. In this case, when the object B detected in the three-dimensional image G1 overlaps the worker-side boundary B1 or the robot-side boundary B2, it may be determined that the object B has entered these boundaries.

DESCRIPTION OF SYMBOLS 1 Object recognition apparatus 2,40 Image pick-up part 6 Object extraction part 7 Search area setting part 9 Image search part 10 Robot 16 Robot control part 70 Robot system 100 Judgment part A Monitoring area A1 Worker recognition area (monitoring area)
A2 Common work area (monitoring area)
B Object C1, C2 Search area M Reference image G1 3D image G2 2D image S1 Imaging step S2 Object extraction step S4 Image search step S5 Determination step

Claims (4)

An imaging unit that captures a predetermined monitoring area and acquires a three-dimensional image and a two-dimensional image;
An object extraction unit that extracts an area having a predetermined range of pixel values in the three-dimensional image acquired by the imaging unit;
In the two-dimensional image acquired by the imaging unit, an image search unit that searches for a reference image registered in advance according to the type of the object;
An object recognition device comprising: a determination unit that determines the type of the object based on whether or not the reference image searched for by the image search unit exists in the region extracted by the object extraction unit;
A robot working in a predetermined work area in the monitoring area;
A robot control unit for controlling the robot,
When the object recognition device recognizes that the object is a safety monitoring target, detects that the object enters the work area,
A robot system in which the robot control unit stops the robot when the object recognition device detects the intrusion of the object into the work area. An imaging unit that captures a predetermined monitoring area and acquires a three-dimensional image and a two-dimensional image;
An object extraction unit that extracts an area having a predetermined range of pixel values in the three-dimensional image acquired by the imaging unit;
In the two-dimensional image acquired by the imaging unit, an image search unit that searches for a reference image registered in advance according to the type of the object;
An object recognition device comprising: a determination unit that determines the type of the object based on whether or not the reference image searched for by the image search unit exists in the region extracted by the object extraction unit;
A robot working in a predetermined work area in the monitoring area;
A robot control unit for controlling the robot,
When the object recognition device recognizes that the object is a safety monitoring target, the object recognition device allows the object to enter the work area, while the object is not recognized as the safety monitoring target. If the object is recognized, it is detected that the object enters the work area,
A robot system that stops the robot when the robot control unit detects that the non-target object has entered the work area by the object recognition device. An imaging step of capturing a predetermined monitoring area and acquiring a three-dimensional image and a two-dimensional image;
An object extraction step of extracting a region having a predetermined range of pixel values in the three-dimensional image acquired by the imaging step;
In the two-dimensional image acquired by the imaging step, an image search step for searching for a reference image registered in advance according to the type of the object;
A determination step of determining a type of the object based on whether or not the reference image searched by the image search step exists in the region extracted by the object extraction step;
A detection step of detecting entry of the object into a predetermined work area in the monitoring area where the robot works when the object is identified as a safety monitoring target by the determination step;
A robot control method comprising: a robot stop step for stopping the robot when the object is detected to enter the work area by the detection step. An imaging step of capturing a predetermined monitoring area and acquiring a three-dimensional image and a two-dimensional image;
An object extraction step of extracting a region having a predetermined range of pixel values in the three-dimensional image acquired by the imaging step;
In the two-dimensional image acquired by the imaging step, an image search step for searching for a reference image registered in advance according to the type of the object;
A determination step of determining a type of the object based on whether or not the reference image searched by the image search step exists in the region extracted by the object extraction step;
When the determination step identifies the object as a safety monitoring target, the object is allowed to enter a predetermined work area in the monitoring area where the robot works, while the object is set as the safety monitoring target. If it is identified as an unrecognized non-target object, a detecting step for detecting the object entering the work area;
A robot control method comprising: a robot stop step of stopping the robot when the non-target object has entered the work area by the detection step.

Images