JP2022138211A - Object detection system and control device - Google Patents

Abstract

To provide a technique for increasing the response speed of object detection.SOLUTION: An object detection system includes: a plurality of imaging means arranged so that measurable regions thereof overlap each other at least partially and outputting periodically distance information within the measurable regions; object detection means for detecting an object based on the distance information output by the plurality of imaging means; and control means for controlling output timings of the distance information by the plurality of imaging means to be shifted from each other.SELECTED DRAWING: Figure 4

Description

The present invention relates to an object detection system and control device.

In many production sites, robots work while collaborating with workers. In such production sites, various measures are taken to ensure the safety of workers. For example, based on safety standards, a virtual protection area is set near a danger source such as a robot, and when an object such as a human body enters the protection area, the danger source is driven at low speed or stopped. An area sensor, for example, is used for object detection.

The protected area is set by considering the time required from object detection to completion of deceleration or stopping of the hazard. That is, the size of the protection area is determined by the safe distance from the hazard (the distance that can guarantee that the hazard completes deceleration and stoppage before an object entering the protection area reaches the hazard). Also, the safe distance is determined in consideration of the response speed of the sensor, the operating speed of the hazard, braking performance, and the like. From the viewpoint of productivity, it is desirable to set the protection area as narrow as possible. From the viewpoint of improving safety, it is advantageous to shorten the time from when an object actually enters the protected area to when the object is detected. Therefore, it is required to quickly detect an object entering the protected area.

JP 2016-197403 A

However, the area sensor speed (distance information output cycle) has a performance limit. Also, from the viewpoint of avoiding cost increase, there are cases where a high-performance sensor cannot be adopted. Therefore, the output period of the distance information from the area sensor becomes rate-determining, and the setting of the protection area and the setting of the operation speed of the hazard may be restricted. Therefore, there is room for improvement in increasing the response speed of object detection.

The inventors are considering adopting a three-dimensional distance sensor as an area sensor in order to detect an object existing in a three-dimensional space with high accuracy. In the case of such a three-dimensional distance sensor, it takes a certain amount of processing time to acquire the three-dimensional distance information, so the above problem becomes more pronounced.

By the way, Patent Literature 1 discloses a technique for improving temporal resolution of detection by using a combination of two magnetic sensors and torque sensors. However, the device of Patent Literature 1 does not relate to area sensors.

An object of the present invention is to provide a technique for increasing the response speed of object detection.

In order to achieve the above objects, the present invention employs the following configurations.

A first aspect of the present invention is a plurality of imaging means arranged so that at least a part of each other's measurable areas overlaps, and periodically outputting distance information within the measurable areas; and a plurality of the imaging means. and control means for controlling output timings of the distance information from the plurality of imaging means so as to be shifted from each other. Provide a detection system.

The “imaging means” is, for example, a TOF (Time of Flight) sensor. The “measurable area” is, for example, a three-dimensional space in the field of view (angle of view range) of each imaging means. However, the depth may be limited to a distance range in which an accurate distance can be measured.

According to this configuration, since the output timings of the distance information from the plurality of imaging means are controlled to be shifted from each other, the distance information from both imaging means can be obtained at different timings, so the apparent response speed is increased. increase. Therefore, the response speed of object detection can be increased.

Further, the output cycle of the distance information by each of the imaging means may be common, and the control means may control the distance information to be output from the plurality of imaging means at equal time intervals. Specifically, when the number of the imaging means is n (n is an integer equal to or greater than 2) and the output period is T, the control means controls the time interval of T/n from the n imaging means. may be controlled so that the distance information is output by . For example, when two imaging means are used, the output timings of the distance information from both are controlled to be staggered. Thereby, the response speed can be efficiently increased.

Further, it has setting means for setting a virtual protection area for object detection, and the protection area set by the setting means is included in an overlapping space in which the measurable areas of the plurality of imaging means overlap each other. Each of the imaging means may be arranged as follows. As a result, a high response speed can be secured in the protected area.

Further, setting means for setting a virtual protection area for object detection is provided, each of the imaging means outputs luminance information together with the distance information, and the control means serves as an index of a desired protection area. Based on the luminance information output by each of the imaging means with the markers arranged, the luminance image corresponding to each of the imaging means is displayed on the screen, and the setting means receives a user's input on the screen. The virtual protection area may be set based on the specified position and the output distance information. As a result, it is possible to assist the work of setting the protection area.

Also, three or more of the markers are arranged on a reference plane, and the setting means designates a position on the screen from the user, the output distance information, and the reference marker corresponding to each of the markers. The virtual protection area may be set based on known height information indicating a position away from the surface in the vertical direction. Alternatively, the markers include three or more first markers arranged on a reference plane, and second markers arranged at positions separated from the reference plane in the vertical direction corresponding to each of the first markers; may be included.

Further, the control means defines a reference plane with each of the imaging means as a reference, based on the distance information output by each of the imaging means in a state in which an index defining a plane is placed on the reference plane. 3, in a spatial coordinate system based on any one of the plurality of imaging means, the overlapped measurable regions of the plurality of imaging means are defined as the overlapping space together with the defined reference plane; A dimension display may be performed, and the setting means may set the virtual protection area based on an instruction from the user in the spatial coordinate system. For example, a protected area is set by specifying a plurality of points in a spatial coordinate system in which overlapping spaces are displayed. As a result, it is possible to assist the work of setting the protection area.

Also, the indices defining the plane may be at least three markers. Also, the imaging means may be a TOF sensor.

A second aspect of the present invention is a work machine, a plurality of imaging means installed so as to be able to image a common area, each periodically imaging at a predetermined time interval, and output from the plurality of imaging means and an object detection means for detecting an object in the area based on an image, wherein the plurality of safety control systems are configured such that the imaging timings of the plurality of safety control systems are staggered by a time shorter than the time interval. A control device is provided, comprising imaging control means for controlling imaging means, and safety control means for restricting the operation of the working machine based on the detection result of the object detection means.

A "working machine" is, for example, a robot, manufacturing equipment, or the like. The “imaging means” is, for example, a TOF (Time of Flight) sensor. The “common area” is, for example, a three-dimensional space in which the fields of view (ranges of angles of view) of a plurality of imaging means overlap.

According to this configuration, the plurality of imaging means are controlled so that the imaging timings are shifted from each other, so images from each imaging means can be obtained at time intervals shorter than the imaging cycle, and the apparent imaging cycle is increased. be able to. Therefore, it is possible to increase the response speed of object detection and safety control.

According to the present invention, the response speed of object detection can be increased.

FIG. 1 is a schematic diagram of an object detection system according to one embodiment of the present invention. FIG. 2 is a schematic diagram showing the measurable area of one three-dimensional distance sensor. FIG. 3 is a block diagram of an object detection system. FIG. 4 is a timing chart showing measurement operations of two three-dimensional distance sensors. FIG. 5 is a flow chart showing sensor installation processing and protection area setting processing in the first method. FIG. 6 is a diagram showing examples of luminance images displayed on two screens. FIG. 7 is a conceptual diagram showing how the relative positional relationship of markers is specified. FIG. 8 is a diagram showing a display example of the protection area. FIG. 9 is a diagram showing an example target with eight markers. FIG. 10 is a flow chart showing sensor installation processing and protection area setting processing in the second method. FIG. 11 is a diagram showing a display example of a luminance image and a visual field range. FIG. 12 is a schematic conceptual diagram showing a display example of overlapping spaces. FIG. 13 is a diagram showing a display example of the brightness image and the visual field range after redoing.

<Application example>
An application example of the object detection system according to the present invention will be described with reference to FIGS. 1 to 4. FIG. FIG. 1 is a schematic diagram of an object detection system 100 according to one embodiment of the invention. FIG. 2 is a schematic diagram showing the measurable area of one three-dimensional distance sensor. FIG. 3 is a block diagram of the object detection system 100. As shown in FIG. FIG. 4 is a timing chart showing measurement operations of two three-dimensional distance sensors 10A and 10B (hereinafter referred to as sensors 10A and 10B).

The configurations of the sensors 10A and 10B are common. As shown in FIG. 2, the three-dimensional shape of the substantial measurable area RA of the sensor 10A is approximately a truncated quadrangular pyramid as an example. This is because the distance range in which an accurate distance can be measured is limited in the field of view of the sensor 10A. The three-dimensional shape of the substantial measurable area RB of the sensor 10B is the same as the measurable area RA. As shown in FIG. 1, the sensors 10A and 10B are arranged such that at least a portion of their measurable regions RA and RB overlap each other. A three-dimensional area where the measurable areas RA and RB overlap is the overlapping space Rx. Each of the sensors 10A and 10B is arranged so that the protection area 41 is included in the overlapping space Rx.

Here, the protected area 41 is a virtual three-dimensional area for object detection, and is set in the vicinity of or around working machines such as robots and manufacturing equipment (hereinafter also referred to as "danger sources"). The protected area 41 is defined according to safety standards, taking into account the operating range of the hazard. For example, when an object such as a human body enters the protection area 41, safety control such as slow driving or stopping of the danger source is performed.

As shown in FIG. 3, each of the sensors 10A and 10B has a light emitter 11, a light receiver 12 and a calculator 13. FIG. The light emitting portion 11 emits light (for example, infrared light), and the light receiving portion 12 receives reflected light. As an example, the sensors 10A and 10B have time of flight (TOF
), a TOF sensor is employed to acquire the range image from . For example, an indirect TOF sensor that estimates the time difference from the phase difference between projected light and reflected light is employed. The sensors 10A and 10B periodically output three-dimensional distance information and luminance information as measurement results. Measurement results by the sensors 10A and 10B are supplied to the control section 30 via the sensor I/F 14 in the sensor control unit 50. FIG. Sensors 10A and 10B are controlled by control unit 30 via sensor I/F 14 .

As shown in FIG. 4, each of the sensors 10A and 10B periodically repeats imaging processing by the light emitting unit 11 and the light receiving unit 12 and arithmetic processing by the arithmetic unit 13 (including data transfer). An imaging process and an arithmetic process are performed in the frame period T1. A frame period T1 is one cycle (output cycle). In the imaging process, in addition to light emission by the light emitting unit 11, exposure, readout of charges received by the light receiving unit 12, and the like are executed. These operations may be performed multiple times in one imaging process. In the arithmetic processing, processing such as indirectly estimating time from the measured amount of light and generating distance information from the estimated time is executed. After arithmetic processing, measurement results (distance information and luminance information) are output.

Here, the control unit 30, which is an example of control means, controls the output timings of the measurement results of the sensor 10A and the sensor 10B to be shifted from each other. As an example, the control unit 30 controls the sensors 10A and 10B so that the measurement results are output from the sensors 10A and 10B at equal time intervals. For example, when there are two sensors, the period T2, which is the interval between the output from the sensor 10A and the output from the sensor 10B, is set to T1/2. That is, when there are n sensors (where n is an integer equal to or greater than 2), control should be performed so that measurement results are output from the n sensors at time intervals of T1/n. In the measurable areas RA and RB that are not the overlapping space Rx, the measurement result is obtained only from one of the sensors 10A and 10B, so the measurement result acquisition cycle is the frame period T1. However, in the overlapping space Rx, measurement results are obtained from both the sensors 10A and 10B, so measurement results are obtained at the period T2. Therefore, the response speed is apparently doubled. In other words, the sensor 10B captures images while the sensor 10A is performing arithmetic processing, thereby substantially increasing the frame rate. Accordingly, by setting the protection area 41 so as to be included in the overlapping space Rx, it is possible to reliably increase the response speed in the protection area 41 .

By the way, in order for the desired protection area to be included in the overlapping space Rx, it is necessary to appropriately adjust the positions and orientations of the sensors 10A and 10B. However, since the light (infrared light) used for measurement is invisible, it is difficult for the user to grasp where the overlapping space Rx is formed. Therefore, in order to help the user install the sensors 10A and 10B, the user can visually recognize the overlapping space Rx or the protection area 41 in the current installation state. This is outlined.

In the first method (described later), the user causes the sensors 10A and 10B to perform measurement operations while placing the markers M1 to M4 (FIG. 1), which serve as indicators of the desired protection area, on the floor or the like. Taking the case where the protection area is a rectangular parallelepiped as an example, the markers M1 to M4 are arranged at the vertices of the bottom surface of the protection area. Further, the user provides the control unit 30 with height information from each of the markers M1 to M4 to the apex of the upper surface of the protection area. Then, the user specifies (instructs the control unit 30) the positions where the markers M1 to M4 are displayed on the luminance images obtained from the outputs from the sensors 10A and 10B.

Then, the control unit 30 determines the distance from the pixel at the same position in the two distance images to each marker, and grasps the relative positional relationship between each sensor and the eight vertices of the protection area. Based on this, the control unit 30 as an example of setting means sets a protection area 41 defined by eight vertices and displays it on the screen as shown in FIG. If the user determines that the displayed protection area 41 is not appropriate, the user can readjust the positions of the sensors 10A and 10B.

As another method (second method; described later), the user places markers M1 to M3 (FIG. 11), which are indices defining a plane, on the floor or the like, and performs measurement operations on the sensors 10A and 10B. to run. From the measurement results, a reference plane 40 is defined with respect to each of the sensors 10A and 10B. The control unit 30 three-dimensionally displays the overlapping space Rx together with the defined reference plane 40 in a spatial coordinate system based on, for example, the sensor 10A (FIG. 12). When the user designates the vertex of a desired protection area within the overlapping space Rx, the corresponding protection area 41 is set and displayed on the screen. If the user determines that the displayed protection area 41 is not appropriate, the user can readjust the positions of the sensors 10A and 10B.

The control unit 30, which is an example of object detection means, detects an object based on the distance information output from the sensors 10A and 10B. When one or both of the sensors 10A and 10B obtain distance information indicating the distance corresponding to the measurable area, the control unit 30 determines that "there is an object in the measurable area RA or the measurable area RB". It can be determined (detected). In addition, when either or both of the sensors 10A and 10B obtain distance information indicating the distance within the overlapping space Rx, the control unit 30 can determine that "there is an object within the overlapping space Rx." can. In particular, since the protection area 41 is within the overlapping space Rx, the control unit 30 can determine whether or not an object has entered the protection area 41 at a high response speed. The control unit 30, which is an example of a safety control means, executes safety control to limit the operation of the working machine (hazard source) based on the object detection result. Since objects can be detected in the protection area 41 at a high response speed, safety control such as stopping or decelerating the work machine can be performed quickly.

The above application examples are examples to aid understanding of the present invention, and are not intended to limit interpretation of the present invention.

<Embodiment>
Next, the configuration of the object detection system 100, the procedure for setting the protection area, and the like will be described in detail. First, the overall configuration of the object detection system 100 will be described with reference to FIG.

Object detection system 100 includes sensor control unit 50 and sensors 10A, 10B. The sensor control unit 50 includes a control section 30 , a sensor I/F 14 , a display section 34 , an operation input section 35 , a storage section 36 and a communication I/F 37 . The control unit 30 includes a CPU 31, a ROM 32, a RAM 33, a timer (not shown), and the like. A control program executed by the CPU 31 is stored in the ROM 32 . The RAM 33 provides a work area when the CPU 31 executes the control program.

The display unit 34 is configured by an LCD or the like, and displays various information. The display unit 34 may have two or more screens, or may have a function of displaying two or more screens by dividing the screen. The operation input unit 35 receives input of various instructions from the user and sends input information to the CPU 31 . Further, the operation input unit 35 may have a function of notifying the user by voice, lamp, or the like based on instructions from the CPU 31 . The storage unit 36 is composed of, for example, a non-volatile memory. The storage unit 36 may be an external memory. Communication I/F 37 performs wired or wireless communication between control unit 30 and an external device.

The CPU 31 drives the sensors 10A and 10B via the sensor I/F 14 and controls the timing of the measurement operation. As described above, CPU 31 controls sensors 10A and 10B so that each of sensors 10A and 10B sequentially outputs measurement results at equal time intervals (cycle T2) (see FIG. 4). At least in the protection area 41 (see FIG. 1), measurement results can be obtained at a frame rate of period T2, so high-speed object detection is possible.

Next, a description will be given of the process of assisting the installation of the sensors 10A and 10B and the setting of the protection area using indices. Techniques in this processing are broadly classified into first and second techniques. Either of the first and second techniques may be adopted.

<First method>
The first method will be described with reference to FIGS. 5 to 9. FIG.

FIG. 5 is a flow chart showing sensor installation processing and protection area setting processing in the first method. This sensor installation processing is executed by the user. The protection area setting process is realized by the CPU 31 developing a program stored in the ROM 32 in the RAM 33 and executing the program. The protection area setting process is started by a user's instruction, and is executed by the CPU 31 in parallel with the sensor installation process.

Since it is assumed that the desired protection area is a rectangular parallelepiped, the first method uses four markers M1 to M4 as indicators of the protection area. As shown in FIG. 1, the user arranges the markers M1 to M4 at the vertices of the bottom surface of the desired protection area on the floor surface as the reference surface. As the markers M1 to M4, for example, stickers are adopted, but they may be three-dimensional objects. In order to reliably perform distance measurement, the brightness of the markers M1 to M4 should be high.

Note that the markers M1 to M4 may not be independent members, and in order to accurately set the relative positional relationship of the markers M1 to M4, the markers may be connected by connecting members. In the first method, the user provides the control unit 30 with height information up to the vertex of the upper surface of the desired protection area. This height information is a known value indicating the position apart from the reference surface (floor surface) in the vertical direction corresponding to each marker, and is stored in the RAM 33 . The reference surface on which the markers M1 to M4 are placed is not limited to the floor surface, and may be a flat surface such as a desk surface.

First, in step S101, the user sets (arranges) the markers M1 to M4 on the floor at the apexes of the bottom surface of the desired protection area. Note that if the markers M1 to M4 have already been placed, the user may redo the placement of the markers M1 to M4. In step S102, the user installs the sensors 10A, 10B or, if installed, adjusts their positions as necessary. The positions of the sensors 10A and 10B include not only three-dimensional positions but also orientations. The user determines the positions and orientations of the sensors 10A and 10B so that the markers M1 to M4 are within the field of view (angle of view) of each sensor. In step S103, the user instructs the CPU 31 to perform distance measurement by the sensors 10A and 10B (distance measurement instruction). Distance measurement here includes acquisition of luminance information as well as distance information.

On the other hand, in step S201, the CPU 31 waits for a distance measurement instruction from the user, and when there is a distance measurement instruction, the process proceeds to step S202. In step S202, the CPU 31 causes the sensors 10A and 10B to perform distance measurement. Measurement results are alternately supplied to the CPU 31 from the sensors 10A and 10B. When the CPU 31 obtains the measurement result for a certain period of time, the CPU 31 executes a protection area setting process in step S203. In this protected area setting process, first, the CPU 31 causes the screen to display a brightness image corresponding to each sensor based on the brightness information output from each sensor among the measurement results.

FIG. 6 is a diagram showing examples of luminance images displayed on two screens. The display unit 34 displays a screen 34A and a screen 34B. Brightness images corresponding to the sensors 10A and 10B are displayed side by side on the screens 34A and 34B. In each luminance image, the portions corresponding to the markers M1 to M4 are particularly bright. The user can recognize the position of the apex of the bottom surface of the protected area on the screen from the four bright positions on the screen.

When the screens 34A and 34B are displayed, in step S104, the user instructs the control unit 30 of the positions of the markers M1 to M4 by specifying the positions where the markers M1 to M4 are displayed on the respective screens. do. The method of specifying the position here is not limited, but for example, an operation such as moving the mouse cursor to the corresponding position and pressing the confirm button is performed. You may input the coordinate on a screen with a numerical value.

When the position on the screen is specified, the CPU 31 further grasps the relative positional relationship between each sensor and the eight vertices of the protection area in the protection area setting process (S203) as follows.

FIG. 7 is a conceptual diagram showing how the relative positional relationships of the markers M1 to M4 are specified. In FIG. 7, markers M1 to M4 of the eight vertices of the protection area are shown for easy understanding.

First, the CPU 31 obtains three-dimensional coordinates of the markers M1 to M4 in a coordinate system based on the original position of the sensor 10A from the distance image based on the distance information obtained from the sensor 10A (F1 in FIG. 7). Similarly, the CPU 31 obtains the three-dimensional coordinates of the markers M1 to M4 in the coordinate system based on the original position of the sensor 10B from the distance image based on the distance information obtained from the sensor 10B (F2 in FIG. 7).

Then, the CPU 31 determines the distances from the pixels at the same position in each distance image to each of the markers M1 to M4. That is, for example, the CPU 31 determines that the three-dimensional coordinates of the marker M1 in F1 and the three-dimensional coordinates of the marker M1 in F2 are the same point. Similarly, the CPU 31 determines that the three-dimensional coordinates of the markers M2, M3 and M4 in F1 and the three-dimensional coordinates of the markers M2, M3 and M4 in F2 are the same point. Subject to these determinations, CPU 31 determines that markers M1-M4 in F1 point to the same positions as markers M1-M4 in F2 in the global coordinate system (XYZ), as indicated by F3. .

Although only four points are shown in FIG. 7, the height information corresponding to each marker is actually used to perform the processing shown in FIG. A relative positional relationship between each sensor and the eight vertices of the protected area is grasped. The height information is stored in the RAM 33 by being input by the user before execution of step S103. After determining the eight vertices in the global coordinate system, the CPU 31 sets the protection area 41 . Note that when the coordinates of the upper four vertices of the protection area are calculated based on the height information stored in the RAM 33, if the coordinates of the upper vertices are measured by either or both of the sensors 10A and 10B, If it is found to be outside the measurable area (that is, if the protected area is not within the measurable area of the sensor), the CPU 31 notifies the user to that effect, re-inputs the height information, or You may be prompted to adjust the position.

In step S204, the CPU 31 causes the set protection area 41 to be displayed. FIG. 8 is a diagram showing a display example of the protection area 41. As shown in FIG. As an example, the CPU 31 stereoscopically displays the protection area 41 together with the reference plane 40, which is the floor surface, on the screen at the angle of view of the sensor 10A. Thereby, the user can visually recognize the position of the protection area 41 in the current installation state of the sensors 10A and 10B. The protection area 41 may be displayed on the screen with the angle of view of the sensor 10B, or may be displayed side by side on two screens with the angles of view of the sensors 10A and 10B.

Here, regarding the setting of the protection area 41, the CPU 31 specifies the coordinates of each vertex of the protection area 41 in the global coordinate system (the coordinate system common to the sensors 10A and 10B), thereby obtaining a three-dimensional image occupied by the protection area 41. A region may be specified. Alternatively, the CPU 31 may define the three-dimensional area occupied by the protection area 41 by specifying depth information (range of mm to mm, etc.) for each pixel based on the original positions of the sensors 10A and 10B. .

In step S105, the user looks at the protection area 41 displayed on the screen, and if it is appropriate, inputs an OK instruction indicating that the protection area 41 is to be fixed. Enter redo instructions. On the other hand, in step S205, CPU 31 waits for an OK instruction or a redo instruction from the user. If an instruction to redo is input, the process returns from step S105 to step S101 and from step S205 to step S201. Therefore, in this case, the user can redo the placement of the markers in step S101 and redo the placement of the sensors 10A and 10B in step S102. Object detection system 100 waits for a distance measurement instruction and executes distance measurement again.

If the OK instruction is input, the processing shown in FIG. 5 ends. In this case, the setting of the protection area 41 is finalized, and the CPU 31 can proceed to subsequent processes such as object detection and safety control.

As described above, according to the first method, the protected area 41 is set by specifying eight vertices on the luminance image after the user sets the marker and measures the distance. Can assist with work.

In the first method, the markers M1 to M4 and the height information are used to set the protection area 41, but the present invention is not limited to this. may be used.

FIG. 9 is a diagram showing an example target with eight markers. This target includes first markers (markers M1 to M4) and second markers (markers M5 to M8) arranged at positions separated from the reference plane 40 in the vertical direction corresponding to each of the first markers. . The markers M1 to M4 on the bottom side are connected by a connecting member 42 to form a rectangle, and the markers M5 to M8 on the top side are also connected by a connecting member 42 to form a rectangle. Furthermore, the markers M1 to M4 and the markers M5 to M8 are also connected by a connecting member . Since the positions of the markers M1 to M4 and the markers M5 to M8 are fixed, it is not necessary to give the above height information. In other processing,
It is the same as that explained in FIG.

<Second method>
The second method will be described with reference to FIGS. 10 to 13. FIG.

FIG. 10 is a flow chart showing sensor installation processing and protection area setting processing in the second method. This sensor installation processing is executed by the user. The protection area setting process is realized by the CPU 31 developing a program stored in the ROM 32 in the RAM 33 and executing the program. The protection area setting process is started by a user's instruction, and is executed by the CPU 31 in parallel with the sensor installation process.

First, in step S301, the user places markers M1 to M3 on the floor. Here, since the markers M1 to M3 in the second method are indices that define a plane, they may be placed on the same plane so as not to line up on a straight line, and placed at the vertices of the bottom surface of the desired protection area. Not necessarily. Note that if the markers M1 to M3 have already been placed, the user may redo the placement of the markers M1 to M3. In step S302, the user installs the sensors 10A, 10B or, if already installed, adjusts their positions as needed. The user determines the positions and orientations of the sensors 10A and 10B so that the markers M1 to M3 are within the field of view (angle of view) of each sensor. Note that if there is already a mark on the floor to replace the marker, there is no need to install the marker again. For example, if a figure or pattern that serves as a mark is drawn on the floor surface, or if there is a characteristic object that serves as a mark, and the user can visually match when teaching the coordinates, the marker is used. can be an alternative to

In steps S303, S401 and S402, the same processes as in steps S103, S201 and S202 of FIG. 5 are executed. When the CPU 31 obtains the measurement result for the certain period of time, it executes overlapping space display processing in step S403. This processing will be described with reference to FIGS. 11 and 12. FIG.

FIG. 11 is a diagram showing a display example of a luminance image and a visual field range. Similar to FIG. 6, luminance images corresponding to the sensors 10A and 10B are displayed side by side on the screens 34A and 34B. In each luminance image, the portions corresponding to the markers M1 to M3 are particularly bright.

CPU 31 defines a reference plane 40 based on each of sensors 10A and 10B based on the distance information in the measurement result. That is, since the positions of the markers M1 to M3 on the screen 34A and the positions of the markers M1 to M3 on the screen 34B point to the same positions in the global coordinate system, the CPU 31 determines the positions of the markers M1 to M3. , and the distance information obtained from the sensors 10A, 10B, each reference plane 40 can be defined.

Then, for example, the CPU 31 three-dimensionally displays the visual field range 43 of the sensor 10B on the screen 34A together with the defined reference plane 40 in the spatial coordinate system with the sensor 10A as a reference. The visual field range 43 has a three-dimensional shape corresponding to the measurable area RB (FIG. 1). In the screen 34B, the visual field range of the sensor 10A (three-dimensional shape corresponding to the measurable area RA) may be stereoscopically displayed together with the defined reference plane 40 in the spatial coordinate system based on the sensor 10B. . Both of these visual field ranges may be displayed side by side on two screens.

After that, for example, the display of the visual field range 43 is changed to the display of the overlapping space Rx according to an instruction from the user. FIG. 12 is a schematic conceptual diagram showing a display example of the overlapping space Rx. The shape of the overlapping space Rx is determined by how the visual field ranges (measurable regions) of the sensors 10A and 10B overlap, and in many cases it has a polyhedral shape, which is schematically shown as a sphere in FIG.

As shown in FIG. 12, the CPU 31 three-dimensionally displays the overlapping space Rx together with the defined reference plane 40 in the spatial coordinate system with the sensor 10A as a reference, for example, on the screen 34A. In the screen 34B, the overlapping space Rx may be three-dimensionally displayed together with the defined reference plane 40 in the spatial coordinate system based on the sensor 10B. Both of these may be displayed side by side on two screens.

When the overlapped space Rx is displayed, in step S304, the user inputs an OK instruction indicating that this will move to the designation of the apex of the protection area, or a redo instruction indicating that the process should be redone. For example, if the user determines that the desired protection area is sufficiently included in the overlapping space Rx, the user inputs an OK instruction. On the other hand, in step S404, CPU 31 waits for an OK instruction or a redo instruction from the user. If an instruction to redo is input, the process returns from step S304 to step S301, and from step S404 to step S401. Therefore, in this case, the user can redo the placement of the markers in step S301 and redo the placement of the sensors 10A and 10B in step S302. The CPU 31 waits for a distance measurement instruction and executes distance measurement again.

When the OK instruction is input, the process proceeds from step S304 to step S305, and from step S404 to step S405. In step S305, the user designates position information (e.g., positions of eight points) indicating the vertices of the desired protection area in the spatial coordinate system in which the overlapping space Rx is displayed. Usually the vertex is specified inside the overlapping space Rx. The input method at this time may be any method as long as it can specify a three-dimensional shape. For example, numerical values indicating the coordinates of individual vertices on the screen and distance information may be input.

When the vertex of the protected area is designated, in step S405, the CPU 31 sets and displays the protected area 41 according to the designated point. The display mode of the protection area 41 at this time is the same as that shown in FIG. Thereby, the user can visually recognize the position of the protection area 41 in the current installation state of the sensors 10A and 10B.

Here, the specification of the three-dimensional coordinates of each vertex of the protection area 41 by the CPU 31 is the same as described in step S203. In steps S306 and S406, processing similar to steps S105 and S205 in FIG. 5 is executed. For example, if an instruction to redo is input, the process returns from step S306 to step S301 and from step S406 to step S401. Therefore, in this case, the user can redo the placement of the markers in step S301 and redo the placement of the sensors 10A and 10B in step S302. Object detection system 100 waits for a distance measurement instruction and executes distance measurement again.

FIG. 13 is a diagram showing a display example of the brightness image and the visual field range after redoing, and corresponds to FIG. As an example, a case where only the sensor 10B is installed is changed. Since the installation position of the sensor 10B has changed from the installation state corresponding to FIG. 11, the contents displayed on the screen 34B in FIG. 13 are different from those in FIG. Accordingly, the visual field range 43 of the sensor 10B displayed on the screen 34A also changes.

If the OK instruction is input, the processing shown in FIG. 10 ends. In this case, the setting of the protection area 41 is finalized, and the CPU 31 can proceed to subsequent processes such as object detection and safety control.

Thus, according to the second method, after the user sets the marker and measures the distance, the protection area 41 is set by specifying eight vertices in the spatial coordinate system in which the overlapping space Rx is displayed. Therefore, the setting work of the protection area 41 can be assisted. In addition, even when the overlapping space Rx is displayed, the placement of markers and sensors can be redone, so that finer adjustments can be made.

According to the present embodiment, CPU 31 controls to shift the output timings of distance information from sensors 10A and 10B arranged so that at least a part of mutual measurable areas RA and RB overlap each other. Thereby, the response speed of object detection can be increased.

In particular, since each of the sensors 10A and 10B is controlled to sequentially output distance information at equal time intervals (T2=T1/2), the response speed can be efficiently increased. Moreover, by arranging each of the sensors 10A and 10B so that the protection area 41 is included in the overlapping space Rx in which the measurable regions overlap each other, a high response speed in the protection area 41 can be ensured.

Further, in the first method (FIG. 5), the CPU 31 displays on the screen a brightness image obtained when the distance is measured with the marker serving as the index of the desired protection area placed on the screen (FIG. 6). A protection area 41 is set based on the designation of the position from and the distance information (FIG. 8). Further, in the second method (FIG. 10), the CPU 31 defines a reference plane 40 based on each sensor based on the distance information when the distance is measured with the markers serving as the indices defining the plane being arranged. do. Then, the CPU 31 three-dimensionally displays the mutually overlapping measurable regions of both sensors as an overlapping space Rx together with the defined reference plane 40 in the spatial coordinate system (FIG. 12). Then, based on the position specified by the user on the screen and the distance information, the protection area 41 is set (FIG. 8). Therefore, it is possible to assist the work of setting the protection area.

<Modification>
The above-described embodiment is merely an example of the configuration of the present invention. The present invention is not limited to the specific forms described above, and various modifications are possible within the technical scope of the present invention.

From the standpoint of obtaining the effect of speeding up the response speed of object detection by shifting the output timing of distance information, the three-dimensional distance sensor used is at least a sensor that periodically outputs three-dimensional distance information. Other types of sensors, if any, may be employed. When a TOF sensor is employed, it may be either of a direct type (direct type) or an indirect type (indirect type).

As described with reference to FIG. 4, each of the sensors 10A and 10B was controlled to sequentially output the measurement results at equal time intervals (cycle T2). Even if the output timings are not at equal time intervals, if they are offset, the effect of speeding up the response speed of object detection can be obtained as compared to detecting with only a single sensor. Also, although the frame period T1 of the sensors 10A and 10B is common, it does not necessarily have to be common.

Also, even when three or more three-dimensional distance sensors are used, it is possible to apply control to shift the output timing. The response speed can be efficiently increased by controlling the output timing of each sensor to be at equal time intervals.

Note that the shape of the protection area is not limited to a polyhedron such as a rectangular parallelepiped. In the first method, the number of markers used for arranging the vertices of the bottom surface of the protected area does not have to be four, and may be three or more.

In addition, in the second method, markers are used as indices for defining a plane, but the number of markers may be four or more. Alternatively, it may be a sheet of known shape. Further, in the second method, the configuration may be such that the control unit 30 can automatically drive the sensors 10A and 10B. In such a configuration, for example, after the protected area 41 is set by the user specifying eight positions in the spatial coordinate system in which the overlapping space Rx is displayed, the protected area 41 is positioned at the center of the spatial coordinate system. The controller 30 may drive the sensors 10A, 10B so as to position them.

It should be noted that it is not essential to set a protection area if the effect of speeding up the object detection response speed is obtained by shifting the output timings of the distance information.

From the standpoint of obtaining the effect of ensuring a high response speed in the protected area, it is not essential to use the above first and second techniques regardless of the technique for setting the protected area.

<Appendix>
[1] a plurality of imaging means (10A, 10B) arranged so that at least parts of the measurable regions (RA, RB) overlap with each other, and periodically outputting distance information of the measurable regions;
an object detection means (30) for detecting an object based on the distance information output by the plurality of imaging means;
a control means (30) for controlling output timings of the distance information by the plurality of imaging means so as to be shifted from each other;
An object detection system (100) comprising:

[2] a working machine;
a plurality of imaging means (10A, 10B) installed so as to be able to image a common area (Rx), each of which periodically images at a predetermined time interval;
an object detection means (30) for detecting an object in the area (Rx) based on the images output from the plurality of imaging means (10A, 10B);
A controller (50) for controlling operation in a safety control system comprising:
an imaging control means (30) for controlling the plurality of imaging means (10A, 10B) so as to shift imaging timings of each other by a time shorter than the time interval;
A control device (50) comprising: safety control means (30) for limiting the operation of the work machine based on the detection result of the object detection means (30).

10A, 10B: three-dimensional distance sensor 30: control unit 31: CPU
41: Protection area 100: Object detection system RA, RB: Measurable area Rx: Overlapping space T1: Frame period T2: Period

Claims (11)

a plurality of imaging means arranged so that at least a part of each other's measurable regions overlaps, and periodically outputting distance information within the measurable regions;
an object detection means for detecting an object based on the distance information output by the plurality of imaging means;
a control means for controlling output timings of the distance information by the plurality of imaging means so as to be shifted from each other;
An object detection system comprising: an output cycle of the distance information by each of the imaging means is common,
2. The object detection system according to claim 1, wherein said control means controls said plurality of imaging means to output said distance information at equal time intervals. When the number of imaging means is n (n is an integer equal to or greater than 2) and the output period is T,
3. The object detection system according to claim 2, wherein said control means controls such that said distance information is output from said n imaging means at time intervals of T/n. Having setting means for setting a virtual protection area for object detection,
3. Each of said imaging means is arranged so that the protection area set by said setting means is included in an overlapping space in which the measurable regions of said plurality of said imaging means overlap each other. 4. The object detection system according to any one of 1 to 3. Having setting means for setting a virtual protection area for object detection,
each of the imaging means outputs luminance information together with the distance information;
The control means causes a brightness image corresponding to each of the imaging means to be displayed on a screen based on the brightness information output by each of the imaging means with a marker serving as an index of a desired protection area arranged. ,
4. The setting device according to claim 1, wherein said setting means sets said virtual protection area based on a user's designation of a position on said screen and said output distance information. 2. The object detection system according to item 1. three or more of the markers are arranged on a reference plane;
The setting means includes a user's designation of a position on the screen, the output distance information, and known height information indicating a position away from the reference plane in the vertical direction corresponding to each of the markers. 6. The object detection system according to claim 5, wherein the virtual protection area is set based on and. The markers include three or more first markers arranged on a reference plane, and second markers arranged at positions separated from the reference plane in the vertical direction corresponding to each of the first markers. 6. The object detection system according to claim 5, characterized in that: The control means defines a reference plane based on each of the imaging means based on the distance information output by each of the imaging means in a state in which an index defining a plane is placed on the reference plane, and a plurality of three-dimensionally displaying the overlapping measurable regions of the plurality of imaging means as the overlapping space in a spatial coordinate system based on any one of the imaging means of let
5. The object detection system according to claim 4, wherein said setting means sets said virtual protection area based on an instruction from a user in said spatial coordinate system. 9. The object detection system of claim 8, wherein the indicia defining the plane are at least three markers. 10. The object detection system according to any one of claims 1 to 9, wherein said imaging means is a TOF sensor. a working machine;
a plurality of imaging means installed so as to be able to image a common area, each of which periodically images at a predetermined time interval;
and object detection means for detecting an object in the area based on the images output from the plurality of imaging means, and
imaging control means for controlling the plurality of imaging means so as to shift imaging timings of each other by a time shorter than the time interval;
and safety control means for restricting the operation of the work machine based on the detection result of the object detection means.

Images