JP7490452B2 - Working robot operation support device, method, and program - Google Patents

Description

An embodiment of the present invention relates to an operation support technology for a remotely controlled work robot.

When remotely operating a work robot such as an articulated manipulator, the work status is monitored using a surveillance camera to avoid collisions with the target structure or surrounding structures. For this reason, depending on the posture of the work robot during operation, the surveillance camera's view may be blocked, creating blind spots, making it difficult to monitor the work status.

The following known techniques have been proposed as methods for eliminating blind spots in the field of view of such surveillance cameras. The first is a known technique in which multiple surveillance cameras are installed in a work area, and a sub-image from one surveillance camera is composited into the blind spot area of the main image from another surveillance camera. The second is a known technique in which a collision-free operation of a work robot is planned in a virtual space based on the three-dimensional shape of structures placed in the work area. Actual measurement data and design data for the work area are used for this three-dimensional shape.

JP 2006-53922 A

However, in the first known technology mentioned above, the number and positions of surveillance cameras that can be installed in the work area may be limited due to constraints of the work environment. In addition, synthesizing a sub-image in the blind spot area of the main image means fitting an image from a different viewpoint, which may cause the continuity of the posture of the reflected structure to be lost, raising concerns that this may lead to erroneous operation during remote work. This concern becomes more pronounced the greater the spacing between multiple surveillance cameras so that they can efficiently complement each other's blind spots.

Furthermore, in the second known technology mentioned above, when design data is used as the three-dimensional shape, there is an issue of lack of reliability, as the shape of the structure placed in the work area may differ from the actual shape, and the technology may be unable to respond to environmental changes such as when the structure is transported or removed. One solution to this issue is to use actual measurement data of the work area. However, if part of the work robot is included in the measurement area of this actual measurement data, there is an issue of not being able to recognize the shape of the structure placed in the work area.

The embodiment of the present invention has been made in consideration of these circumstances, and aims to provide an operation support technology for a work robot that can recognize the shape of structures in a work area with high reliability.

In an embodiment, the operation support device for a working robot includes a receiving unit that sequentially receives measurement signals of a structure measured by a shape measuring sensor in a working area, a coordinate conversion unit that converts point cloud data including a three-dimensional shape of the structure reproduced from the measurement signals from a coordinate system of the shape measuring sensor to a coordinate system of the working area, a development unit that develops three-dimensional data indicating the three-dimensional shape of the working robot into the coordinate system of the working area based on control information, a detection unit that detects an overlapping area between the point cloud data and the three-dimensional data in the coordinate system of the working area, and a generation unit that generates the point cloud data as a shape map of the working area by overwriting and updating only an update area excluding the overlapping area based on the measurement signals of the latest timing sequentially received, a calculation unit that calculates an elapsed time obtained by taking the difference between a measurement time of the shape measuring sensor associated with each of the point cloud data constituting the shape map and a current time, and a reliability evaluation unit that evaluates the reliability of each of the point cloud data based on the elapsed time, and performs initialization to evacuate the working robot from the measurement area of the shape measuring sensor based on the reliability evaluation parameter. The present invention is characterized in that it includes a calculation unit that calculates an elapsed time obtained by taking the difference between a measurement time of the shape measuring sensor associated with each of the point cloud data constituting the shape map and a current time, and a reliability evaluation unit that evaluates the reliability of each of the point cloud data based on the elapsed time, and that it changes the position of the shape measuring sensor and changes the measurement area based on the reliability evaluation parameter.It is also characterized in that it includes a calculation unit that calculates an elapsed time obtained by taking the difference between a measurement time of the shape measuring sensor associated with each of the point cloud data constituting the shape map and a current time, and a reliability evaluation unit that evaluates the reliability of each of the point cloud data based on the elapsed time, and that it evaluates the reliability of the point cloud data based on the elapsed time, and that the reliability is evaluated based on the distance between the point cloud data of the structure and the three-dimensional data of the work robot.

Embodiments of the present invention provide a technology to assist the operation of a work robot that can recognize the shape of structures in a work area with high reliability.

1 is a configuration diagram of a working robot operation assistance device according to a first embodiment of the present invention; FIG. 13 is a configuration diagram showing an initialized working robot. The 3D shape of the work robot is represented by point cloud data measured by a camera immediately after initialization. A three-dimensional shape expressed as point cloud data measured by a camera during the operation of a work robot. A shape map of the working area generated during the operation process of the work robot. FIG. 11 is a configuration diagram of a working robot operation assistance device according to a second embodiment of the present invention. 3 is a flowchart illustrating an embodiment of a method and program for assisting operation of a working robot.

First Embodiment
Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. Fig. 1 is a configuration diagram of a working robot operation support device 10A according to a first embodiment of the present invention. As described above, the operation support device 10A (10) includes a measurement signal receiving unit 11, which is a receiving unit that sequentially receives measurement signals 26 of a structure 25 (25a, 25b) measured by a camera 21, which is a shape measurement sensor, in a working area 20, a coordinate conversion unit 12 that converts point cloud data 27 including the three-dimensional shape of the structure 25 reproduced from the measurement signals 26 from the coordinate system of the camera 21 to the coordinate system of the working area 20, a development unit 15 that develops stereoscopic data 28 showing the three-dimensional shape of the working robot 22 into the coordinate system of the working area 20 based on control information 31, an overlapping area detection unit 16, which is a detection unit that detects an overlapping area 35 between the point cloud data 27 and the stereoscopic data 28 in the coordinate system of the working area 20, and a shape map generation unit 17, which is a generation unit that updates the point cloud data 27 in an update area 36 (Fig. 5) excluding the overlapping area 35 and generates a shape map 37 of the working area 20.

In this embodiment, an articulated manipulator is exemplified as the work robot 22, and the work is exemplified as grasping and moving a target structure 25a (structure 25) to be worked on in the work area 20. The work robot 22 is remotely controlled by an operator (not shown), and the operator grasps the work status for the target structure 25a via video images from the camera 21.

The work robot 22 is equipped with a gripper 34 at its hand and is placed in the work area 20. Note that, although the work robot 22 is exemplified as one end of which is fixed to the floor surface of the work area 20, there are no limitations on the installation form. Also, a multi-jointed manipulator can be mounted on a cart (not shown) and moved to any position in the work area 20. In this case, it is necessary to measure the movement position of the cart in the coordinate system of the work area 20 and reflect the position information of the work robot 22, such as the manipulator, in the processing of the deployment unit 15.

In addition to the target structure 25a, surrounding structures 25b are also arranged as structures 25 in the work area 20. The operator (not shown) performs operations on the target structure 25a while paying attention to mechanical interference between the surrounding structures 25b and the work robot 22.

The camera 21 is installed in the work area 20 and is a device capable of measuring the three-dimensional shape of the structure 25. Examples of the camera 21 include a device that measures the shape by stereoscopic vision by combining multiple optical imaging devices, or a device that measures the shape by a laser range finder or a distance image sensor, and a general shape measurement sensor that is a measuring device for three-dimensional shapes can be used.

The camera 21 then processes the measured surface shape of the structure 25 as point cloud data in the camera coordinate system, and sequentially transmits the measured shape as a measurement signal 26. The measurement signal receiving unit 11 then sequentially receives the measurement signals 26 transmitted from the camera 21, and captures the point cloud data 27 as a moving image when there is movement in the work robot 22 or the structure 25.

The coordinate conversion unit 12 converts the point cloud data 27, which is defined in the coordinate system of the camera 21 in the measurement signal 26, so that it is defined in the coordinate system of the work area 20. Here, the coordinate system of the work area 20 is a coordinate system for reproducing the point cloud data 27 of the structure 25 received from the camera 21 and the three-dimensional data 28 of the work robot 22 deployed from the control information 31 in a common three-dimensional space. Note that, as will be described later, this point cloud data 27 may include not only the structure 25 but also part of the work robot 22 as the three-dimensional shape within the work area 20.

The work robot 22 is composed of multiple links 38, servo motors 39 that connect the ends of each link 38 and function as joints, and a gripper 34 for gripping the target structure 25a. The work robot control unit 30 controls the operation of the servo motors 39 that make up each joint and controls the posture of the work robot 22 according to a predetermined motion plan or operation commands from an operator (not shown).

The work robot control unit 30 not only sends control signals 33 for driving the servo motor 39 etc. directly to the work robot 22, but also sends control information 31 indicating the drive amount of the servo motor 39 to the operation assistance device 10. Note that this control information 31 may also be transmitted directly from the work robot 22, rather than from the work robot control unit 30.

The unfolding unit 15 virtually reproduces three-dimensional data 28 indicating the three-dimensional shape (posture) of the working robot 22 from a robot part shape model 32, which is a shape model of the parts of the working robot 22 (e.g., link 38), and control information 31 of the parts of the working robot 22 (e.g., servo motor 39). The robot part shape model 32 can be prepared in advance using design information such as three-dimensional CAD data, and the control information 31 can use real-time angular displacement data of the servo motor 39. As a result, the three-dimensional data 28 is unfolded while being sequentially updated in the coordinate system of the working area 20 shared with the point cloud data 27, and its movement is virtually reproduced.

Figure 2 is a configuration diagram showing an initialized work robot 22. Figure 3 shows the three-dimensional shape represented by point cloud data 27 measured by camera 21 immediately after the work robot 22 is initialized. Initialization means making the work robot 22 assume a posture that retreats from the measurement area of camera 21.

Figure 4 shows the three-dimensional shape represented by point cloud data 27 measured by camera 21 during the operation of work robot 22. In this way, when work robot 22 gets between camera 21 and structure 25, the acquired point cloud data 27 includes the three-dimensional shape of work robot 22. In other words, the three-dimensional shape of structure 25 that is in the blind spot of work robot 22 in the measurement area of camera 21 is not reflected in point cloud data 27.

From the point cloud data 27 itself reproduced from the measurement signal 26, it is not possible to distinguish whether the three-dimensional shape corresponds to the structure 25 or the work robot 22. Therefore, when capturing images with the camera 21, the work robot 22 is initialized before starting the capture, and the point cloud data 27 that accurately reflects the arrangement of the structure 25 in the work area 20 is acquired.

Returning to FIG.
When work robot 22 is captured in the measurement area of camera 21 during the operation of work robot 22, an overlap area 35 between point cloud data 27 and three-dimensional data 28 occurs in the captured portion in the coordinate system of work area 20. Overlap area detection unit 16 compares point cloud data 27 and three-dimensional data 28 in the coordinate system of work area 20, and detects overlap area 35 between them.

Figure 5 shows a shape map 37 of the work area 20 generated during the operation of the work robot 22. Now, in the measurement signals 26 that are sequentially received, let us assume that the frame switches from point cloud data 27 of only the structure 25 as shown in Figure 3 to point cloud data 27 that includes the work robot 22 as shown in Figure 4. The overlapping area 35 is detected for the first time at the timing of the switch from Figure 3 to Figure 4.

Returning to FIG.
The shape map generator 17 sets the point cloud data 27 constituting the frame of the latest timing as shown in Fig. 4 as an update region 36 (Fig. 5) excluding the overlap region 35. Then, it overwrites and updates only the update region 36 portion of the point cloud data 27 of the frame of the timing shown in Fig. 3 immediately before that.

As a result, as shown in FIG. 5, a shape map 37 is generated in which the update area 36 is composed of the point cloud data 27 from the latest timing, and the overlap area 35 is composed of the point cloud data 27 from the timing immediately prior to that. This makes it possible to grasp the three-dimensional shape of the structure 25 in the blind spot even if the work robot 22 is captured in the measurement area of the camera 21.

In addition, assume that the work involves movement of the three-dimensional shape of the structure 25, such as gripping and moving the target structure 25a with the work robot 22. Even in this case, the point cloud data 27 of the update area 36 that is not in the blind spot of the work robot 22 is updated to the latest state. Therefore, the gripped and moving target structure 25a can be accurately tracked by video, and the operation of the work robot 22 can be performed appropriately.

The monitor 18 displays a moving image of the shape map 37. The operator operates the work robot 22 while visually viewing the moving image on the monitor 18. The shape map 37 is also fed back to the work robot control unit 30, and is involved in the operation control of the work robot 22, which enhances safety by, for example, operating the work robot 22 at a slower speed near areas that are blind spots of the work robot 22 and cannot be measured.

Second Embodiment
Next, a second embodiment of the present invention will be described with reference to Fig. 6. Fig. 6 is a configuration diagram of a working robot operation support device 10B according to the second embodiment of the present invention. In Fig. 6, parts having the same configuration or function as Fig. 1 are indicated by the same reference numerals, and duplicated explanations will be omitted.

The operation support device 10B (10) of the second embodiment includes, in addition to the configuration of the operation support device 10A (Figure 1) for a work robot of the first embodiment, an elapsed time calculation unit 41 that is a calculation unit that calculates an elapsed time 46 based on the camera measurement time 45 associated with each point cloud data 27 that constitutes the shape map 37, and a reliability evaluation unit 42 that evaluates the reliability of each point cloud data 27 based on the elapsed time 46.

The shape map generation unit 17 associates each of the point cloud data 27 constituting the shape map 37 with a camera measurement time 45 when it was measured by the camera 21. The elapsed time calculation unit 41 then acquires the camera measurement time 45 associated with each of the point cloud data 27 constituting the shape map 37, and calculates the elapsed time 46 by taking the difference from the current time.

This elapsed time 46 is essentially zero for point cloud data 27 belonging to the update region 36, and for point cloud data 27 belonging to the overlap region 35, the longer the point cloud data 27 has been in the overlap region 35, the longer the elapsed time 46 becomes. In other words, an area of the shape map 37 composed of point cloud data 27 with a long elapsed time 46 reflects an old work state and is therefore unreliable.

The reliability evaluation unit 42 evaluates the reliability of each point cloud data 27 based on the elapsed time 46. The reciprocal of the elapsed time 46 is defined as a reliability evaluation parameter 47. Based on the evaluation parameter 47 defined in this way, the fact that the point cloud data 27 has not been updated for a long period of time can be clearly indicated in the corresponding area of the shape map 37 displayed on the monitor 18, and the operator can be alerted.

This evaluation parameter 47 is also fed back to the work robot control unit 30, and control is implemented to enhance safety, such as by slowing down the work robot 22 in the vicinity of areas that are blind spots of the work robot 22 and cannot be measured. Furthermore, if the evaluation parameter 47 falls below a reference value and reliability has decreased to an unacceptable level, the work robot 22 is initialized and the work robot 22 is moved out of the measurement area of the camera 21. As a result, the overlapping area 35 in the shape map 37 is eliminated, and the entire area becomes the updated area 36, improving the evaluation parameters 47 in all point cloud data 27 and restoring reliability.

Furthermore, in the operation support device 10B of the second embodiment, the camera 21 is supported by an adjustment mechanism 48 that can freely adjust the measurement area. Although a multi-joint arm mechanism is exemplified as this adjustment mechanism 48, there is no particular limitation, and a mechanism with pan/tilt adjustment or a linear movement mechanism may also be used.

This adjustment mechanism unit 48 inputs a drive signal 49 output from the camera position drive unit 40, and adjusts the position and angle of the camera 21 in the work area 20. The camera position drive unit 40 receives evaluation parameters 47 of the point cloud data 27 from the reliability evaluation unit 42, and adjusts the position and angle of the camera 21 so that the surface of the structure 25 corresponding to the low-reliability point cloud data 27 is included in the measurement area. In this way, the evaluation parameters 47 change the measurement area of the camera 21 to increase the reliability of the point cloud data 27.

Furthermore, the operation support device 10B of the second embodiment can evaluate the reliability of each point cloud data 27 based on the distance between the point cloud data 27 of the structure 25 and the three-dimensional data 28 of the work robot 22. The work robot 22 has a wide operating range in the work area 20, but the work area actually used is often limited. In particular, when the work robot 22 performs work close to the target structure 25a, the work area is limited to only the vicinity of the target structure 25a.

Therefore, the reliability evaluation parameter 47 is set lower for point cloud data 27 closer to the work position of the work robot 22 since it is considered to be more important, and conversely, the evaluation parameter for point cloud data 27 farther from the work position is set higher.

The measurement area by the camera 21, whose position and angle are fixed, is limited within the wide work area 20. Therefore, the surface area of the structure 25 with a lower evaluation parameter 47 (closer to the work robot 22) is more likely to become an update area 36 of the point cloud data 27, and the position and angle of the camera 21 are adjusted to change the measurement area. This ensures high reliability of the point cloud data 27 that forms the work area near the work robot 22, even if the work area 20 is wide.

Furthermore, the point cloud data 27 forming the work area near the work robot 22 can be made more reliable while generating a shape map 37 covering the entire vast work area 20. This allows the shape map 37 to be used in a wider range of motion plans for the work robot 22, such as when approaching the target structure 25a from a distance. Note that while the present embodiment aims to increase the reliability of the point cloud data 27 constituting the area near the target structure 25a that is the work target of the work robot 22, it is not limited to this, and areas for which the reliability of the point cloud data 27 is increased may be set individually depending on the characteristics of the work area 20 and the purpose of the work.

An embodiment of a method and program for supporting the operation of a work robot will be described based on the flowchart in Figure 7 (see Figure 6 as appropriate). Measurement signals 26 of a structure 25 (25a, 25b) measured by a camera 21, which is an example of a shape measurement sensor, are sequentially received (S11). Furthermore, point cloud data 27 including the three-dimensional shape of the structure 25 reproduced from this measurement signal 26 is converted from the coordinate system of the camera 21 to the coordinate system of the work area 20 (S12).

Next, the stereoscopic data 28 showing the three-dimensional shape of the work robot 22 is expanded into the coordinate system of the work area 20 based on the control information 31 (S13). Then, an overlapping area 35 between the point cloud data 27 and the stereoscopic data 28 is detected in the coordinate system of the work area 20 (S14). Furthermore, the point cloud data 27 is updated in an update area 36 (FIG. 5) excluding this overlapping area 35 (S15), and a shape map 37 of the work area 20 is generated and displayed on the monitor 18 (S16).

Next, the elapsed time 46 is calculated based on the camera measurement time 45 associated with each point cloud data 27 that constitutes the shape map 37 (S17), and an evaluation parameter for the reliability of each point cloud data 27 is calculated based on this elapsed time 46. If this evaluation parameter falls below a reference value (S18 Yes), a warning is displayed in the corresponding area of the shape map 37 on the monitor 18, and the work robot 22 is initialized (the work robot 22 is evacuated from the corresponding area) or the position and angle of the camera 21 is adjusted (S19). Then, the flow from (S11) to (S19) is repeated until the operation of the work robot 22 is completed (S20 No: Yes END).

According to at least one of the embodiments of the work robot operation support device described above, it is possible to detect overlapping areas between the camera's point cloud data and the work robot's three-dimensional data, thereby making it possible to recognize the shape of structures in the work area with high reliability.

Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations can be made without departing from the gist of the invention. These embodiments and their variations are within the scope of the invention and its equivalents as well as the scope and gist of the invention. In addition, the components of the operation assistance device can be realized by a computer processor and can be operated by an operation assistance program.

The work robot operation support device 10 described above includes a control device with a highly integrated processor such as a dedicated chip, FPGA (Field Programmable Gate Array), GPU (Graphics Processing Unit), or CPU (Central Processing Unit), a storage device such as ROM (Read Only Memory) or RAM (Random Access Memory), an external storage device such as HDD (Hard Disk Drive) or SSD (Solid State Drive), a display device such as a display, an input device such as a mouse or keyboard, and a communication I/F, and can be realized with a hardware configuration using a normal computer.

The operation support device program for the work robot is provided by being pre-installed in a ROM or the like. Alternatively, the program may be provided by being stored in a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD, or flexible disk (FD) in the form of an installable or executable file.

The operation assistance program for the work robot according to this embodiment may be stored on a computer connected to a network such as the Internet and provided by downloading it via the network. The device 10 may also be configured by combining separate modules that independently perform the functions of the components and are interconnected via a network or dedicated lines.

10 (10A, 10B)...operation support device, 11...measurement signal receiving unit (receiving unit), 12...coordinate conversion unit, 15...expansion unit, 16...overlap area detection unit (detection unit), 17...shape map generation unit (generation unit), 18...monitor, 20...work area, 21...camera (shape measurement sensor), 22...work robot, 25...structure, 25a...target structure, 25b...surrounding structure, 26...measurement signal, 27...point cloud data, 28...three-dimensional data, 30...work robot control unit, 31...control information, 32...robot part shape model, 33...control signal, 34...gripper, 35...overlap area, 36...update area, 37...shape map, 38...link, 39...servo motor, 40...camera position drive unit, 41...elapsed time calculation unit (calculation unit), 42...reliability evaluation unit, 45...camera measurement time, 46...elapsed time, 47...evaluation parameter, 48...adjustment mechanism unit, 49...drive signal.

Claims (9)

a receiving unit that sequentially receives measurement signals of the structure measured by the shape measuring sensor in the work area;
a coordinate conversion unit that converts point cloud data including a three-dimensional shape of the structure reproduced from the measurement signals from a coordinate system of the shape measurement sensor into a coordinate system of the work area;
a development unit that develops stereoscopic data indicating a three-dimensional shape of the working robot into a coordinate system of the working area based on control information;
a detection unit that detects an overlapping area between the point cloud data and the three-dimensional data in a coordinate system of the work area;
a generation unit that generates, as a shape map of the work area, the point cloud data in which only an update area excluding the overlap area is overwritten and updated based on the sequentially received measurement signals at the latest timing,
a calculation unit that calculates an elapsed time by taking a difference between a measurement time of the shape measuring sensor associated with each of the point cloud data constituting the shape map and a current time;
a reliability evaluation unit that evaluates reliability of each of the point cloud data based on the elapsed time,
A work robot operation support device that executes initialization to retreat the work robot from a measurement area of the shape measurement sensor based on the reliability evaluation parameter. a receiving unit that sequentially receives measurement signals of the structure measured by the shape measuring sensor in the work area;
a coordinate conversion unit that converts point cloud data including a three-dimensional shape of the structure reproduced from the measurement signals from a coordinate system of the shape measurement sensor into a coordinate system of the work area;
a development unit that develops stereoscopic data indicating a three-dimensional shape of the working robot into a coordinate system of the working area based on control information;
a detection unit that detects an overlapping area between the point cloud data and the three-dimensional data in a coordinate system of the work area;
a generation unit that generates, as a shape map of the work area, the point cloud data in which only an update area excluding the overlap area is overwritten and updated based on the sequentially received measurement signals at the latest timing,
a calculation unit that calculates an elapsed time by taking a difference between a measurement time of the shape measuring sensor associated with each of the point cloud data constituting the shape map and a current time;
a reliability evaluation unit that evaluates reliability of each of the point cloud data based on the elapsed time,
An operation support device for a working robot that changes the position of the shape measurement sensor and changes the measurement area based on the reliability evaluation parameter. a receiving unit that sequentially receives measurement signals of the structure measured by the shape measuring sensor in the work area;
a coordinate conversion unit that converts point cloud data including a three-dimensional shape of the structure reproduced from the measurement signals from a coordinate system of the shape measurement sensor into a coordinate system of the work area;
a development unit that develops stereoscopic data indicating a three-dimensional shape of the working robot into a coordinate system of the working area based on control information;
a detection unit that detects an overlapping area between the point cloud data and the three-dimensional data in a coordinate system of the work area;
a generation unit that generates, as a shape map of the work area, the point cloud data in which only an update area excluding the overlap area is overwritten and updated based on the sequentially received measurement signals at the latest timing,
a calculation unit that calculates an elapsed time by taking a difference between a measurement time of the shape measuring sensor associated with each of the point cloud data constituting the shape map and a current time;
a reliability evaluation unit that evaluates reliability of each of the point cloud data based on the elapsed time,
An operation support device for a working robot, wherein the reliability is evaluated also based on the distance between the point cloud data of the structure and the three-dimensional data of the working robot. A receiving unit sequentially receives measurement signals of the structure measured by the shape measurement sensor in the work area;
a step of converting point cloud data including a three-dimensional shape of the structure reproduced from the measurement signals from a coordinate system of the shape measurement sensor into a coordinate system of the work area by a coordinate conversion unit;
a developing unit developing, based on control information, stereoscopic data indicating a three-dimensional shape of the working robot into a coordinate system of the working area;
a detection unit detecting an overlapping area between the point cloud data and the three-dimensional data in a coordinate system of the work area;
a generating unit generating the point cloud data by overwriting and updating only an update area excluding the overlap area based on the sequentially received measurement signals at the latest timing, as a shape map of the work area,
a calculation unit calculating an elapsed time by taking a difference between a measurement time of the shape measurement sensor associated with each of the point cloud data constituting the shape map and a current time;
A reliability evaluation unit evaluates reliability of each of the point cloud data based on the elapsed time;
and executing an initialization step of retracting the work robot from a measurement area of the shape measurement sensor based on the reliability evaluation parameter. A receiving unit sequentially receives measurement signals of the structure measured by the shape measurement sensor in the work area;
a step of converting point cloud data including a three-dimensional shape of the structure reproduced from the measurement signals from a coordinate system of the shape measurement sensor into a coordinate system of the work area by a coordinate conversion unit;
a developing unit developing, based on control information, stereoscopic data indicating a three-dimensional shape of the working robot into a coordinate system of the working area;
a detection unit detecting an overlapping area between the point cloud data and the three-dimensional data in a coordinate system of the work area;
a generating unit generating the point cloud data by overwriting and updating only an update area excluding the overlap area based on the sequentially received measurement signals at the latest timing, as a shape map of the work area,
a calculation unit calculating an elapsed time by taking a difference between a measurement time of the shape measurement sensor associated with each of the point cloud data constituting the shape map and a current time;
A reliability evaluation unit evaluates reliability of each of the point cloud data based on the elapsed time;
and changing the position of the shape measurement sensor and changing the measurement area based on the reliability evaluation parameter. A receiving unit sequentially receives measurement signals of the structure measured by the shape measurement sensor in the work area;
a step of converting point cloud data including a three-dimensional shape of the structure reproduced from the measurement signals from a coordinate system of the shape measurement sensor into a coordinate system of the work area by a coordinate conversion unit;
a developing unit developing, based on control information, stereoscopic data indicating a three-dimensional shape of the working robot into a coordinate system of the working area;
a detection unit detecting an overlapping area between the point cloud data and the three-dimensional data in a coordinate system of the work area;
a generating unit generating the point cloud data by overwriting and updating only an update area excluding the overlap area based on the sequentially received measurement signals at the latest timing, as a shape map of the work area,
a calculation unit calculating an elapsed time by taking a difference between a measurement time of the shape measurement sensor associated with each of the point cloud data constituting the shape map and a current time;
A reliability evaluation unit evaluates reliability of each of the point cloud data based on the elapsed time,
A method for supporting operation of a working robot, wherein the reliability is evaluated also based on a distance between the point cloud data of the structure and the three-dimensional data of the working robot. On the computer,
Sequentially receiving measurement signals of the structure measured by the shape measurement sensor in the work area;
A step of converting point cloud data including a three-dimensional shape of the structure reconstructed from the measurement signals from a coordinate system of the shape measurement sensor to a coordinate system of the work area;
a step of expanding stereoscopic data indicating a three-dimensional shape of the working robot into a coordinate system of the working area based on control information;
detecting an overlapping area between the point cloud data and the three-dimensional data in a coordinate system of the work area;
generating, as a shape map of the work area, the point cloud data in which only an update area excluding the overlap area is overwritten and updated based on the sequentially received measurement signals at the latest timing, the point cloud data comprising:
The computer includes:
calculating an elapsed time obtained by taking a difference between a measurement time of the shape measuring sensor associated with each of the point cloud data constituting the shape map and a current time;
evaluating the reliability of each of the point cloud data based on the elapsed time;
a step of executing an initialization for withdrawing the work robot from a measurement area of the shape measurement sensor based on the reliability evaluation parameter. On the computer,
Sequentially receiving measurement signals of the structure measured by the shape measurement sensor in the work area;
A step of converting point cloud data including a three-dimensional shape of the structure reconstructed from the measurement signals from a coordinate system of the shape measurement sensor to a coordinate system of the work area;
a step of converting stereoscopic data representing a three-dimensional shape of a working robot into a coordinate system of the working area based on control information;
detecting an overlapping area between the point cloud data and the three-dimensional data in a coordinate system of the work area;
generating, as a shape map of the work area, the point cloud data in which only an update area excluding the overlap area is overwritten and updated based on the sequentially received measurement signals at the latest timing, the point cloud data comprising:
The computer includes:
calculating an elapsed time obtained by taking a difference between a measurement time of the shape measuring sensor associated with each of the point cloud data constituting the shape map and a current time;
evaluating the reliability of each of the point cloud data based on the elapsed time;
A work robot operation assistance program that executes a step of changing the position of the shape measurement sensor and changing the measurement area based on the reliability evaluation parameter. On the computer,
Sequentially receiving measurement signals of the structure measured by the shape measurement sensor in the work area;
A step of converting point cloud data including a three-dimensional shape of the structure reconstructed from the measurement signals from a coordinate system of the shape measurement sensor to a coordinate system of the work area;
a step of converting stereoscopic data representing a three-dimensional shape of a working robot into a coordinate system of the working area based on control information;
detecting an overlapping area between the point cloud data and the three-dimensional data in a coordinate system of the work area;
generating, as a shape map of the work area, the point cloud data in which only an update area excluding the overlap area is overwritten and updated based on the sequentially received measurement signals at the latest timing, the point cloud data comprising:
The computer includes:
calculating an elapsed time obtained by taking a difference between a measurement time of the shape measuring sensor associated with each of the point cloud data constituting the shape map and a current time;
evaluating the reliability of each of the point cloud data based on the elapsed time;
The reliability is evaluated also based on a distance between the point cloud data of the structure and the three-dimensional data of the work robot .

Images