WO2023084695A1 - 処理装置、処理方法、およびプログラム - Google Patents

処理装置、処理方法、およびプログラム Download PDF

Info

Publication number
WO2023084695A1
WO2023084695A1 PCT/JP2021/041549 JP2021041549W WO2023084695A1 WO 2023084695 A1 WO2023084695 A1 WO 2023084695A1 JP 2021041549 W JP2021041549 W JP 2021041549W WO 2023084695 A1 WO2023084695 A1 WO 2023084695A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
unit
controlled
area
controlled object
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/041549
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
峰斗 佐藤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Priority to JP2023559309A priority Critical patent/JP7816372B2/ja
Priority to PCT/JP2021/041549 priority patent/WO2023084695A1/ja
Priority to US18/704,508 priority patent/US20240417955A1/en
Publication of WO2023084695A1 publication Critical patent/WO2023084695A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • E—FIXED CONSTRUCTIONS
    • E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
    • E02F—DREDGING; SOIL-SHIFTING
    • E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
    • E02F9/20—Drives; Control devices
    • E02F9/2025—Particular purposes of control systems not otherwise provided for
    • E02F9/2033—Limiting the movement of frames or implements, e.g. to avoid collision between implements and the cabin
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1656—Program controls characterised by programming, planning systems for manipulators
    • B25J9/1664—Program controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
    • B25J9/1666—Avoiding collision or forbidden zones
    • E—FIXED CONSTRUCTIONS
    • E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
    • E02F—DREDGING; SOIL-SHIFTING
    • E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
    • E02F9/24—Safety devices, e.g. for preventing overload
    • E—FIXED CONSTRUCTIONS
    • E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
    • E02F—DREDGING; SOIL-SHIFTING
    • E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
    • E02F9/26—Indicating devices

Definitions

  • the present invention relates to a processing device, a processing method, and a program.
  • Japanese Patent Laid-Open No. 2004-200002 discloses a technique related to control for decelerating or stopping the operation of an excavator when the excavator to be controlled enters an impenetrable area set against an obstacle.
  • An example of an object of the present invention is to provide a processing device, processing method, and program that solve the above-described problems.
  • the processing device determines whether the controlled object has entered an area other than an area in which the controlled object is allowed to enter, in an environment including at least a part of the controlled object having a movable part. and processing means for executing a predetermined process when the determination means determines that the controlled object has entered an area other than the area where the controlled object is allowed to enter.
  • the processing method is characterized in that, in an environment having a range including at least a part of a controlled object having a movable part, the and executing a predetermined process when it is determined that the controlled object has entered an area other than the area into which the controlled object is allowed to enter.
  • a recording medium for recording a program allows a computer to perform a program within an environment including at least a part of a controlled object having a movable part, other than an area where the controlled object is allowed to enter. Determining whether or not the controlled object has entered an area; and executing a predetermined process when it is determined that the controlled object has entered an area other than an area that permits entry of the controlled object. let it run.
  • FIG. 4 is a flow chart showing an example of a procedure of processing performed by the control system according to the first embodiment; It is a figure which shows an example of a structure of the control system which concerns on 2nd Embodiment.
  • 9 is a flow chart showing an example of a procedure of processing performed by a control system according to a second embodiment; It is a figure which shows an example of a structure of the control system which concerns on 3rd Embodiment.
  • FIG. 10 is a diagram showing an example of a configuration of a control system of a first application;
  • FIG. FIG. 11 is a diagram for explaining a specific example of processing of an information comparison unit according to the first application example;
  • FIG. 10 is a diagram for explaining processing of a determination processing unit according to the first application example;
  • FIG. 10 is a diagram for explaining processing of a determination processing unit according to the first application example;
  • FIG. 11 is a diagram showing an example of the configuration of a control system according to a second application;
  • FIG. 11 is a diagram for explaining processing of a determination processing unit according to a second application example;
  • FIG. 11 is a flow chart regarding processing of a determination processing unit according to a second application example;
  • 1 is a schematic block diagram showing a configuration of a computer according to at least one embodiment;
  • FIG. 1 is a diagram showing an example of the configuration of a control system 100 according to the first embodiment.
  • the control system 100 includes a mobile device 1, an observation device 2, and an obstacle detection device 3 (an example of a processing device).
  • the control system 100 is a system in which the control unit 12 (to be described later) controls the controlled unit 11 having the movable unit 11a based on the information obtained from the observation device 2 in the movable unit 1 .
  • the mobile device 1 includes a controlled portion 11 (an example of a controlled object) that is to be controlled, and a control portion 12 that controls the controlled portion 11 (an example of control means).
  • the movable device 1 is, for example, a robot having an arm, a transport vehicle, a construction machine (hereinafter referred to as a construction machine), etc., but is not limited to these.
  • construction machines having arms include power shovels, backhoes, cranes, and forklifts.
  • the controlled section 11 is a housing portion such as an arm, a bucket, and a shovel that performs work.
  • the controlled portion 11 has a movable portion 11a.
  • the movable part 11a is, for example, an actuator.
  • the control unit 12 controls the controlled unit 11 . As the control section 12 controls the controlled section 11, the movable section 11a of the controlled section 11 operates.
  • the observation device 2 observes at least the space in which the mobile device 1 operates, and outputs observed observation information (an example of actual measurement values).
  • the observation device 2 acquires the movable range of the controlled unit 11 as image (RGB) or three-dimensional image (RGB-D) data, for example, a monocular, compound eye, monochrome, RGB, depth camera, ToF (Time of Flight)
  • RGB image
  • RGB-D three-dimensional image
  • it is a device that optically measures the distance to the target, such as a lidar (Light Detection And Ranging; LiDAR), or a device that measures with radio waves, such as a radar ( Radio Detection and Ranging (Radar), etc., and the specific configurations of these devices are not limited in this embodiment.
  • the observation device 2 may be a single device such as these, or may be a combination of multiple devices.
  • the observation device 2 also acquires observation data for an observation area including at least the movable range of the controlled section 11 . Therefore, the observation data includes at least part of the housing of the controlled unit 11 .
  • the observation data includes information about the surrounding environment such as obstacles and information about the controlled section 11 of the mobile device 1 to be controlled.
  • the observation area of the observation device 2 is determined by conditions such as the installation position and installation direction (angle) when the observation device 2 is installed, and the performance and parameters specific to the observation device.
  • the installation of the observation device 2 is appropriately determined based on the type and performance of the observation device 2, the specifications (for example, type, size, movable range, etc.) of the movable device 1 to be observed, the work content, and the surrounding environment. and is not limited by the present invention.
  • the types are different measurement methods, and examples of types include cameras, video cameras, lidars, and radars. Examples of performance include field of view (FOV), maximum measurable distance, and resolution.
  • FOV field of view
  • the observation device 2 may be mounted on the mobile device 1 .
  • the obstacle detection device 3 includes a position and orientation data acquisition unit 31 that acquires position and orientation data of the controlled unit 11 , an observation data acquisition unit 32 that acquires data of the observation device 2 , and information obtained by the observation device 2 .
  • An information exclusion unit 33 (an example of exclusion means) that excludes information about the controlled unit 11 and outputs information after the exclusion, information output from the information exclusion unit 33, and information described later in the virtual environment 4
  • the determination processing unit 34 (an example of determination means, an example of processing means) that performs determination processing regarding detection of an obstacle for the control system 100 and at least the controlled unit 11 are calculated. It comprises a virtual environment 4 simulated above.
  • the position and orientation data acquisition section 31 acquires the position and orientation data of the controlled section 11 .
  • the mobile device 1 is an arm of a robot, that is, a so-called articulated robot arm
  • angle data of each joint of the arm is acquired as the position/orientation data.
  • This angle data can typically be acquired as an electrical signal by a sensor (for example, a rotary encoder) attached to an actuator that drives each joint.
  • a sensor for example, a rotary encoder
  • the position and orientation data is acquired by sensors attached to each movable portion 11a of the controlled portion 11 or the housing.
  • sensors include an external sensor such as an inclination sensor, a gyro sensor, an acceleration sensor, an encoder, and a hydraulic sensor.
  • the installation positions and the number of sensors can be appropriately designed for each operation of the mobile device 1 to be detected.
  • the position/attitude data follows the temporal change of the movable portion 11 a of the controlled portion 11 . That is, the electrical signal information acquired by the position/orientation data acquisition unit 31 is information acquired corresponding to the operation of the controlled unit 11 within a certain range of error and delay time.
  • the temporal frequency (sampling rate) and spatial resolution (accuracy) of the signal are not particularly limited, and can be appropriately determined according to the size and characteristics of the movable device 1, work content, and the like.
  • the observation data acquisition unit 32 preferably acquires the observation data output by the observation device 2.
  • the observation data acquisition unit 32 may acquire information as observation data from other means, such as a sensor mounted on the mobile device 1 .
  • the virtual environment 4 is an environment that simulates at least the controlled unit 11 in terms of calculation.
  • the virtual environment 4 refers to a so-called digital twin, which is an environment in which the dynamics of the controlled unit 11 and the surrounding real environment are reproduced by executing a simulation using a simulator, a mathematical model, or the like.
  • the virtual environment 4 is not restricted to digital twins.
  • the first point is the shape of the controlled portion 11 .
  • the virtual environment 4 has a model that reproduces the external shape, that is, the size and three-dimensional shape of the controlled part 11 within the same or a certain range of error or scale as the real controlled part 11 .
  • the model of the controlled part 11 is based on, for example, the blueprint or CAD data of the controlled part 11, the image data of the controlled part 11, or the like, by polygons, or by a set of polygons (that is, a mesh). can be built.
  • the model of the controlled unit 11 is represented by polygons
  • approximation is performed according to the shape, size, density, and the like of the polygons.
  • the degree of approximation can be appropriately determined depending on the size of the controlled section 11 to be controlled.
  • the model of the controlled portion 11 is represented by polygons, the model represents a three-dimensional shape, so it is not necessary to reproduce the material, texture, pattern, etc. of the surface.
  • the method of constructing the model of the controlled unit 11 is not limited to the method described above.
  • the second point is the movement of the controlled portion 11, that is, the movement (dynamics) of each movable portion 11a.
  • the controlled part 11 includes a movable part 11a (actuator) controlled by at least one or more control parts 12, and the controlled part 11 in the virtual environment 4 described in the first point of the viewpoint of simulating the controlled part 11.
  • the movable portion 11a is the same as the real controlled portion 11 or is reproduced within a certain range of error. It should be noted that reproduction of movement means that the same position and angle as the movable part 11a of the real thing can be displaced.
  • the configuration method of the portion 11a is not limited.
  • the virtual environment 4 may include virtual observation means corresponding to the real observation device 2 and an observation area to be observed. Virtual observation means will be described later.
  • a controlled device 43 (an example of a model of the controlled section 11) that simulates the real controlled section 11 in the real environment and the controlled device 43 are set (the information generating section 42 is set to the observation device 2).
  • the observation means is further set to the same position and orientation as the observation device 2)
  • Environment setting unit 41 and information about the simulated controlled unit 11 is output.
  • an information generation unit 42 for generating the information In the virtual environment 4, the environment setting unit 41 arranges a model simulating the real controlled part 11 (including the movable part 11a) in the controlled device 43 (that is, sets the position and posture), The position and orientation of the virtual observation device 2 simulating the observation device 2 are set.
  • This model of the controlled part 11 and the virtual observation device 2 are related to the relative positions and orientations of the real controlled part 11 and the real observation device 2 in the three-dimensional area handled in the virtual environment 4 . Arranged to be identical or to be reproduced within a certain margin of error or scale. That is, when either the model of the controlled unit 11 or the virtual observation device 2 is used as a reference for the position and orientation, the difference in distance and angle to the other is the same as the real thing, or within a certain error range or scale. It has become. It should be noted that the scale here corresponds to the scale of the model of the controlled section 11 described above.
  • the virtual environment 4 handles an area including the movable range of the real controlled part 11
  • the model simulating the real controlled part 11 and the virtual observation device 2 are positioned at the same position as the real controlled part 11 . and posture.
  • Such setting of the relationship between the model of the controlled unit 11 and the position and orientation of the observation device 2 is generally called calibration. That is, the model of the controlled unit 11 and the virtual observation device 2 are set to a calibrated state. It should be noted that the setting of structures other than the model of the controlled unit 11 and the boundaries of the space such as the ground are not essential.
  • the movable part 11 a of the model of the controlled part 11 is set based on the information about the real controlled part 11 acquired by the position/orientation data acquisition part 31 .
  • the three-dimensional shape of the model of the controlled portion 11 is the same as that of the movable portion 11a of the actual controlled portion 11, or is set within a certain range of error, so that the three-dimensional shape of the model of the controlled portion 11 is similar to that of the actual object. It can be represented similarly to the shape of the controlled portion 11 .
  • the temporal displacement of the movable portion 11 a of the real controlled portion 11 is set by the environment setting portion 41 based on the information acquired by the position/orientation data acquisition portion 31 . Therefore, the model of the controlled part 11 in the virtual environment 4 can be moved in the same manner as the real controlled part 11 within a certain range of error or delay time.
  • the model of the controlled part 11 in the virtual environment 4 is synchronized (synchronized) with the real controlled part 11 .
  • the information generating unit 42 generates at least information about the model in the virtual environment 4 in which the real controlled unit 11 is simulated.
  • the model of the controlled portion 11 reproduces the shape and movement of the real controlled portion 11, the shape and movement of the controlled portion 11 can be obtained by executing a simulation using the model.
  • Information corresponding to motion is generated.
  • the generated information is a set of three-dimensional positions occupied by the three-dimensional shape of the model of the controlled unit 11 at a certain time in the three-dimensional space handled in the virtual environment 4, or Time-series values of three-dimensional positions according to the temporal displacement of the model of part 11 (for example, gray grids in part (b) of FIG. 7 to be described later).
  • the generated information is a set of positional information of each polygon representing the three-dimensional shape of the model of the controlled section 11 .
  • the spatial resolution of this positional information depends on the size of the polygon representing the model of the controlled unit 11 and the like.
  • the resolution can be changed by, for example, interpolating (up-sampling) or thinning (down-sampling) the positional information of polygons.
  • the resolution is changed by increasing the spatial resolution, that is, by expressing with fine polygons or performing upsampling. can be executed by lowering the resolution, that is, by down-sampling the polygon position information.
  • virtual observation means corresponding to the observation device 2 can also be used.
  • This observation means is similar to the real observation device 2 by installing the virtual observation device 2 in a virtual three-dimensional space corresponding to the position and orientation of the real observation device 2 set by the environment setting unit 41 .
  • Observation data that is, images and three-dimensional images can be obtained virtually.
  • the observation means has a function of simulating the observation device 2 and simulating and outputting observation information observed from the installed position and orientation of the observation device 2 .
  • the observation range of the observation device 2 includes at least the movable range of the controlled section 11, and the observation information output by this observation means is information indicating the same range.
  • the observation means outputs information (an example of an estimated value) obtained by observing a model simulating the controlled section 11 . That is, information on the shape indicated by the model of the controlled portion 11, information on the position of the model of the controlled portion 11 in the three-dimensional space, and time-series information corresponding to the movement indicated by the model of the controlled portion 11 are obtained.
  • This virtual observation means is preferably an observation device having the same performance as the observation device 2, ie, an imaging range and resolution. It should be noted that the virtual observation means may be appropriately adjusted according to the processing power of the computer that processes the information generating section 42 and the like. Further, the information generating section 42 may generate information other than the part corresponding to the model of the controlled section 11 .
  • the information generating unit 42 generates information on structures other than the model of the controlled unit 11 reproduced in the virtual environment 4, information on the boundary of space such as the ground, or information dependent on the work performed by the mobile device 1. may be generated.
  • the movable device 1 is a robot arm or a construction machine
  • the information depending on the work is the target object, the work area, and the like on which the controlled section 11 works.
  • the information exclusion unit 33 performs processing to exclude information from the information acquired by the observation data acquisition unit 32 based on the information generated by the information generation unit 42 of the virtual environment 4. Specifically, the three-dimensional shape generated by the information generating unit 42 of the virtual environment 4 is excluded from the information obtained by the observation data obtaining unit 32, that is, the observation information corresponding to the observation data output by the observation device 2. Do (filtering, masking) processing. As described above, the observation information includes at least part of the controlled portion 11, and the information generated by the information generating portion 42 includes at least the shape information of the model in which the controlled portion 11 is simulated. .
  • the information exclusion unit 33 can output information in which the controlled unit 11 is excluded from the observation information, in other words, observation information that does not include the controlled unit 11 by the exclusion processing of these two pieces of information.
  • Observation information that does not include the controlled part 11 is, for example, other structures. is defined as information (obstacle candidate information) that includes an area that becomes an obstacle to
  • region is an area
  • the obstacle candidate information does not include areas that permit approaching or entering.
  • the obstacle candidate information output by the information exclusion unit 33 based on the time-series data of the observation data acquisition unit 32 and the information generation unit 42 corresponding to the movement of the controlled unit 11 is also It becomes time series data. That is, in synchronization with the movement of the controlled portion 11, the area corresponding to the controlled portion 11 is excluded.
  • Methods for excluding the region corresponding to the controlled portion 11 include a method of comparing the three-dimensional information of each of the observation data acquisition portion 32 and the information generation portion 42, and a method of comparing three-dimensional information of each of the observation data acquisition portion 32 and the information generation portion 42.
  • Information can also be represented by regular grids (voxels) occupied in a three-dimensional space, and a process of detecting overlap between the grids, such as logical operations such as XOR (Exclusive OR), can also be used.
  • a process of detecting overlap between the grids such as logical operations such as XOR (Exclusive OR)
  • the method of excluding the area corresponding to the controlled portion 11 is not limited to these methods.
  • the information exclusion portion 33 can perform processing for excluding an area corresponding to the controlled portion 11 with a sufficiently small delay, and the obstacle candidate information includes the controlled portion The region corresponding to 11 is not included.
  • the information exclusion unit 33 cannot properly exclude the area corresponding to the controlled unit 11, and the obstacle candidate information includes a part of the area corresponding to the controlled unit 11.
  • the information excluding section 33 may exclude an area slightly larger than the three-dimensional information output by the information generating section 42, for example. It can be adjusted so that the region is not included.
  • This adjustment may be performed by multiplying the three-dimensional area output by the information generation unit 42 by a coefficient exceeding 1.
  • This coefficient is used as a parameter such as the operating speed of the controlled unit 11 and the processing capacity of the information exclusion unit 33. can be adjusted as appropriate. Note that the above adjustment is only an example and is not limited to this.
  • the determination processing unit 34 receives the obstacle candidate information output from the information exclusion unit 33 and the information output from the information generation unit 42 in the virtual environment 4, and determines the obstacle detection. Perform judgment processing.
  • the obstacle candidate information output by the information exclusion unit 33 is information including an area that the controlled unit 11 should not approach or enter, that is, an obstacle area.
  • the shape information output from the information generation unit 42 is information within the virtual environment 4 corresponding to the shape and movement of the controlled unit 11 .
  • the obstacle candidate information output by the information exclusion unit 33 is based on observation information obtained by the observation device 2 in the real environment.
  • the shape information output from the information generation unit 42 is information in the virtual environment.
  • the determination processing unit 34 compares the obstacle candidate information output by the information excluding unit 33 with the information dynamically representing the controlled unit 11, which is the shape information output from the information generating unit 42, to It can be determined whether the control unit 11 is approaching or entering (in contact with) an area of an obstacle. For this determination, for example, the distance between the three-dimensional position indicated by the obstacle candidate information and the set of positions indicated by the three-dimensional position set information output by the information generation unit 42 is calculated, and a set threshold value is calculated. It can be realized by evaluating whether or not the limit is exceeded.
  • Collective information which is a collection of three-dimensional position information, can be expressed as a set of points representing three-dimensional coordinates, for example, point cloud data, and the distance between each set can be expressed as, for example, It can be calculated as the Euclidean distance between centroids, the Euclidean distance of the closest points (nearest neighbor points), and the like.
  • a method of finding the nearest neighbor point is, for example, a method of using an algorithm such as nearest neighbor search or k-nearest neighbor search.
  • the method of finding the nearest point is not limited to methods using nearest neighbor search or k-nearest neighbor search algorithms. Also, this determination can be realized by the processing opposite to that of the information exclusion unit 33 described above.
  • each set information of the obstacle candidate information and the three-dimensional position output by the information generation unit 42 is represented by a three-dimensional regular grid (voxel), and the grid Matching grids between or between surrounding grids mean that there are locations in three dimensions that are close together. Therefore, in the determination, for example, a process (for example, XOR operation) is performed to see the overlap between the grids with a certain predetermined resolution. It indicates that the area of the obstacle is not approached, and if overlap is detected, it indicates that the controlled section 11 is approaching the area of the obstacle.
  • a process for example, XOR operation
  • the resolution of this overlap detection that is, the size of the lattice (voxel) depends on the point cloud density (that is, the size of the mesh) of each three-dimensional information, and also depends on the processing capability of the determination processing unit 34. It can be set as appropriate. Preferably, by setting a wide grid size, the approach is determined early, that is, when the distance between the obstacle region and the controlled section 11 approaches the set grid size. On the other hand, by setting a narrow grid size, the spatial resolution for determining the distance between the obstacle region and the controlled part 11, that is, the spatial accuracy is improved. Even if it is, it is possible to determine with high accuracy. Note that these determination methods are examples, and any method may be used as long as it can determine whether or not the controlled portion 11 is close to an obstacle area.
  • the result of the determination by the determination processing unit 34 may be announced on a display or the like (not shown).
  • the control command output by the control unit 12 of the mobile device 1 to the controlled unit 11 may be changed based on the determination result. For example, by changing the control command output to the controlled unit 11, the control unit 12 may limit the operating range of the controlled unit 11, limit the operating speed of the controlled unit 11, or limit the operating speed of the controlled unit 11. 11 may be stopped. Any method may be used to change these control commands.
  • the resolution in space is the threshold for determining the distance between a set of points representing three-dimensional coordinates indicated by three-dimensional information, or the resolution when expressed in voxels (lattice size, that is, the mesh size). magnitude), but this value need not be set to a single value.
  • a plurality of different values may be set as thresholds and grid sizes.
  • the determination processing unit 34 can make determinations in parallel. As described above, the time to determination and spatial accuracy are traded off depending on the spatial resolution setting.
  • a large value and a small value are set as the distance threshold value, and in the case of a large value, the judgment processing unit 34 judges quickly when the distance is long, so an instruction to decelerate the controlled unit 11 is issued.
  • the determination processing unit 34 makes a highly accurate determination when the distance becomes close, and therefore, when an instruction to stop the controlled unit 11 is issued, it is possible to determine whether to decelerate or stop.
  • the determination processing unit 34 performs determinations using a wide grid size (coarse resolution) and a narrow grid size (fine resolution) in parallel.
  • An instruction to decelerate may be output when the size is determined, and an instruction to stop may be output when the determination processing unit 34 determines a narrow grid size.
  • the judgment processing unit 34 makes a judgment using a large distance threshold value or a wide grid size, and after the controlled unit 11 is decelerated, the judgment processing unit 34 judges that it is not close to the obstacle area. In this case, the control by the control unit 12 may be returned to the original control.
  • the control unit 12 can efficiently operate the movable device 1 without excessively stopping the controlled unit 11 .
  • the above determinations by the determination processing unit 34 are examples, and the present invention is not limited to these.
  • the determination processing unit 34 may set multi-step (multi-value) resolution and perform multi-step determination.
  • FIG. 2 is a flowchart showing an example of the procedure of processing performed by the control system 100 according to the first embodiment. Next, the processing of the control system 100 will be described with reference to FIG.
  • the position/posture data acquisition unit 31 of the obstacle detection device 3 acquires position/posture data from the mobile device 1, and the observation data acquisition unit 32 acquires observation data from the observation device 2 (step S101).
  • the environment setting unit 41 of the obstacle detection device 3 sets the virtual environment in the virtual environment 4 based on the configuration of the real environment and the acquired position and orientation data (step S102). Specifically, the environment setting unit 41 sets the relationship between the position and orientation of the model of the controlled unit 11 simulated in the virtual environment and the real observation device 2, that is, calibration processing, Perform processing to reflect the acquired position and orientation data in the model.
  • the information generation unit 42 outputs shape information based on the state of the controlled unit 11 in the real environment for the simulated model (step S103). Specifically, the information generating unit 42 generates, for example, a set of three-dimensional positions occupied by the three-dimensional shape of the model in the virtual environment 4 synchronized with the controlled unit 11 in the real environment, or It outputs time-series values of three-dimensional positions according to the temporal displacement of the model in the virtual environment 4 synchronized with the unit 11 .
  • the information exclusion unit 33 excludes regions that are not determined as obstacles from the observation data of the real environment, and outputs the excluded obstacle candidate information (step S104).
  • the area not determined as an obstacle is, for example, an area corresponding to the controlled unit 11 or an area scheduled to be approached or entered during work by the movable device 1, and the user registers an area that is not determined as an obstacle.
  • the information exclusion unit 33 may register it as information in advance.
  • the determination processing unit 34 determines from the obstacle candidate information and the shape information the distance between the two pieces of information (that is, the distance between the obstacle area and the controlled unit 11).
  • a related value preferably proportional to the distance
  • the specified determination value is output (step S105). Examples of the determination value include a value proportional to the distance between the obstacle area and the controlled section 11, and "overlap" corresponding to the distance between the obstacle area and the controlled section 11 (for example, FIG. 8 part indicated by a dot-patterned lattice where the obstacle area and the controlled part overlap when the distance to the obstacle is less than the threshold value).
  • the determination value indicates the distance between the three-dimensional area representing the obstacle candidate information and the three-dimensional area representing the dynamic shape of the controlled section 11 .
  • a threshold value is appropriately set according to the type of determination value output by the determination processing unit 34 .
  • the determination processing unit 34 determines whether or not the determination value is equal to or greater than the threshold (step S106). If the determination value is equal to or greater than the threshold value (YES in step S106), the determination processing unit 34 determines that the distance between the controlled unit 11 and the obstacle area is long, and therefore determines that the mobile device 1 continues to operate. , the flow returns to the start (the process starting from step S101 is repeated thereafter).
  • step S106 NO determines that the determination value is less than the threshold value (step S106 NO)
  • the determination processing unit 34 determines that the controlled unit 11 is approaching or entering the obstacle area, and indicates that an obstacle has been detected.
  • An alert is output (step S107).
  • the determination processing unit 34 then outputs an instruction to the control unit 12 of the mobile device 1 .
  • the instruction to the control unit 12 by the determination processing unit 34 is an instruction to restrict the operation range of the controlled unit 11, limit the operation speed, or stop the operation. It may be set so as to provide different instructions.
  • step S107 After the process when an obstacle is detected (the process of step S107), the flow basically returns to the start (the process starting from step S101 is repeated). However, if the operation returns to the start even once, or if the controlled unit 11 is stopped by an instruction to the control unit 12, recovery work for moving the controlled unit 11 away from the failure area is performed as appropriate.
  • the controlled section 11 of the mobile device 1 can be operated safely without approaching or entering the area of the obstacle observed by the observation device 2 .
  • the instructions to the control unit 12 described in the processing when an obstacle is detected step S107
  • a decrease in work efficiency due to the stop is prevented, You can work safely and efficiently.
  • control system 100 (advantage) The control system 100 according to the first embodiment has been described above. Advantages of control system 100 over control systems to be compared will now be described.
  • the first method is to preset an area to be determined as an obstacle based on the observation results and the movable range of the movable device 1 . Since this method is set in advance, errors in judgment and oversight are less likely to occur. However, since it is necessary to set the area in advance, it is difficult to set the area according to the minimum required area or the dynamically changing area for changing environments and obstacles. Therefore, in this method, an area that is wider than necessary (that is, with a margin) is set in advance, and the determination processing unit 34 may make excessive determinations.
  • the second is a method of detecting an obstacle and estimating its position based on observation information.
  • object detection methods using deep learning can be applied, but there are cases where it is necessary to learn the object detection target in advance, and there are cases where there is no guarantee that unknown obstacles can be reliably detected. . In other words, erroneous detection or oversight (missed detection) may occur.
  • erroneous detection or oversight missed detection
  • the control system 100 according to the first embodiment does not perform processing for preliminarily setting areas and objects related to obstacles, and preliminarily detecting obstacles and objects. All areas excluding areas and objects determined to be non-existent, that is, areas into which entry is permitted depending on the controlled unit 11 and work are set as obstacle candidate information. That is, in the control system 100, an oversight that an obstacle is not detected does not occur. Then, the control system 100 compares the obstacle candidate information with the virtual environment information in which the actual shape and motion of the controlled unit 11 are simulated.
  • the obstacle candidate information and the information of the controlled part 11 are generated as different information and compared, so that the area of the controlled part 11 can be extracted from the observed information, or the same observation information can be obtained. , the process of estimating the distance between the obstacle and the controlled unit 11 does not occur. That is, in the control system 100, no processing errors or estimation errors occur. Furthermore, in the control system 100, even if part of the observation information about the controlled part 11 is lost, that is, even if part of the controlled part 11 is shielded, the information generated in the virtual environment is controlled. Since it is based on the shape model of the part 11, the control system 100 is not affected by information loss or shielding, and has high robustness. In this way, the control system 100 is characterized by not using an object detection technique and making decisions based on observed information and information based on a model of the virtual environment. It is possible to precisely control the controlled unit 11 with good work efficiency while detecting obstacles with high accuracy and certainty.
  • FIG. 3 is a diagram showing an example of the configuration of a control system 200 according to the second embodiment.
  • the obstacle detection device 3 further includes an information comparison section 35 in addition to the configuration of the control system 100 according to the first embodiment shown in FIG. Since other configurations are the same as those of the control system 100 according to the first embodiment, description thereof will be omitted below.
  • the information comparison unit 35 stores the observation information obtained by the observation data acquisition unit 32, in which the controlled unit 11 is included in the observation range, and the controlled object generated by the information generation unit 42 of the virtual environment 4. Shape information about a model simulating the part 11 is input.
  • three-dimensional information reflecting the shape of the controlled part 11 included in the observation information and the shape of the model synchronized with the controlled part 11 generated by the information generating unit 42 of the virtual environment 4 are reflected.
  • Three-dimensional information agrees within a certain margin of error.
  • the first point is based on the definition of the model that simulates the controlled unit 11 in the virtual environment 4 . Since this model simulates the shape of the real controlled part 11, the three-dimensional information based on the shape, that is, the three-dimensional information of the portion occupied by the controlled part 11 represented by the model in the virtual space is , is equal to three-dimensional information obtained by observing the controlled portion 11 with the observation device 2 in the real space.
  • the second point is that the coordinate system of the observation device 2 in the real space and the coordinate system for generating the shape information indicated by the model match.
  • the environment setting unit 41 makes the relationship between the positions and orientations of the controlled unit 11 and the observation device 2 match the relationship between the model in the virtual environment 4 and the reference point when generating the shape information included in the model. This is because it is set (calibrated) as
  • the third point is that the dynamic displacement of the controlled unit 11 is acquired via the position/orientation data acquisition unit 31 and reflected in the model in the virtual environment 4 by the environment setting unit 41 . That is, it can be considered that the operations of the controlled section 11 and the model are synchronized within a certain specified delay time range. Therefore, even when the controlled section 11 moves, both pieces of three-dimensional information input to the information comparing section 35 match within a prescribed delay time range.
  • the operating state is not ideal. Specifically, the situation is as follows. First, in response to the first reason, there is a mismatch in shape between the actual controlled unit 11 and the model in the virtual environment 4 . This state can occur, for example, when a mobile device different from the assumed mobile device 1 is connected, or when there is an error in the processing of the environment setting section 41 of the virtual environment 4 . Next, there is a deviation in the coordinate system corresponding to the second reason.
  • This state can occur when calibration is inappropriate, or when the position and orientation of the observation device 2 change after calibration.
  • the position and orientation data of the controlled unit 11 cannot be properly acquired.
  • a malfunction of a sensor that acquires the position and orientation of the controlled unit 11 a malfunction of the route connecting the movable device 1 and the obstacle detection device 3, a malfunction of the processing in the position and orientation data acquisition unit 31, and the like occur. can occur if
  • Determination of these defects can be realized by evaluating the distance between information input to the information comparing unit 35, that is, the distance between set information of points representing three-dimensional coordinates indicated by the three-dimensional information.
  • a method for calculating the distance for example, a method equivalent to the processing in the determination processing unit 34 described in the first embodiment can be applied. Specifically, if the distance between the two pieces of input information is less than a threshold value, the information comparison unit 35 determines that the two pieces of information match, that is, that there is no problem. On the other hand, if the distance is equal to or greater than the threshold, the information comparing section 35 determines that the two pieces of information do not match, that is, that there is a problem.
  • the determination processing unit 34 When determining that there is a problem, the determination processing unit 34 sends an alert or an instruction to the control unit 12 in the same manner as when an obstacle is detected.
  • the threshold for determination can be appropriately set according to the size, operating speed, amount of information (possible), and the like of the controlled unit 11 .
  • FIG. 4 is a flowchart showing an example of the procedure of processing performed by the control system 200 according to the second embodiment. Processing of the control system 200 will be described with reference to FIG. Among the processes shown in FIG. 4, the same step numbers are assigned to the same processes as those of the control system 100 according to the first embodiment, and descriptions thereof are omitted.
  • the information comparison unit 35 collects the observation data of the real environment acquired by the observation data acquisition unit 32 and the virtual environment input the shape information generated by the information generation unit 42 for the model, and output a comparison value related to the distance between both pieces of information (that is, the distance between the obstacle area and the controlled unit 11) (Step S201).
  • this comparison value is compared with a threshold value, and if it is less than the threshold value (step S202 YES), the information comparison unit 35 determines that there is no problem with the control system 200, and thereafter the first embodiment is performed.
  • a similar flow (steps S104 to S106) is operated.
  • the information comparison unit 35 determines that there is a problem with the control system 200, and outputs an alert for detecting a problem or an obstacle (step S203).
  • This flow is similar to the control system 100 according to the first embodiment when an obstacle is detected (step S106 NO), but in the control system 200 according to the second embodiment, An alert is output even if there is a problem.
  • failure and obstacle detection are determined differently (steps S201 and S106), they may be output as identifiable alerts.
  • an instruction may be output to the control unit 12 in addition to the alert.
  • control system 200 has been described above.
  • control system 200 according to the second embodiment in addition to the control system 100 according to the first embodiment, by further including an information comparison unit 35, correspondence between the movable device 1 and the virtual environment 4 as described above , the position and orientation of the observation device 2 , and calibration, and the signal path connecting the movable device 1 and the sensor that acquires the position and orientation information of the controlled unit 11 . That is, before determining whether or not the controlled unit 11 is approaching or entering an obstacle area, whether the movable device 1, the observation device 2, and the obstacle detection device 3 are in a state in which the determination can be performed normally. It is possible to detect whether or not there is a problem with the control system 200 . This makes it possible to detect obstacles separately from other system failures. Therefore, the control system 200 can more reliably detect obstacles by taking recovery measures when a malfunction state is detected.
  • FIG. 5 is a diagram showing an example of the configuration of a control system 300 according to the third embodiment.
  • the obstacle detection device 3 further includes a control plan data acquisition unit 36 in addition to the control system 100 according to the first embodiment. Since other configurations are the same as those of the control system 100 according to the first embodiment, description thereof will be omitted below.
  • a configuration combined with the second embodiment, that is, a configuration further including an information comparison unit 35 is also possible.
  • the control plan data acquisition unit 36 acquires control plan information for controlling the controlled unit 11 of the movable device 1 .
  • the control plan data acquisition section 36 acquires the control signal generated by the control section 12 .
  • any generation method and acquisition route can be used. It may be a route.
  • the information of the control plan is, for example, information of a target position when a specific part of the controlled part 11 moves from the current position to the target position, or information of the movable part 11a (actuator) constituting the controlled part 11 at that time.
  • the control plan data acquisition unit 36 acquires the position/orientation information for which future control is planned (scheduled). get.
  • the frequency of acquiring this future control plan information may be acquired for each specific operation or time when the target position changes, or periodically. That is, the information on the future control plan is time-series information, like the current position and orientation information acquired by the position and orientation data acquisition unit 31 .
  • the future control plan information acquired by the control plan data acquisition unit 36 is input to the environment setting unit 41 of the virtual environment 4 .
  • the environment setting unit 41 sets the position and orientation of the model simulating the controlled unit 11 based on the current position and orientation information acquired by the position and orientation data acquisition unit 31. do.
  • the model is in a state synchronized with the current state of the real controlled unit 11 .
  • the third embodiment does not change this point, but differs from the first and second embodiments in that it has a model that simulates another controlled unit 11 .
  • the position and posture of this model are set based on the control plan information acquired by the control plan data acquisition unit 36 . That is, the model will be in sync with the states given in the control plan.
  • the third embodiment is characterized in that different states of the controlled unit 11, that is, the current state and the state based on the control plan, are reproduced in the virtual environment 4.
  • FIG. an example of the present and one type of control plan, that is, two states is shown, but the number of states to be reproduced is not limited to this. That is, a plurality of different states may be reproduced based on control plan information at a plurality of different timings.
  • the information generation unit 42 in the virtual environment 4 of the third embodiment processes the multiple different state models described above. That is, the information generation unit 42 generates position information occupied by the three-dimensional shape of the model corresponding to the current position and orientation of the controlled unit 11, and the position and orientation of the controlled unit 11 planned based on the control plan. Position information occupied by the three-dimensional shape of the model is generated. Note that the same method as in the first embodiment can be applied to this generation by the information generation unit 42, and the number of pieces of information to be generated corresponds to the number of different models. That is, as described above, when a plurality of different models are reproduced based on a plurality of different control plans, the information generator 42 generates information matching the number of different models.
  • the inputs and outputs of the observation device 2, the observation data acquisition unit 32, and the information exclusion unit 33 are the same as in the first embodiment, so descriptions thereof will be omitted.
  • the processing performed by the control system 300 according to the third embodiment is basically the same as the flowchart of the control system 100 according to the first embodiment shown in FIG. Moreover, as described above, the control system 300 according to the third embodiment can also be applied to the control system 200 according to the second embodiment. Therefore, the processing performed by the control system 300 according to the third embodiment can be the same processing as the flowchart of the control system 200 according to the second embodiment shown in FIG.
  • the control system 300 according to the third embodiment has been described above. Similar to the determination processing unit 34 of the first embodiment, the determination processing unit 34 of the third embodiment includes obstacle candidate information output by the information exclusion unit 33 and a plurality of models output by the information generation unit 42. 3D shape information based on is input. A method of performing determination processing based on these pieces of information will be described.
  • the control system 300 of the third embodiment is similar to the control system 300 of the first embodiment in that the obstacle candidate information and the shape information output by the information generation unit 42 output a judgment value related to the distance between the two pieces of information. is similar to the control system 100 by, and similar methods are applicable.
  • the control system 300 processes each piece of shape information. That is, the control plan data acquisition unit 36 in the control system 300 performs determination processing by inputting obstacle candidate information and shape information generated from a model corresponding to the current state of the controlled unit 11, and obstacle candidate information and a process of inputting and determining shape information generated from a model corresponding to the state of the controlled part 11 based on the control plan. Preferably, even when there is a plurality of pieces of shape information, the control plan data acquisition unit 36 can perform the above processes in parallel.
  • the control plan data acquisition unit 36 can output determinations for each piece of shape information, and can instruct the control unit 12 to take different actions, that is, to instruct the movable device 1 .
  • the control plan data acquisition unit 36 outputs an instruction to decelerate the controlled unit 11 based on the determination result based on the control plan, and an instruction to stop the controlled unit 11 based on the determination result based on the current state of the controlled unit 11. can be output.
  • the control plan data acquisition unit 36 determines not only the current state but also the future planned state, so that the control system 300 can take early action before it actually starts moving. It is possible.
  • the control unit 12 may control may not be in time.
  • the control system 300 by applying the control system 300 according to the third embodiment, even if the movable device 1 with a high operating speed or a large delay, the obstacle is determined and the control unit 12 detects the obstacle. It becomes possible to control the control unit 11 .
  • a first application example is an example in which the mobile device 1 in the first or second embodiment is a robot having an arm, a so-called articulated robot arm.
  • FIG. 6 is a diagram showing an example of the configuration of the control system 400 of the first application.
  • the mobile device 1 has a robot arm 11
  • the observation device 2 is a device capable of acquiring three-dimensional information such as a depth camera or LiDAR
  • the obstacle detection device 3 is the first to third It shows the configuration of a control system 400 in the case of any one obstacle detection device 3 in the embodiment.
  • the mobile device 1 and the observation device 2 are each connected to the same obstacle detection device 3, but the number and configuration of the mobile device 1 and the observation device 2 to be connected are different from this. Not as long.
  • a plurality of movable devices 1 and one observation device 2 may be connected to the obstacle detection device 3 .
  • the mobile device 1 includes at least a controlled unit 11 and a control unit 12, like the mobile device 1 of the first or second embodiment.
  • the robot arm 11 is the controlled unit 11 and the controller 12 is the control unit 12 that controls the robot arm 11 .
  • the controlled unit 11 may be mounted on a moving device such as a movable unmanned (autonomous) guided vehicle (AGV: Automatic Guided Vehicle), and the configuration related to the hardware of the movable device 1 is the first application example It is not limited to the configuration described.
  • AGV Automatic Guided Vehicle
  • the robot arm 11 includes a movable portion 11a, which may approach or enter surrounding obstacles or obstructed areas.
  • the control unit 12 may be included in the mobile device 1 or may exist in another location connected by a network, and the configuration of the control unit 12 and the generation of control signals may be controlled as desired.
  • the configuration and control signal of the control unit 12 may be of any type as long as they can generate desired control signals.
  • the observation device 2 is a device capable of acquiring three-dimensional information, such as a depth camera or LiDAR, like the observation device 2 of the first or second embodiment.
  • the position where the observation device 2 is installed is not particularly limited, but at least a part of the housing of the robot arm 11 is included.
  • FIG. 6 shows an example of an observation area 50 observed (captured) by the observation device 2 .
  • the observation device 2 may be mounted on the robot arm 11, and when the robot arm 11 is mounted on a mobile device such as an autonomous carrier, the observation device 2 may be mounted on the mobile device. It's okay to be there.
  • FIG. 6 shows an example of an observation area 50 observed by the observation device 2 .
  • the observation area 50 includes at least part of the robot arm 11 .
  • the object grasped in this task is assumed to be the object 51, and the object 51 is assumed to be included in the observation area 50.
  • FIG. 6 illustrates a case where there are two target objects 51, the present invention is not limited to this.
  • the robot arm 11 in order to perform the task of gripping the target object 51 by the robot arm 11 , the robot arm 11 needs to approach the target object 51 and finally come into contact with the target object 51 .
  • the robot arm 11 has an end effector such as a robot hand, and the robot arm 11 performs the gripping task by bringing the end effector and the object 51 into contact.
  • the robot arm 11 contacts the object 51, but the object 51 is not an obstacle, i.e. it needs to be allowed to approach or touch. Therefore, the area that the robot arm 11 allows contact with is shown in FIG. 6 as a target area 52 .
  • the first application example is an example of a task in which two objects 51 are grasped, so the target area 52 is defined as an area including the two objects 51, but the setting method of this target area 52 is not limited.
  • the target area 52 may be set for each object according to the outer peripheral surface of the object, may be set to include a specified margin on the outer peripheral surface of the object, or may be set to include a plurality of objects.
  • can be Also shown in FIG. 6 is an obstacle or obstructed area 53 that the robot arm 11 is not allowed to approach or enter.
  • the obstruction area 53 may be, for example, a structure, another object that is not to be grasped, or an area that does not have a physical shape but is not allowed to enter. may be a fault area.
  • the obstacle area 53 is defined as a range included in the observation area 50 . That is, when the faulty area continues beyond the observation area 50 , the range defined by the observational area 50 becomes the faulty area 53 .
  • the position/orientation data acquisition unit 31 of the obstacle detection device 3 acquires information on each joint (movable unit 11 a ) that configures the robot arm 11
  • the observation data acquisition unit 32 acquires three-dimensional information on the observation area 50 .
  • the virtual environment 4 constructs a model that simulates the three-dimensional shape and movement of the robot arm 11 .
  • the model is set by the environment setting unit 41 based on the information acquired by the position/orientation data acquisition unit 31, so that the real robot arm 11 and the model in the virtual environment 4 are in a synchronized state, that is, a certain regulation is established.
  • the position and posture match within the error range of .
  • the model is set by the environment setting unit 41 in the virtual environment 4, that is, calibrated.
  • FIG. 7 is a diagram for explaining a specific example of processing of the information comparison unit 35 according to the first application example.
  • a specific example of processing of the information comparison unit 35 when using the obstacle detection device 3 of the second embodiment will be described with reference to FIG.
  • the position in the three-dimensional space occupied by the robot arm 11 is shown as the real environment in part (a) of FIG. It is also, the position in the three-dimensional space occupied by the model generated by the information generation unit 42 in the virtual environment 4 is shown as the virtual environment in the part (b) of FIG.
  • FIG. 7 indicates a state in which the comparison value output by the information comparison unit 35 is less than the threshold value, that is, the comparison value between the real environment and the virtual environment matches within a prescribed error range. show.
  • the lower part of FIG. 7 shows a state in which the comparison value output by the information comparing section 35 is equal to or greater than the threshold value, that is, the control system 200 described in the second embodiment has a problem.
  • the information that is actually input to the information comparison unit 35 is three-dimensional, it is shown as two-dimensional in FIG. 7 for convenience.
  • the grid shown in FIG. 7 corresponds to the resolution of the coordinates when processed by the information comparing section 35, and is generally represented by a regular grid (voxel) in the case of three dimensions.
  • a grid containing each three-dimensional coordinate is occupied by an object, which is represented in black in the grid shown in FIG.
  • other grids that are not included in the input three-dimensional coordinates are represented in white in FIG. That is, as shown in FIG. 7, the grid occupied by the robot arm 11 is represented in black, and the other grids are represented in white.
  • the state of each lattice can be represented by a binary value (binary variable: 0 or 1) indicating whether it is occupied (black: 1) or unoccupied (white: 0). can.
  • the state of the k-th grid in the real environment shown in FIG. 7(a) is Creal,k
  • the state of the k-th grid in the virtual environment shown in FIG. 7(b) is Csim.
  • k the overlap ⁇ Ck of lattice k is given by
  • the state of lattice k is the same in the real environment shown in part (a) of FIG. 7 and in the virtual environment shown in part (b) of FIG. , the overlap ⁇ Ck of lattice k is zero.
  • the lattice k is occupied by either the real environment shown in part (a) of FIG. 7 or the virtual environment shown in part (b) of FIG. Become.
  • the comparison value output by the information comparison unit 35 is, for example, a value obtained by adding the overlap ⁇ Ck of grid k represented by Equation (1) for all grid points. , that is, can be expressed as Equation (2).
  • the number N of grid points is determined according to the volume of the target observation region 50 and the resolution (grid size) of the grid, and the computational complexity increases as N increases. increases.
  • three-dimensional information can be expressed by an octree to enable high-speed calculation.
  • the first application example is not limited to the calculation method using this octree.
  • the overlap ⁇ Ck of grid k is 0, that is, the value of equation (2) is 0.
  • the control system 400 can operate the processing (step S202) for determining the failure of the control system 200 shown in the flowchart of FIG.
  • the example of the method of determining the malfunction mentioned above is an example, and is not limited to this method.
  • the information exclusion unit 33 performs processing for excluding information from the information acquired by the observation data acquisition unit 32 based on the information generated by the information generation unit 42 . That is, the process is a process of removing the information of the portion corresponding to the robot arm 11 in the virtual environment shown in FIG. 7(b) from the information shown in the real environment shown in FIG. 7(a). Therefore, when there is no problem in the control system 400 (the comparison value is less than the threshold value in FIG.
  • each piece of information is represented by a three-dimensional octree, and equally occupied lattices are replaced with information indicating that they are not occupied. There is a way.
  • the reference position of each housing such as the center of gravity or center position, and the distance from there to the housing surface are calculated. be.
  • the information exclusion unit 33 filters the portion of the robot arm 11 from the observation data.
  • the method of excluding data by the information excluding unit 33 is not limited to these methods.
  • the process of excluding data by the information excluding unit 33 is dynamically executed according to the motion of the robot arm 11 . In other words, even when the robot arm 11 moves, the data continues to be excluded. If there is a problem in the control system 400 (the comparison value is equal to or greater than the threshold value in FIG. 7), the real environment shown in part (a) of FIG.
  • part corresponding to the robot arm 11 in the virtual environment shown in part (b) of (b) does not match. Therefore, even if the exclusion process is performed by the information exclusion unit 33, part of the portion corresponding to the robot arm 11 remains without being excluded. In other words, the portion of the robot arm 11 that has not been excluded is determined as an obstacle, and the processing of the determination processing unit 34 cannot be executed appropriately.
  • the robot arm 11 executes the task of gripping the object 51, so the object 51 must be excluded from the determination as an obstacle. Therefore, the processing by the information exclusion unit 33 is performed on the area corresponding to the robot arm 11 and the target area 52 including the target object 51 .
  • the area corresponding to the robot arm 11 is as described above.
  • the environment setting unit 41 in the virtual environment 4 sets a three-dimensional area corresponding to the target area 52, that is, a model, and three-dimensional information about the area is set. is output by the information generator 42 .
  • the position of the model corresponding to the target region 52 is determined based on the result of recognizing the position (and posture) of the target object 51 from the observation information about the target object 51 .
  • the method of recognizing the position of the target object 51 is not limited in the first application example, but it may employ autonomous object recognition using point group processing or deep learning, or a position designated by a user or another device. It is also possible to recognize the position of the object 51 by using it. In this manner, the target area 52 is identified in the coordinate system of the observation device 2, similarly to the robot arm 11. FIG. Therefore, the target area 52 can be excluded from the information obtained by the observation data acquisition unit 32, similarly to the process of excluding the portion corresponding to the robot arm 11. FIG.
  • the information excluding the robot arm 11 and the target area 52 becomes the obstacle candidate information of the first to third embodiments.
  • the first application example only the area where the object is gripped has been considered, but in an actual task using the robot arm 11, an area where the gripped object is placed may be set. Including such cases, areas to be excluded can be arbitrarily added like the target area 52 of the first application based on the task or instructions of the user, and the number of areas to be excluded is not particularly limited.
  • the addition of the area to be excluded and its exclusion method are the same as the addition of the area to be excluded of the target area 52 and its exclusion method.
  • the above processing corresponds to the operation of step S104 in the flowchart shown in FIG. 2 or 4 of the first or second embodiment.
  • FIG. 8 is a diagram for explaining the processing of the determination processing unit 34 according to the first application example.
  • determination processing based on overlap between grids will be described.
  • FIG. 8 shows controlled part information shown in part (a) of FIG. 3D information indicating the obstacle candidate information shown in the part (b) of FIG. Note that for convenience, FIG. 8 is shown two-dimensionally, like FIG. 7, and the state of each grid is represented in black if the grid is occupied, and unoccupied or has no information. are represented in white. Further, in the obstacle candidate information shown in part (b) of FIG.
  • a cube is schematically represented as an example of an obstacle.
  • a lattice corresponding to the robot arm 11, which is control unit information, is shown in an intermediate color between white and black (for example, gray).
  • the information output from the information generation unit 42 and the obstacle candidate information output from the information exclusion unit 33 in the virtual environment 4 are represented by voxels.
  • the state of the k-th lattice based on the controlled part information shown in part (a) of FIG. 8 is Crobot,k', and the obstacle candidate information shown in part (b) of FIG.
  • the state of the th lattice is Cenv. , k′.
  • the overlap of each lattice k is
  • FIG. 8 shows an example of "when the distance to the obstacle is equal to or greater than the threshold".
  • the controlled unit information shown in part a) and the obstacle candidate information shown in part (b) of FIG. That is, the value of overlap ⁇ Ck′ of grid k in equation (3) is 1 for grid k that is occupied. Therefore, the sum of the values of the overlap ⁇ Ck of the grid k with the number of grids occupied is equal to the number of grids occupied, as in Equation (2).
  • the grids corresponding to the robot arm 11 (gray) and the grids corresponding to the obstacles (black) are in the same state, that is, the grids in which the grids overlap are indicated by diagonal lines.
  • the value of the overlap ⁇ Ck' of the lattice k in the equation (3) is zero for the lattices having the same occupancy state (hatched lines). Therefore, when the sum of the values of the overlap ⁇ Ck' of the lattice k is calculated with the number of occupied lattices, the sum is smaller than the number of occupied lattices because the number of overlapping lattices is 0.
  • the distance to the obstacle is greater than or less than the threshold value, that is, whether the robot arm 11 is approaching or entering the obstacle area. It is possible to determine whether or not there is
  • the threshold for the distance to the obstacle depends on the resolution when expressed in voxels, that is, the grid size shown in FIG. The larger the grid size, the longer the threshold distance.
  • the grid size can be appropriately determined according to the size and operating speed of the robot arm 11, the task to be executed, the processing capability of the determination processing unit 34, and the like. It should be noted that the method of calculating the overlap by expressing with voxels in this way has the advantage of high calculation efficiency because it is an expression using an octotree, as in the information comparison unit 35 described above.
  • FIG. 9 is a diagram for explaining the processing of the determination processing unit 34 according to the first application example.
  • FIG. 9 shows an example of determination processing performed by the determination processing unit 34 based on the nearest neighbor distance.
  • the information output by the information generation unit 42 of the virtual environment 4 and the obstacle candidate information output by the information exclusion unit 33 are each expressed as a set of three-dimensional position information, for example, a set of points representing three-dimensional coordinates called point cloud data. can do.
  • FIG. 9 shows three-dimensional information indicating the robot arm 11 as the information output by the information generating unit 42, and three-dimensional information indicating a cube as an example of the obstacle candidate information.
  • the points with the closest Euclidean distance between the robot arm 11 and the obstacle are indicated by black dots, and the distances between the closest points are schematically indicated by arrows.
  • the upper part of FIG. 9 shows "when the distance to the obstacle is greater than or equal to the threshold value", and as is clear from FIG. 9, the closest distance between the robot arm 11 and the cube is far.
  • "When the distance to the obstacle is less than the threshold value” in the lower part of FIG. 9 indicates that the closest distance between the robot arm 11 and the cube is close.
  • the threshold value can be appropriately determined according to the size and movement speed of the robot arm 11, the task to be executed, the processing capability of the determination processing unit 34, and the like.
  • algorithms such as nearest neighbor search and k-nearest neighbor search can be used as a method of finding the nearest point.
  • the above processing corresponds to the operation of step S105 in the flowchart shown in FIG. 2 or 4 of the first or second embodiment.
  • the two types of specific processing methods of the determination processing unit 34 have been described above, the present invention is not limited to these.
  • the mobile device 1 is a robot arm
  • an instruction to limit the movement range or movement speed of the robot arm 11 or an instruction to stop the robot arm 11 is given, thereby improving work efficiency and safety.
  • High and precise control of the controlled unit 11 can be realized.
  • any mobile device 1 having a movable portion 11a such as other robots, machine tools, and assembly machines, can be applied.
  • it can be suitably applied to a working machine in which the movable part 11a such as an arm may enter the obstruction area.
  • the obstacle the case where there is one cube has been described as an example, but the shape and number of obstacles are not limited to this.
  • a 2nd application example shows the example of a backhoe as a case where the movable apparatus 1 in 1st or 2nd embodiment is a construction machine.
  • FIG. 10 is a diagram showing an example of the configuration of a control system 500 according to the second application.
  • the mobile device 1 of the second application includes at least a backhoe 11, a control unit 12 that controls the backhoe 11, and an observation device 2 mounted on the backhoe 11, as shown in FIG.
  • the observation device 2 is a device capable of acquiring three-dimensional information such as a depth camera or LiDAR.
  • the obstacle detection device 3 is the same as the obstacle detection device 3 in the first to third embodiments.
  • the configuration of the control system 500 shown in FIG. 10 is a configuration in which the movable device 1 and the obstacle detection device 3 are connected one-to-one, the number and configuration to be connected are not limited to this.
  • the control system 500 may be configured to have a plurality of mobile devices 1 , that is, a plurality of backhoes 11 .
  • the mobile device 1 includes at least the controlled unit 11 and the control unit 12, as in the first to third embodiments, and in the second application example, the backhoe 11 is the controlled unit 11 and controls the backhoe 11 Controller 12 is control unit 12 .
  • the control unit 12 may be included in the mobile device 1 or may exist in another location connected by a network. not.
  • the backhoe 11 may be automatically (autonomously) operated by the control unit 12, operated by an operator, or the operator may remotely transmit a control signal instead of the control unit 12. There are no restrictions on the method of controlling or manipulating 11.
  • the operator may be warned by an alert or the like. may intervene in the operator's operation by sending a deceleration or stop signal to the
  • FIG. 10 shows an example of an observation area 50 observed by the observation device 2 .
  • the observation area 50 includes at least part of the backhoe 11 .
  • a task assumed in the second application example is a task of excavating earth and sand present in a part of the target area 52 shown in FIG.
  • FIG. 10 shows an obstacle or obstruction area 53 that the backhoe 11 is not allowed to approach or enter. This failure area 53 has the same meaning as the failure area 53 of the first application example.
  • the position/orientation data acquisition unit 31 of the obstacle detection device 3 acquires information on each movable unit 11 a that constitutes the backhoe 11
  • the observation data acquisition unit 32 acquires three-dimensional information on the observation area 50 .
  • the position and orientation data may be obtained by a sensor attached to each movable portion 11a or the housing.
  • the sensor may be, for example, an externally installed sensor such as an inclination sensor, a gyro sensor, an acceleration sensor, or an encoder.
  • the virtual environment 4 constructs a model simulating the three-dimensional shape and movement of the backhoe 11 .
  • the environment setting unit 41 sets the model based on the information acquired by the position and orientation data acquisition unit 31, so that the real backhoe 11 and the model in the virtual environment 4 are synchronized. The position and orientation can be matched within the error range of . Also, based on the relationship between the positions and orientations of the real backhoe 11 and observation device 2, a model is set by the environment setting unit 41 in the virtual environment 4, that is, calibrated.
  • the position in the three-dimensional space occupied by the backhoe 11 and the position in the three-dimensional space occupied by the model generated by the information generation unit 42 are not They match within the margin of error.
  • control system 500 can determine defects in the sensors attached to each movable unit 11a or the housing described above.
  • the operation of control system 500 can be considered similar to control system 200 according to the second embodiment.
  • FIG. 11 is a diagram for explaining the processing of the determination processing unit 34 according to the second application example.
  • FIG. 11 schematically shows the processing of the determination processing section 34 when the obstacle detection device 3 of the third embodiment having the control plan data acquisition section 36 is applied.
  • FIG. 11 shows two models of the virtual environment 4 simulating the real backhoe 11 .
  • One is a model whose position and orientation are set based on the control plan acquired by the control plan data acquisition unit 36, and the other model reflects the current position and orientation acquired by the position and orientation data acquisition unit 31. model.
  • FIG. 11 shows a cubic shape as an example of the obstacle region 53 .
  • FIG. 11 shows the states of the respective models at two different times.
  • the time shown on the left side of FIG. 11 is the first time
  • the time after a certain period of time has passed from the first time shown on the right side of FIG. 11 is the second time.
  • the state of the model is set based on both planning information and current information.
  • the plan information is assumed to be the state after a certain period of time from the current state. That is, under ideal control, the current state will match the state based on the plan information after a certain period of time has passed.
  • FIG. 11 shows, as an example of the processing of the determination processing unit 34, the determination method based on the nearest neighbor distance described in the first application example.
  • arrows indicate the shape information of the model based on the plan information and the current information, and the nearest neighbor distance of the cube of the obstacle area 53 included in the obstacle candidate information. From FIG. 11, it can be seen that the model based on the planning information is close to the trouble area, but the model based on the current state is far from the trouble area.
  • the determination based on the plan information it is determined that "the distance from the failure area is less than the threshold value (similar to the lower part of FIG. 9)", and in the determination based on the current information, "the distance from the failure area is too short”. Threshold value or more (upper part of FIG. 9)”.
  • an instruction to decelerate the backhoe 11 is output by the determination based on the plan information, and the control of the backhoe 11 is continued without outputting the instruction by the determination based on the current state.
  • the controller 12 that controls the real backhoe 11 can only receive one instruction, so it is necessary to select one of the instructions.
  • This instruction selection can be realized by preparing a predetermined rule (algorithm) in advance. For example, since there is more time to make a decision based on plan information than a decision based on the current state, if the decision is made based on the plan information, the instruction should be "decelerate".
  • the control system 500 updates the model based on the current position and orientation information, for example, without updating based on the planning information.
  • the reason for this is that the plan information is up to a certain target value or is in a certain sequence, whereas the current information changes from moment to moment as long as the backhoe 11 is moving toward the target value.
  • the position of the backhoe 11 indicated by the model based on the current information is closer to the position of the backhoe 11 indicated by the model based on the plan information than at the first time. That is, the position of the backhoe 11 indicated by the model based on the current information is also close to the obstacle. Therefore, at the second time, it is assumed that "the distance to the faulty area is less than the threshold". Then, at the second time, the approach of the backhoe 11 to the obstacle is detected in both the model based on the plan information and the model based on the current information.
  • control plan data acquisition unit 36 selects to stop the backhoe 11 from the judgment result obtained by integrating the judgment result by the model based on the control plan and the judgment result by the model based on the current information. can be done.
  • the integration of determination results described above is an example, and the integration of determination results by the control plan data acquisition unit 36 is not limited to stopping the backhoe 11 .
  • FIG. 12 is a flowchart relating to processing of the determination processing unit 34 according to the second application example.
  • the processing of steps S501 to S506 performed by the determination processing unit 34 based on the current position and orientation information is the same as the processing of steps S101 to S106 of the first embodiment shown in FIG.
  • the determination processing unit 34 according to the second application acquires control plan information (step S507).
  • the determination processing unit 34 sets the virtual environment based on the configuration of the real environment and the information on the control plan (step S508).
  • the determination processing unit 34 outputs shape information based on the control plan for the model of the virtual environment (step S509).
  • step S504 the process of step S504 is performed, and the determination processing unit 34 outputs a determination value related to the distance between the obstacle candidate information and the shape information of the control plan (step S510).
  • the determination processing unit 34 determines whether or not the determination value is equal to or greater than the threshold (step S511).
  • the determination processing unit 34 confirms whether the determination in the process of step S6 is YES or NO. Determination processing part 34 continues operation of mobile 1, when determination in processing of Step S6 is YES (Step S512). If the determination in step S6 is NO, the determination processing unit 34 integrates the determination results (step S513). As described above, the integration can be determined based on predefined rules such as, for example, alert only or instruction to slow down when based on plan information, or instruction to stop when based on current information. This rule can be appropriately determined in consideration of the work environment, the content of the task, the performance of the mobile device 1, and the like. Then, based on the integrated determination result, the determination processing unit 34 outputs an alert display and an instruction to the control unit 12 (step S514).
  • predefined rules such as, for example, alert only or instruction to slow down when based on plan information, or instruction to stop when based on current information. This rule can be appropriately determined in consideration of the work environment, the content of the task, the performance of the mobile device 1, and the
  • step S511 NO the determination processing unit 34 confirms whether the determination in the process of step S6 is YES or NO. If the determination in step S6 is YES, the determination processing unit 34 integrates the determination results (step S513). If the determination in the process of step S6 is NO, the determination processing unit 34 proceeds to the process of step S13.
  • the second application example in which the movable device 1 is a construction machine and the controlled unit 11 is the backhoe 11 has been described above.
  • the backhoe 11 approaches or enters an obstructed area, an instruction to limit the operating range or operating speed of the backhoe 11 or an instruction to stop the backhoe 11 is given, thereby improving work efficiency and safety. Precise control can be realized.
  • the backhoe 11 was shown as an example of the to-be-controlled part 11 with which the mobile device 1 is equipped, it is applicable if it is the mobile device 1 which has the movable part 11a, such as another construction machine and a civil engineering construction machine.
  • the technology described in the second application can be suitably applied to a working machine in which the movable portion 11a such as an arm may enter the obstruction area.
  • the obstacles the case where there is one cubic obstacle has been described as an example, but the shape and number of the obstacles are not limited to this.
  • FIG. 13 is a diagram for explaining the processing of the information generation unit 42 and the determination processing unit 34 according to the third application example.
  • the information generation unit 42 according to the third application example generates information classified for each part of the robot arm 11 (arm 1, joint 1, arm 2, joint 2, arm 3, joint 3). That is, it outputs the occupation information of the three-dimensional space for each part.
  • the "position information occupied by a plurality of three-dimensional shapes based on a plurality of control plans" in the third embodiment corresponds to the "occupancy information of the three-dimensional space for each part" in the third application example.
  • the classification of the parts of the robot arm 11 described above is only an example and is not limited to this. A similar classification can also be applied to the second application example.
  • the information generation unit 42 in the first application example outputs information as to whether or not the three-dimensional space is occupied by the robot arm 11 . That is, in the first application example, all the three-dimensional information indicating the robot arm 11 are of the same classification.
  • the lower part of FIG. 13 shows a processing example of "when the distance to the obstacle is less than the threshold value" described in FIG. 8 in the first application example.
  • the grid in which the obstacle region and the controlled section 11 overlap is the same grid (dot pattern) as in FIG. 8 in the first application example.
  • the determination processing unit 34 is described as outputting information as to whether or not there is an overlap.
  • the determination processing unit 34 provides status information indicating whether or not there is an overlap for each of the classified parts of the robot arm 11 (overlapping status information), as shown on the right side of the lower part of FIG. : detected, no overlap :-) is output.
  • the obstacle candidate information is the same as the obstacle candidate information in the first application example, and the information generation unit 42 outputs a plurality of pieces of shape information for each part. Therefore, the process of outputting the status information by the determination processing unit 34 is the process of outputting a determination value related to the distance between the obstacle candidate information and the shape information (step S105) in the operation flow shown in FIG. ) and the process of determining whether the determination value is equal to or greater than the threshold value (step S106) can be realized by repeating the number of parts or executing them in parallel for each part.
  • the present invention has been described with the above-described embodiments and application examples as examples. However, the invention is not limited to what has been described above.
  • the present invention can be applied to various forms without departing from the gist of the present invention.
  • some or all of the functions of the movable device 1, the observation device 2, and the obstacle detection device 3 may be provided in a device different from the own device.
  • the device including the determination processing unit 34 is the processing device.
  • FIG. 14 is a diagram showing the minimum configuration of the processing device 1000 according to the embodiment.
  • the processing device 1000 includes a determination unit 1000a (an example of determination means) and a processing unit 1000b (an example of processing means).
  • the determination unit 1000a determines whether or not the controlled object has entered a region other than the region where the controlled object is allowed to enter, in an environment that includes at least a part of the controlled object having a movable part.
  • the processing unit 1000b executes a predetermined process when the determination unit determines that the controlled object has entered an area other than the area into which the controlled object is allowed to enter. Examples of the predetermined processing include notification that the controlled object has entered an area other than the area where entry of the controlled object is permitted.
  • the determination unit 1000a determines whether or not the controlled object has entered a region other than the region where the controlled object is allowed to enter, in an environment that includes at least a part of the controlled object having a movable part (step S1001).
  • the processing unit 1000b executes a predetermined process when the determining unit determines that the controlled object has entered an area other than the area into which the controlled object is allowed to enter (step S1002).
  • the processing apparatus 1000 with the minimum configuration according to the embodiment of the present invention has been described above. With this processing device 1000, precise control of the controlled object can be realized.
  • control systems 100, 200, 300, 400, 500, the movable device 1, the observation device 2, the obstacle detection device 3, and other control devices have computer systems therein. You may have The process of the above-described processing is stored in a computer-readable recording medium in the form of a program, and the above-described processing is performed by reading and executing this program by a computer. Specific examples of computers are shown below.
  • FIG. 16 is a schematic block diagram showing the configuration of a computer according to at least one embodiment.
  • the computer 5 includes a CPU 6, a main memory 7, a storage 8, and an interface 9, as shown in FIG.
  • each of the control systems 100 , 200 , 300 , 400 , 500 , the mobile device 1 , the observation device 2 , the obstacle detection device 3 , and other control devices described above is implemented in the computer 5 .
  • the operation of each processing unit described above is stored in the storage 8 in the form of a program.
  • the CPU 6 reads out the program from the storage 8, develops it in the main memory 7, and executes the above process according to the program.
  • the CPU 6 secures storage areas corresponding to the storage units described above in the main memory 7 according to the program.
  • storage 8 examples include HDD (Hard Disk Drive), SSD (Solid State Drive), magnetic disk, magneto-optical disk, CD-ROM (Compact Disc Read Only Memory), DVD-ROM (Digital Versatile Disc Read Only Memory) , semiconductor memory, and the like.
  • the storage 8 may be an internal medium directly connected to the bus of the computer 5, or an external medium connected to the computer 5 via the interface 9 or communication line. Further, when this program is distributed to the computer 5 via a communication line, the computer 5 that receives the distribution may develop the program in the main memory 7 and execute the above process.
  • storage 8 is a non-transitory, tangible storage medium.
  • the above program may implement part of the functions described above.
  • the program may be a file capable of realizing the above-described functions in combination with a program already recorded in the computer system, that is, a so-called difference file (difference program).
  • processing device processing method, program (agent: please describe according to the claims) device, etc. according to the present invention, it is possible to realize precise control of the controlled object.

Landscapes

  • Engineering & Computer Science (AREA)
  • Mining & Mineral Resources (AREA)
  • Civil Engineering (AREA)
  • General Engineering & Computer Science (AREA)
  • Structural Engineering (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
PCT/JP2021/041549 2021-11-11 2021-11-11 処理装置、処理方法、およびプログラム Ceased WO2023084695A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2023559309A JP7816372B2 (ja) 2021-11-11 2021-11-11 処理装置、処理方法、およびプログラム
PCT/JP2021/041549 WO2023084695A1 (ja) 2021-11-11 2021-11-11 処理装置、処理方法、およびプログラム
US18/704,508 US20240417955A1 (en) 2021-11-11 2021-11-11 Processing device, processing method, and program

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2021/041549 WO2023084695A1 (ja) 2021-11-11 2021-11-11 処理装置、処理方法、およびプログラム

Publications (1)

Publication Number Publication Date
WO2023084695A1 true WO2023084695A1 (ja) 2023-05-19

Family

ID=86335351

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/041549 Ceased WO2023084695A1 (ja) 2021-11-11 2021-11-11 処理装置、処理方法、およびプログラム

Country Status (3)

Country Link
US (1) US20240417955A1 (https=)
JP (1) JP7816372B2 (https=)
WO (1) WO2023084695A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116604558A (zh) * 2023-05-26 2023-08-18 佛山显扬科技有限公司 未完全遮挡物体的抓取方法、装置、计算机设备及介质

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017170581A (ja) * 2016-03-24 2017-09-28 ファナック株式会社 複数の機構ユニットにより構成されたロボットを制御するロボットシステム、該機構ユニット、およびロボット制御装置
JP2021146422A (ja) * 2020-03-17 2021-09-27 日本電産株式会社 ロボットの動作領域モデリング装置及びプログラム

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4844453B2 (ja) * 2007-04-09 2011-12-28 株式会社デンソーウェーブ ロボットの教示装置及び教示方法

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017170581A (ja) * 2016-03-24 2017-09-28 ファナック株式会社 複数の機構ユニットにより構成されたロボットを制御するロボットシステム、該機構ユニット、およびロボット制御装置
JP2021146422A (ja) * 2020-03-17 2021-09-27 日本電産株式会社 ロボットの動作領域モデリング装置及びプログラム

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116604558A (zh) * 2023-05-26 2023-08-18 佛山显扬科技有限公司 未完全遮挡物体的抓取方法、装置、计算机设备及介质

Also Published As

Publication number Publication date
US20240417955A1 (en) 2024-12-19
JPWO2023084695A1 (https=) 2023-05-19
JP7816372B2 (ja) 2026-02-18

Similar Documents

Publication Publication Date Title
CN113107043B (zh) 使用传感器融合控制机器的运动
US11745355B2 (en) Control device, control method, and non-transitory computer-readable storage medium
CN110805093B (zh) 利用视觉传感器进行反馈回路控制的容器角度感测
JP6564739B2 (ja) 作業機械
US12194630B2 (en) Industrial robot system and method for controlling an industrial robot
JP7365122B2 (ja) 画像処理システムおよび画像処理方法
EP2781314A1 (en) Robot picking system, control device, and method of manufacturing a workpiece
JP7180695B2 (ja) 障害物回避制御装置、障害物回避制御システム、障害物回避制御方法およびプログラム
AU2017276225B2 (en) Systems and methods for preparing a worksite for additive construction
JP7323057B2 (ja) 制御装置、制御方法、および、制御プログラム
JP2009193240A (ja) 移動ロボット及び環境地図の生成方法
CN112508912B (zh) 地面点云数据滤除方法及装置、臂架防碰撞方法及系统
Zhang et al. Toward autonomous mining: design and development of an unmanned electric shovel via point cloud-based optimal trajectory planning
EP3961556B1 (en) Object recognition device and object recognition method
WO2022091366A1 (ja) 情報処理システム、情報処理装置、情報処理方法、及び、記録媒体
CN116188567A (zh) 到动态对象的最小距离的有效计算方法
CN120443704A (zh) 一种挖掘机轨迹控制方法、系统及挖掘机
JP7816372B2 (ja) 処理装置、処理方法、およびプログラム
JP6615058B2 (ja) 作業機械
WO2022230880A1 (ja) 制御方法、制御システム、及び制御装置
Satoh Digital twin-based collision avoidance system for autonomous excavator with automatic 3d lidar sensor calibration
CN112127417B (zh) 用于生成工程机械周围的环境数据的装置及包括其的工程机械
Aalerud et al. Industrial environment mapping using distributed static 3d sensor nodes
CN119292066A (zh) 一种掘进机负载自适应控制方法及系统
WO2024201823A1 (ja) 情報処理システム、情報処理方法、記憶媒体

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21964050

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18704508

Country of ref document: US

ENP Entry into the national phase

Ref document number: 2023559309

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21964050

Country of ref document: EP

Kind code of ref document: A1