US10393515B2 - Three-dimensional scanner and measurement assistance processing method for same - Google Patents

Three-dimensional scanner and measurement assistance processing method for same Download PDF

Info

Publication number
US10393515B2
US10393515B2 US16/070,025 US201616070025A US10393515B2 US 10393515 B2 US10393515 B2 US 10393515B2 US 201616070025 A US201616070025 A US 201616070025A US 10393515 B2 US10393515 B2 US 10393515B2
Authority
US
United States
Prior art keywords
depth sensor
dimensional
mapping
measurement
main body
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
US16/070,025
Other languages
English (en)
Other versions
US20190049240A1 (en
Inventor
Ryosuke Kawanishi
Makito Seki
Keisuke SAMBONGI
Yu ENOMOTO
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mitsubishi Electric Corp
Original Assignee
Mitsubishi Electric Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mitsubishi Electric Corp filed Critical Mitsubishi Electric Corp
Assigned to MITSUBISHI ELECTRIC CORPORATION reassignment MITSUBISHI ELECTRIC CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: Enomoto, Yu, SAMBONGI, Keisuke, SEKI, MAKITO, KAWANISHI, Ryosuke
Publication of US20190049240A1 publication Critical patent/US20190049240A1/en
Application granted granted Critical
Publication of US10393515B2 publication Critical patent/US10393515B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G01B11/245—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures using a plurality of fixed, simultaneously operating transducers
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
    • G01B11/2513—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object with several lines being projected in more than one direction, e.g. grids, patterns
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C15/00—Surveying instruments or accessories not provided for in groups G01C1/00 - G01C13/00
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C3/00—Measuring distances in line of sight; Optical rangefinders
    • G01C3/02—Details
    • G01C3/06—Use of electric means to obtain final indication
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/4808—Evaluating distance, position or velocity data
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/50—Systems of measurement based on relative movement of target
    • G01S17/58—Velocity or trajectory determination systems; Sense-of-movement determination systems

Definitions

  • the present invention relates to: a three-dimensional scanner configured to measure the dimensions of a three-dimensional space, for example, a production site and an elevator shaft, to create a three-dimensional map thereof; and a measurement assistance processing method for the three-dimensional scanner.
  • mapping As one method of acquiring the shape of the three-dimensional space, there is a method of manually scanning through use of a three-dimensional (3D) sensor. In this case, a feature is extracted from each of a plurality of pieces of measurement data. Then, the pieces of measurement data are subjected to registration so that the features observed in common between the pieces of measurement data overlap each other, to thereby create (update) a three-dimensional map. Such processing for the registration of pieces of data is called mapping.
  • a control unit includes a three-dimensional map generation unit.
  • the three-dimensional map generation unit moves a camera and generates a partial three-dimensional map based on two two-dimensional images picked up at two spots.
  • the control unit causes the camera to perform photographing from each of different spots on a guide rail, and generates and stores an entire three-dimensional map from the acquired image. Further, the control unit derives, based on the stored entire three-dimensional map, a spot from which a freely-selected image pickup target spot can be viewed without an obstruction.
  • control unit causes the camera to continue to pick up an image until a three-dimensional map including the image pickup target spot can be generated, and to generate the three-dimensional map of the part to acquire the three-dimensional map of the image target spot (see, for example, Patent Literature 1).
  • a self position is estimated by detecting a feature point in an image picked up by an image capture device mounted to a mobile body and detecting the position of an object in the periphery of the mobile body from a change of the feature point on the image in accordance with the movement of the mobile body.
  • the image pickup direction of the image capture device is set as an initial direction.
  • the image pickup direction of the image capture device is set to be a direction that enables the pickup of at least a part of feature points existing in the image acquired before the mobile body rotates.
  • the present invention has been made in order to solve the above-mentioned problems, and has an object to obtain a three-dimensional scanner and a measurement assistance processing method for the three-dimensional scanner, which are capable of creating a three-dimensional map more smoothly with higher accuracy.
  • a three-dimensional scanner including: a depth sensor capable of acquiring measurement data on a distance to a measurement target as a set of points in a three-dimensional space; and a scanner main body configured to: extract features from a plurality of pieces of measurement data acquired by the depth sensor; and create a three-dimensional map by superimposing the extracted features on one another, wherein the scanner main body is configured to: calculate a position and direction of the depth sensor; determine a movement candidate, which is a candidate for a position and direction to/in which the depth sensor is to be moved next; acquire a feature within the movement candidate, which is the feature observable by the depth sensor from the movement candidate; evaluate stability of mapping performed from the movement candidate through use of the feature within the movement candidate; and present at least any one of a moving direction or moving speed of the depth sensor to a user based on an evaluation result.
  • measurement assistance processing method for a three-dimensional scanner, the three-dimensional scanner including: a depth sensor capable of acquiring measurement data on a distance to a measurement target as a set of points in a three-dimensional space; and a scanner main body configured to: extract features from a plurality of pieces of measurement data acquired by the depth sensor; and create a three-dimensional map by superimposing the extracted features on one another, the measurement assistance processing method being a processing method for assisting three-dimensional measurement by the scanner main body of the three-dimensional scanner, the measurement assistance processing method including the steps of: calculating a position and direction of the depth sensor; determining a movement candidate, which is a candidate for a position and direction to/in which the depth sensor is to be moved next; acquiring a feature within the movement candidate, which is the feature observable by the depth sensor from the movement candidate; evaluating stability of mapping performed from the movement candidate through use of the feature within the movement candidate; and presenting at least any one of a moving direction or moving speed of
  • the moving direction or moving speed of the depth sensor is presented to the user based on a result of evaluating the stability of the mapping, and hence it is possible to create the three-dimensional map more smoothly with higher accuracy.
  • FIG. 1 is a block diagram for illustrating a three-dimensional scanner according to a first embodiment of the present invention.
  • FIG. 2 is a plan view for illustrating how a user scans a measurement target by the three-dimensional scanner according to the first embodiment to create a three-dimensional map.
  • FIG. 3 is a view of a state of work illustrated in FIG. 2 , which is viewed from behind the user.
  • FIG. 4 is an explanatory diagram for illustrating an example of a method of extracting shape features by a feature extraction module of FIG. 1 .
  • FIG. 5 is an explanatory diagram for illustrating first and second feature point groups, which are obtained by the feature extraction module of FIG. 1 , side by side.
  • FIG. 6 is an explanatory diagram for illustrating a state under which common parts between the first and second feature point groups of FIG. 5 are caused to overlap each other.
  • FIG. 7 is an explanatory diagram for illustrating a state under which common parts between third and fourth feature point groups are caused to overlap the first and second feature point groups of FIG. 6 .
  • FIG. 8 is an explanatory diagram for illustrating an example of a mapping result obtained when a prediction error is small.
  • FIG. 9 is an explanatory diagram for illustrating an example of a mapping result obtained when the prediction error is large.
  • FIG. 10 is an explanatory diagram for illustrating an example of display contents displayed on a display device by an output module of FIG. 1 .
  • FIG. 11 is an explanatory diagram for illustrating a state of mapping work corresponding to a three-dimensional map display area of FIG. 10 .
  • FIG. 12 is an explanatory diagram for illustrating an example of a horizontal cross section of an elevator shaft displayed on the display device by the output module of FIG. 1 .
  • FIG. 13 is an explanatory diagram for illustrating an example of a vertical cross section of the elevator shaft displayed on the display device by the output module of FIG. 1 .
  • FIG. 14 is a flow chart for illustrating an operation of a scanner main body of FIG. 1 .
  • FIG. 15 is a block diagram for illustrating a three-dimensional scanner according to a second embodiment of the present invention.
  • FIG. 16 is an explanatory diagram for illustrating an example of a method of extracting visual features by a feature extraction module of FIG. 15 .
  • FIG. 17 is a front view for illustrating a three-dimensional scanner according to a third embodiment of the present invention.
  • FIG. 18 is a back view for illustrating the three-dimensional scanner of FIG. 17 .
  • FIG. 19 is a block diagram for illustrating a three-dimensional scanner according to a fourth embodiment of the present invention.
  • FIG. 20 is a flow chart for illustrating an operation of a scanner main body of FIG. 19 .
  • FIG. 21 is a block diagram for illustrating a three-dimensional scanner according to a fifth embodiment of the present invention.
  • FIG. 22 is a flow chart for illustrating an operation of a scanner main body of FIG. 21 .
  • FIG. 23 is a block diagram for illustrating a three-dimensional scanner according to a sixth embodiment of the present invention.
  • FIG. 24 is a flow chart for illustrating an operation of a scanner main body of FIG. 23 .
  • FIG. 25 is a plan view for illustrating how a three-dimensional map of an inside of a pit of the elevator shaft is created through use of the three-dimensional scanner according to each of the first to sixth embodiments.
  • FIG. 26 is a perspective view for illustrating a state under which the three-dimensional scanner according to each of the first to sixth embodiments is mounted to a mobile robot.
  • FIG. 1 is a block diagram for illustrating a three-dimensional scanner according to a first embodiment of the present invention.
  • the three-dimensional scanner includes a scanner main body 1 and a depth sensor 2 connected to the scanner main body 1 .
  • the depth sensor 2 is a sensor capable of acquiring distance data, which is measurement data on a distance to a measurement target, as a set (three-dimensional point cloud) of points in a three-dimensional space.
  • distance data is measurement data on a distance to a measurement target, as a set (three-dimensional point cloud) of points in a three-dimensional space.
  • a method of measuring the distance it is possible to use various methods, for example, a time-of-flight (ToF) method and an active stereo method, which uses pattern projection performed by a system using a projector and a camera.
  • ToF time-of-flight
  • the depth sensor 2 may be manually moved by a user while being held by hand, or may be mounted to a carriage or the like so that measurement is performed by moving the carriage.
  • the distance can be measured at a timing at which the user inputs an instruction, or can be automatically measured at every time interval set in advance.
  • the scanner main body 1 can be formed of a computer including an arithmetic processing unit (CPU), a storage (including a ROM, a RAM, and a hard disk drive), and a signal input/output module.
  • the scanner main body 1 has functions implemented by the computer. That is, a program for implementing the functions of the scanner main body 1 is stored in the storage of the computer.
  • a tablet PC can be used as the computer that forms the scanner main body 1 .
  • FIG. 2 is a plan view for illustrating how the user scans the measurement target by the three-dimensional scanner according to the first embodiment to create a three-dimensional map.
  • FIG. 3 is a view of a state of work illustrated in FIG. 2 , which is viewed from behind the user. As illustrated in FIG. 2 and FIG. 3 , when a tablet PC is used as the scanner main body 1 with the depth sensor 2 being fixed to the scanner main body 1 , the user can hold the three-dimensional scanner by hand to move or rotate the three-dimensional scanner. That is, it is possible to handle the six degrees of freedom of translation and rotation.
  • the scanner main body 1 includes, as functional blocks, a feature extraction module 11 , a mapping module 12 , a storage 13 , a movement candidate determination module 14 , a registered feature acquisition module 15 , a mapping evaluation module 16 , and an output module 17 .
  • the feature extraction module 11 extracts a feature of the measurement target based on a plurality of pieces of measurement data obtained from the depth sensor 2 .
  • the feature to be extracted includes a shape feature and a so-called primitive.
  • the shape feature includes, for example, an edge line or other such straight line, vertices or other such points, a direction of the straight line, and a normal to a plane.
  • the primitive includes, for example, a circle, a rectangle, and a sphere.
  • a curvature calculation method can be used for extracting vertices.
  • a Hough transform method can be used for the extraction of an edge line.
  • a model fitting method using a 3D model can be used for the extraction of a primitive.
  • FIG. 4 is an explanatory diagram for illustrating an example of a method of extracting shape features by the feature extraction module 11 of FIG. 1 .
  • a rectangular parallelepiped is measured by the depth sensor 2
  • such a three-dimensional point cloud 21 as illustrated in FIG. 4 is obtained as measurement data.
  • a sphere or the like can also be extracted by similar primitive fitting.
  • edge lines 21 b by detecting points exhibiting a steep gradient change in the periphery of the three-dimensional point cloud 21 and detecting a straight line connecting those points. Further, it is possible to extract a plurality of vertices 21 c from information on, for example, a point of interest having the periphery exhibiting a high curvature or an intersection point of edge lines.
  • the mapping module 12 causes features observed in common between different pieces of measurement data to overlap each other through use of the extracted features, and subjects the pieces of measurement data to registration, to thereby create and update a three-dimensional map.
  • the mapping module 12 then calculates the position and direction of the depth sensor 2 (hereinafter referred to as “sensor position/direction”) exhibited at the time of data acquisition based on a result of the registration.
  • the mapping module 12 further adds and updates three-dimensional map information, sensor position information, and sensor direction information in the storage 13 .
  • the storage 13 stores the information obtained from the mapping module 12 .
  • the mapping module 12 first selects at least two pieces of measurement data, namely, 3D data. Subsequently, the mapping module 12 extracts a feature from the selected pieces of 3D data. At this time, different constraint conditions are obtained at the time of the calculation for the overlapping depending on what kind of features are extracted and used.
  • the mapping module 12 associates features of the pieces of 3D data with each other. That is, the mapping module 12 finds common features. Then, coordinate transformation is performed so that the features of the pieces of 3D data overlap each other. For example, with the pieces of 3D data before the movement being used as a reference, the pieces of 3D data after the movement are subjected to the coordinate transformation so that a feature in the pieces of 3D data after the movement overlaps a feature in the pieces of 3D data before the movement.
  • a relative position/posture relationship of the depth sensor 2 after the movement with respect to the depth sensor 2 before the movement is obtained.
  • a transformation matrix for coordinate transformation that is, a matrix formed of relative position-posture information on the depth sensor 2 . At least one kind of feature among features including points (any one of 2D or 3D is acceptable), lines (2D, 3D), and planes (3D) is used to calculate the transformation matrix.
  • FIG. 5 is an explanatory diagram for illustrating first and second feature point groups t 1 and t 2 , which are obtained by the feature extraction module 11 of FIG. 1 , side by side.
  • FIG. 6 is an explanatory diagram for illustrating a state under which common parts between the first and second feature point groups t 1 and t 2 of FIG. 5 are caused to overlap each other.
  • FIG. 7 is an explanatory diagram for illustrating a state under which common parts between third and fourth feature point groups of FIG. 6 are caused to overlap the first and second feature point groups t 1 and t 2 .
  • the mapping module 12 compares the first feature point group t 1 included in first measurement data and the second feature point group t 2 included in second measurement data to detect common parts therebetween. Then, the coordinates of the second measurement data are transformed so that the common parts (within the dotted line in FIG. 6 ) overlap each other.
  • the mapping module 12 compares the third feature point group t 3 included in third measurement data with the feature point groups t 1 and t 2 to detect common parts therebetween, and transforms the coordinates of the third measurement data so that the common parts overlap each other. Subsequently, the mapping module 12 compares the fourth feature point group t 4 included in fourth measurement data with the feature point groups t 1 , t 2 , and t 3 to detect common parts therebetween, and transforms the coordinates of the fourth measurement data so that the common parts overlap each other. Such processing is repeated to create a three-dimensional map.
  • the movement candidate determination module 14 determines a movement candidate being a candidate for a position and direction to/in which the depth sensor 2 is to be moved next. There may be only one movement candidate, or may be two or more movement candidates.
  • the movement candidate for the position may be the current position.
  • a spot moved to/in a relatively fixed position and direction with respect to the current sensor position/direction for example, moved in four directions by a translation amount of 10 cm and a rotation amount of 10 degrees in each of up, down, right, and left directions, may be set as the movement candidate.
  • a position and direction to/in which the movement is to be performed may be manually input.
  • the position and direction to/in which the depth sensor 2 is to be moved next may be predicted from the movement locus of the depth sensor 2 to the current position, and the predicted position and direction may be set as the movement candidate.
  • the registered feature acquisition module 15 acquires a feature within the movement candidate, which is a feature estimated to be observable by the depth sensor 2 from the movement candidate, from among the features registered in the three-dimensional map.
  • the mapping evaluation module 16 evaluates the stability of the mapping performed from the movement candidate through use of the feature within the movement candidate, which has been acquired by the registered feature acquisition module 15 .
  • a method of evaluating the stability of the mapping there is a method of determining whether or not the minimum number of features required for the registration are observable, that is, determining whether or not the calculation itself for the registration can be performed. In this case, for example, when the number of features is equal to or smaller than a number set in advance, it is determined that the calculation cannot be performed.
  • mapping As another method of evaluating the stability of the mapping, there is a method of calculating a prediction error in mapping from the kind and arrangement of features.
  • the arrangement of features represents a distribution of observable features within the sensor field of view from the movement candidate.
  • the sensor position/direction calculated through the mapping is the same as the actual sensor position/direction. Even in a case where there is a measurement error, when the common parts between the different feature point groups t 1 and t 2 are dispersed over a relatively wide range as illustrated in FIG. 8 , the sensor position/direction calculated through the mapping is estimated to be substantially the same as the actual sensor position/direction, and the prediction error is small.
  • the minute change amount of the position of the feature can be defined as the amount of displacement of the feature, which may occur when the feature is actually extracted from the sensor data, and there is a method of determining the minute change amount as the resolution of the distance measurement (for example, measurement error in a depth direction or distance measurement interval in a horizontal (vertical) direction) when the depth sensor 2 uses a ToF method and as one pixel when the depth sensor 2 uses a camera.
  • the registration error of the measurement data that occurs at the time of the mapping is expected to become zero irrespective of the magnitude of a prediction error.
  • an error always occurs in actual measurement, and hence it is assumed that, as the prediction error becomes larger, the registration error of the measurement data at the time of the actual mapping becomes larger as well.
  • the mapping evaluation module 16 first selects one sensor position/posture for which the mapping is to be evaluated. At this time, for example, the position/posture is shifted by a fixed amount from the current sensor position/posture. Subsequently, the mapping evaluation module 16 acquires a feature observable from the selected sensor position/posture from the map that has been created.
  • the mapping evaluation module 16 determines whether or not the mapping can be performed through use of the acquired feature. For example, at least three different points are required as the 3D points, and at least three different planes are required as the planes. When a plurality of features are used in combination, the required minimum number of features varies depending on the combination of features, for example, two 3D points and one plane are required. A criterion for determining whether or not the mapping can be performed thus differs depending on the feature to be used.
  • the mapping is executed through use of all features.
  • the mapping is also executed by shifting the position (direction in the case of a normal to a plane) of the feature to be used for the mapping. At this time, for example, the position of the feature is shifted by one pixel on an image, or shifted by 1 mm in the space.
  • mapping is executed through use of a minimum number or a larger number of features that enable the mapping from among the acquired features.
  • a statistic using the relative change amount of the sensor position/posture is acquired from a plurality of mapping results obtained by any one of those mapping methods or by a combination thereof, and an evaluation value is calculated based on the statistic.
  • a reprojection error of the feature may be set for the evaluation value.
  • the reprojection error is obtained by projecting the feature in the map onto a 2D image acquired at each sensor position/posture through use of the calculated position/posture of the sensor, and calculating a difference between the 2D position of the feature on the original 2D image and the 2D position of the projected feature. This is one of basic evaluation methods in the relevant technical field.
  • At least any one of the deviation or density of the distribution of features within the sensor field of view may be used as the evaluation value.
  • the evaluation becomes more unsatisfactory as the deviation becomes larger, and that the evaluation becomes more satisfactory as the density becomes higher.
  • the predicted moving direction of the depth sensor 2 may be used for evaluation.
  • the evaluation is assumed to be satisfactory when, for example, a large number of features exist in the moving direction of the sensor.
  • the output module 17 presents at least any one of the moving direction or moving speed of the depth sensor 2 corresponding to the movement candidate to the user based on an evaluation result obtained from the mapping evaluation module 16 .
  • a method for the presentation there are a method of performing indication in an illustration through use of a display or other such display device and a method using voice.
  • the moving direction is obtained as a direction from the current position of the depth sensor 2 toward a movement candidate for which the stability of the mapping has been evaluated.
  • the moving speed is obtained as a speed at which a blur ascribable to the moving speed does not occur in the acquired data of the depth sensor 2 , or is obtained by, for example, dividing a distance from the current position of the depth sensor 2 to the movement candidate for which the stability of the mapping has been evaluated by the time interval for the data acquisition of the depth sensor 2 .
  • the moving speed may be acquired as an absolute value or may be acquired as a relative value with respect to the moving speed of the depth sensor 2 at the time of the previous data acquisition.
  • FIG. 10 is an explanatory diagram for illustrating an example of display contents displayed on the display device by the output module 17 of FIG. 1 .
  • FIG. 10 an example of performing the display on a display 1 a of a tablet PC being the display device is illustrated.
  • the display 1 a is divided into two parts of a left part and a right part.
  • the left part is the measurement situation display area 1 b
  • the right part is a three-dimensional map display area 1 c.
  • the current situation of measurement performed by the depth sensor 2 and the presentation contents of the moving direction and moving speed of the depth sensor 2 output by the output module 17 are displayed.
  • the three-dimensional map display area 1 c a three-dimensional map being created is displayed.
  • a frame portion 1 d is provided to the four peripheral sides of the measurement situation display area 1 b .
  • a direction to a mapped area and a direction to an unmeasured area are displayed so as to be distinguished from each other.
  • the direction to the mapped area and the direction to the unmeasured area are distinguished from each other by different colors or tones.
  • the darker part indicates the direction to the mapped area.
  • the extracted features and the presented moving direction and moving speed of the depth sensor 2 are displayed.
  • the features are indicated by the triangles
  • the moving directions of the depth sensor 2 are indicated by the directions of the arrows
  • the moving speeds of the depth sensor 2 are indicated by the lengths of the arrows.
  • FIG. 11 is an explanatory diagram for illustrating a state of the mapping work corresponding to the three-dimensional map display area 1 c of FIG. 10 .
  • the path of the depth sensor 2 is indicated by the arrows, and the field of view of the depth sensor 2 is indicated by the dotted-line rectangles.
  • the display becomes closer to the two-dimensional drawing, which facilitates an intuitive understanding.
  • the projection style of a vertical sectional view is perspectively projected, the display becomes closer to a three-dimensional appearance, which facilitates an intuitive understanding.
  • FIG. 14 is a flow chart for illustrating an operation of the scanner main body 1 of FIG. 1 .
  • the scanner main body 1 acquires measurement data from the depth sensor 2 (Step S 1 ), and extracts a feature from the measurement data (Step S 2 ). Subsequently, the scanner main body 1 creates a three-dimensional map by superimposing the features of a plurality of pieces of measurement data (Step S 3 ). After that, the scanner main body 1 determines whether or not the mapping is successful (Step S 4 ), and when the mapping is successful, updates the three-dimensional map (Step S 5 ).
  • the scanner main body 1 inquires of the user whether or not to end the mapping (Step S 12 ). When the mapping is to be continued, the scanner main body 1 notifies the user to move the depth sensor 2 (Step S 13 ), and the procedure returns to Step 1 .
  • the scanner main body 1 calculates and updates the current position and direction of the depth sensor 2 (Step S 6 ). Subsequently, the scanner main body 1 determines the movement candidate of the depth sensor 2 (Step S 7 ), and acquires the feature within the movement candidate (Step S 8 ).
  • the scanner main body 1 evaluates the stability of the mapping performed from the movement candidate through use of the feature within the movement candidate (Step S 9 ).
  • the mapping evaluation is performed until the stability of the mapping is evaluated for all the movement candidates (Step S 9 and Step S 10 ).
  • the scanner main body 1 presents at least any one of the moving direction or moving speed of the depth sensor 2 to the user as an evaluation result (Step S 11 ). Then, the scanner main body 1 inquires of the user whether or not to end the mapping (Step S 12 ). When the mapping is to be continued, the scanner main body 1 notifies the user to move the depth sensor 2 (Step S 13 ), and the procedure returns to the Step 1 . When an instruction to end the mapping is input by the user, the processing is brought to an end.
  • the stability of the mapping is evaluated for the movement candidate by the scanner main body 1 , and the moving direction of the sensor in which the mapping is expected to fail is known in advance. Therefore, it is possible to prevent a failure in mapping, and to reduce time and labor to redo scanning work. In addition, it is possible to know the moving direction that enables the mapping to be performed with high accuracy, and hence the accuracy of the mapping is expected to be improved. Further, it is no longer required to repeatedly redo scanning by trial and error until the accuracy is satisfactory.
  • FIG. 15 is a block diagram for illustrating a three-dimensional scanner according to a second embodiment of the present invention.
  • a visible camera 4 capable of acquiring a color or monochrome two-dimensional image is used in combination, and the mapping is performed by using not only the shape feature extracted from the distance data but also a visual feature extracted from image data at the same time.
  • the feature extraction module 11 extracts a feature from the distance data obtained from the depth sensor 2 , and also extracts visual features, for example, a feature point and a straight line, from the image data obtained from the visible camera 4 .
  • visual features for example, a feature point and a straight line
  • Hough transform can be applied to a corner point corresponding to the corner of an object.
  • Hough transform can be applied to the contour of an object after the contour is detected from an image.
  • FIG. 16 is an explanatory diagram for illustrating an example of a method of extracting visual features by the feature extraction module 11 of FIG. 15 .
  • FIG. 16 an image acquired by the visible camera 4 is illustrated.
  • the x marks indicate the extracted feature points.
  • a point exhibiting a large change in hue around a point of interest is extracted as a feature point.
  • it is possible to extract a straight line but a point that cannot be distinguished from a neighboring point, for example, a point on a straight line, is not extracted as a feature point even when the change in hue is large.
  • the corners of a pillar and the corners of a poster 22 hanging on a wall are extracted as feature points.
  • the feature points can also be extracted from the pattern of the poster 22 and the like.
  • the feature obtained from the image data is also used in addition to the feature obtained from the distance data. All other configurations and operations are similar or identical to the first embodiment.
  • the visible camera 4 may be integrated with the depth sensor 2 , or may be separately provided.
  • FIG. 17 is a front view for illustrating a three-dimensional scanner according to a third embodiment of the present invention.
  • FIG. 18 is a back view for illustrating the three-dimensional scanner of FIG. 17 .
  • a casing 5 to which the depth sensor 2 and the visible camera 4 are mounted is fixed to the scanner main body 1 .
  • the depth sensor 2 a camera for measuring a distance is used.
  • an illumination device 6 and a projector 7 are mounted to the casing 5 .
  • the casing 5 and the scanner main body 1 are connected to each other through a cable 8 .
  • the devices mounted to the casing 5 are supplied with power from the scanner main body 1 through the cable 8 .
  • the depth sensor 2 and the visible camera 4 transmit data to the scanner main body 1 through the cable 8 .
  • the devices mounted to the casing 5 also receive an instruction signal from the scanner main body 1 through the cable 8 .
  • the projector 7 projects a pattern onto the measurement target.
  • the depth sensor 2 photographs the projected pattern to detect the distance data by an active stereo method.
  • the illumination device 6 irradiates the measurement target with light. All other configurations and operations are similar or identical to the second embodiment.
  • an active stereo method using pattern projection performed by a system using a projector and a camera may be used.
  • light is applied to the measurement target by the illumination device 6 , thereby facilitating the extraction of visual features from the image data obtained by the visible camera 4 even when the measurement is performed in a pit of the elevator shaft or other such dark place.
  • the depth sensor 2 and the visible camera 4 are integrated with the illumination device 6 , and hence a positional relationship therebetween is constantly fixed, which can reduce a difference in appearance due to a difference in point of view. It is also possible to reduce a probability of a shadow formed due to the presence of a shielding object between the measurement target and the illumination device 6 .
  • the illumination device 6 is integrated with the depth sensor 2 and the visible camera 4 , but an illumination device installed in a measurement environment, for example, a hoistway pit, may be used.
  • FIG. 19 is a block diagram for illustrating a three-dimensional scanner according to a fourth embodiment of the present invention.
  • the scanner main body 1 according to the fourth embodiment further includes a stable position search module 18 configured to search the three-dimensional map being created for a photographing position at which the mapping is stable.
  • the mapping evaluation module 16 registers, in the storage, the evaluation result obtained for the registered sensor position/direction.
  • the output module 17 presents the photographing position obtained by the stable position search module 18 to the user.
  • the stable position search module 18 searches for a position for which the evaluation result is satisfactory from among mapping evaluation results obtained for respective sensor positions/directions, which are registered in the storage 13 .
  • a criterion for the priority of the position to be output is determined by the value of an evaluation function obtained in consideration of the magnitude of the evaluation value of the mapping and the degree of closeness to the current position.
  • the evaluation function to be used may be the same at all times, or may be changed by the user selecting a priority item depending on the situation.
  • the stable position search module 18 starts to search for the stable position when the mapping fails or when the mapping evaluation value for the current position becomes equal to or smaller than a given value. In addition to any one of those situations, the search may be started when the user designates the search. All other configurations and operations are similar or identical to the first, second, or third embodiment.
  • FIG. 20 is a flow chart for illustrating an operation of the scanner main body 1 of FIG. 19 .
  • the scanner main body 1 determines whether or not to search for the stable position (Step S 14 ) depending on the determination result as to whether or not the mapping is successful (Step S 4 ), the mapping evaluation result (Step S 11 ), and the presence or absence of the user's designation.
  • the scanner main body 1 searches for the position at which the mapping is stable (Step S 15 ), and presents the retrieved position to the user (Step S 16 ).
  • the above-mentioned three-dimensional scanner and measurement assistance processing method for the same produce such an effect that it is possible to easily recover from a state under which the mapping is unstable by returning to the photographing position at which the mapping is stable when the mapping becomes unstable due to a decrease in number of features within the sensor field of view.
  • FIG. 21 is a block diagram for illustrating a three-dimensional scanner according to a fifth embodiment of the present invention.
  • the scanner main body 1 according to the fifth embodiment further includes a sensor movement path calculation module 19 configured to calculate a sensor movement path between freely-selected positions in the three-dimensional map being created, which enables movement without interruption of the mapping.
  • the sensor movement path calculated by the sensor movement path calculation module 19 includes a start point, an end point, and a plurality of via points on the sensor movement path.
  • the initial values of the via points are set to positions sampled on a line segment connecting between the start point and the end point of the sensor movement path at regular intervals.
  • a sensor direction at each of the via points is calculated by linear interpolation from a sensor direction at the start point and a sensor direction at the end point.
  • the via points may be manually set by the user.
  • the number of via points may be increased by further sampling positions between the via points set by the user.
  • Through manual setting of the via points it is possible to calculate a more complicated sensor movement path.
  • by sampling the positions between the via points set by the user it is possible to save time and labor for the user to set a large number of via points.
  • the movement candidate determination module 14 determines the above-mentioned start point, end point, and via points as movement candidates.
  • the mapping evaluation module 16 executes the mapping evaluation for the above-mentioned movement candidates. At this time, when it is determined that any one of the start point and the end point cannot be mapped, the processing for calculating the sensor movement path is brought to an end.
  • the via points include a given via point having a position/direction from which the mapping cannot be performed, a sensor position/direction in the vicinity of the given via point is determined as a movement candidate, and the mapping evaluation is executed again. At this time, when the via point has been changed a predetermined number of times or more, the processing for calculating the sensor movement path is brought to an end.
  • the output module 17 presents a result of calculating the sensor movement path to the user.
  • the manner of displaying the sensor position/direction on the path is changed depending on whether or not the mapping can be performed therefrom.
  • different colors are used for the display so that a sensor position from which the mapping can be performed is displayed in green and a sensor position from which the mapping cannot be performed is displayed in red.
  • FIG. 22 is a flow chart for illustrating an operation of the scanner main body 1 of FIG. 21 .
  • the scanner main body 1 updates the current position and direction of the depth sensor 2 (Step S 6 ), and then determines whether or not to calculate the sensor movement path (Step S 17 ). When determining that it is required to calculate the sensor movement path, the scanner main body 1 calculates the sensor movement path (Step S 18 ).
  • the scanner main body 1 determines whether or not to evaluate the sensor movement path (Step S 19 ). When determining that it is required to evaluate the sensor movement path, the scanner main body 1 determines whether or not the mapping can be performed from all the points on the sensor movement path (Step S 20 ).
  • Step S 18 When there is a point from which the mapping cannot be performed on the sensor movement path, the procedure returns to Step S 18 .
  • the scanner main body 1 After determining that the mapping can be performed from all the points on the sensor movement path, after determining that it is not required to search for the stable position (Step S 14 ), and after the stable position is retrieved (Step S 15 ) and presented to the user (Step S 16 ), the scanner main body 1 determines whether or not to present the sensor movement path to the user (Step S 21 ).
  • the scanner main body 1 When determining that it is required to present the sensor movement path, the scanner main body 1 presents the sensor movement path to the user (Step S 22 ), and the procedure advances to Step S 12 . When it is not required to present the sensor movement path, the procedure directly advances to Step S 12 .
  • the above-mentioned three-dimensional scanner and measurement assistance processing method for the same produce such an effect that the sensor movement path calculation module 19 allows the user to follow a path presented in advance to move to a destination, and hence it is possible to reduce work time required for scanning.
  • the work time required for scanning may become longer when the scanner main body 1 moves while keeping discovering a path by trial and error so as not to interruption of the mapping before moving to a portion for which the measurement has not been performed and performing the scanning.
  • an optimum sensor movement path is presented, and hence it is possible to reduce the work time.
  • FIG. 23 is a block diagram for illustrating a three-dimensional scanner according to a sixth embodiment of the present invention.
  • the scanner main body 1 according to the sixth embodiment further includes an unmeasured area detection module 20 configured to detect an unmeasured area in the three-dimensional map being created.
  • the unmeasured area detection module 20 uses the three-dimensional map, the sensor position/direction, a sensor viewing angle, and a distance that enables the distance measurement to calculate a three-dimensional area that has been observed by the depth sensor 2 and the visible camera 4 , and detects an area that has not been observed as an unmeasured area.
  • the output module 17 presents the retrieved unmeasured area to the user.
  • a method of displaying the unmeasured area on the display 1 a there is a method of displaying the unmeasured area by the frame portion 1 d of FIG. 10 . All other configurations and operations are similar or identical to the fifth embodiment.
  • FIG. 24 is a flow chart for illustrating an operation of the scanner main body 1 of FIG. 23 .
  • the scanner main body 1 determines whether or not to present the sensor movement path (Step S 21 ), and then determines whether or not to detect an unmeasured area (Step S 23 ).
  • the scanner main body 1 detects an unmeasured area (Step S 24 ), and presents the unmeasured area to the user (Step S 25 ).
  • the above-mentioned three-dimensional scanner and measurement assistance processing method for the same produce such an effect that it is possible to easily examine which part of an object to be scanned has not been measured, which can prevent such rework as to redo the measurement found to have been skipped after the scanning.
  • FIG. 25 is a plan view for illustrating how the three-dimensional map of the inside of a pit of the elevator shaft is created through use of the three-dimensional scanner according to each of the first to sixth embodiments.
  • a pair of guide rails 31 a and 31 b for guiding the ascending and descending of a car (not shown), a buffer 32 , and the like are installed.
  • a distance from the depth sensor 2 to the measurement target tends to be short.
  • the number of features observed in the sensor field of view is relatively small, which leads to a fear that the stability of the mapping may deteriorate.
  • the evaluation value becomes more satisfactory as the number of features becomes larger. Therefore, in order to scan a narrow space so as to obtain a sensor posture with a satisfactory evaluation value, the user is guided to perform the measurement along a circle while maintaining a posture for measuring an opposite hoistway wall across the center of the space.
  • FIG. 26 is a perspective view for illustrating a state under which the three-dimensional scanner according to each of the first to sixth embodiments is mounted to a mobile robot 33 , which is of an autonomous traveling type and a wheeled type. In this manner, the three-dimensional scanner is mounted to the mobile robot 33 , to thereby be able to generate the three-dimensional map of the surrounding environment of the mobile robot 33 .
  • the robot In the generation of a three-dimensional map through use of the mobile robot 33 , the robot is required to be able to constantly recognize where the robot itself is positioned in the three-dimensional map (self position). However, when the mapping fails, the self position becomes unclear, which leads to a failure of the system. against this backdrop, the three-dimensional scanner according to each of the first to sixth embodiments performs the mapping evaluation in advance before the subsequent movement is started, which allows the mobile robot 33 to select a path that prevents the mapping from failing.
  • the functions of the scanner main body 1 may be separately executed by two or more computers or electronic circuits.
  • the present invention can also be applied to three-dimensional measurement other than that of the elevator shaft.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Electromagnetism (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Length Measuring Devices With Unspecified Measuring Means (AREA)
US16/070,025 2016-01-20 2016-01-20 Three-dimensional scanner and measurement assistance processing method for same Active US10393515B2 (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2016/051598 WO2017126060A1 (fr) 2016-01-20 2016-01-20 Dispositif de mesure tridimensionnelle et son procédé de traitement d'aide à la mesure

Publications (2)

Publication Number Publication Date
US20190049240A1 US20190049240A1 (en) 2019-02-14
US10393515B2 true US10393515B2 (en) 2019-08-27

Family

ID=59362170

Family Applications (1)

Application Number Title Priority Date Filing Date
US16/070,025 Active US10393515B2 (en) 2016-01-20 2016-01-20 Three-dimensional scanner and measurement assistance processing method for same

Country Status (7)

Country Link
US (1) US10393515B2 (fr)
JP (1) JP6489566B2 (fr)
KR (1) KR101973917B1 (fr)
CN (1) CN108474653B (fr)
DE (1) DE112016006262B4 (fr)
HK (1) HK1254590A1 (fr)
WO (1) WO2017126060A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11788845B2 (en) * 2018-06-29 2023-10-17 Baidu Usa Llc Systems and methods for robust self-relocalization in a visual map

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10701343B2 (en) * 2016-05-26 2020-06-30 Asustek Computer Inc. Measurement device and processor configured to execute measurement method
US11423005B2 (en) * 2017-04-03 2022-08-23 Mitsubishi Electric Corporation Map data generator and method for generating map data
KR101997109B1 (ko) * 2017-11-21 2019-07-05 인천대학교 산학협력단 루트 정보를 이용한 이동 이력 데이터 저장 장치 및 방법
EP3671115B1 (fr) * 2018-12-17 2023-10-11 Leica Geosystems AG Système de délimitation géodésique
KR102140594B1 (ko) * 2019-04-09 2020-08-03 한국과학기술연구원 프리미티브 기반의 3차원 자동 스캔 방법 및 시스템
JP7146271B2 (ja) * 2019-05-14 2022-10-04 ジオ・サーチ株式会社 埋設物計測装置、方法、及びプログラム
US20220155448A1 (en) * 2019-05-16 2022-05-19 Mitsubishi Electric Corporation Information processing device, information processing method, and storage medium
KR102242744B1 (ko) * 2019-06-14 2021-04-20 고려대학교 산학협력단 이동 로봇과 이동 로봇의 엘리베이터 인식 방법
US20220107414A1 (en) * 2020-10-07 2022-04-07 Luminar, Llc Velocity determination with a scanned lidar system
KR102228939B1 (ko) * 2021-02-05 2021-03-17 한국과학기술연구원 3차원 자동 스캔 시스템 및 방법
WO2023238465A1 (fr) * 2022-06-09 2023-12-14 日産自動車株式会社 Procédé et dispositif d'inspection de composant

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009155056A (ja) 2007-12-27 2009-07-16 Seiko Precision Inc 画像処理装置、荷物搬送設備、及びプログラム
JP5310285B2 (ja) 2009-06-12 2013-10-09 日産自動車株式会社 自己位置推定装置及び自己位置推定方法
JP2015515655A (ja) 2012-06-29 2015-05-28 三菱電機株式会社 データを位置合わせする方法
US20170140539A1 (en) * 2015-11-16 2017-05-18 Abb Technology Ag Three-dimensional visual servoing for robot positioning

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS5310285B1 (fr) 1969-09-16 1978-04-12
JPS5310285A (en) 1976-07-15 1978-01-30 Mitsubishi Electric Corp Reverse conducting thyristor
DE4206608A1 (de) * 1992-03-03 1993-09-09 Optoelektronische Inspektions Vorrichtung zur innenvermessung von rohren
JP2964402B1 (ja) * 1998-08-28 1999-10-18 株式会社ゼンリン 三次元地図データベースの作成方法及び装置
JP2007010419A (ja) * 2005-06-29 2007-01-18 Komatsu Engineering Corp 対象物の３次元形状検証システム。
JP2007316966A (ja) * 2006-05-26 2007-12-06 Fujitsu Ltd 移動ロボット、その制御方法及びプログラム
JP4800163B2 (ja) * 2006-09-29 2011-10-26 株式会社トプコン 位置測定装置及びその方法
JP4608533B2 (ja) 2007-10-16 2011-01-12 日信工業株式会社 車両用ディスクブレーキ
CN101520319B (zh) * 2008-02-27 2012-02-22 邹小平 复合式三维激光测量系统及测量方法
WO2010007960A1 (fr) * 2008-07-14 2010-01-21 クラリオン株式会社 Système d'image vidéo de conversion de point de vue pour une caméra montée sur un véhicule et procédé d'acquisition d'image vidéo de conversion de point de vue
JP2012015674A (ja) * 2010-06-30 2012-01-19 Fujifilm Corp 撮像装置ならびにその動作制御方法およびそのプログラム
JP5762131B2 (ja) * 2011-05-23 2015-08-12 三菱電機株式会社 キャリブレーション装置、キャリブレーション装置のキャリブレーション方法およびキャリブレーションプログラム
JP6355710B2 (ja) 2013-03-15 2018-07-11 ファロ テクノロジーズ インコーポレーテッド 非接触型光学三次元測定装置
CN104515478B (zh) * 2014-12-11 2017-08-01 华中科技大学 一种高精度的航空发动机叶片自动三维测量方法和系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009155056A (ja) 2007-12-27 2009-07-16 Seiko Precision Inc 画像処理装置、荷物搬送設備、及びプログラム
JP5310285B2 (ja) 2009-06-12 2013-10-09 日産自動車株式会社 自己位置推定装置及び自己位置推定方法
JP2015515655A (ja) 2012-06-29 2015-05-28 三菱電機株式会社 データを位置合わせする方法
US9183631B2 (en) 2012-06-29 2015-11-10 Mitsubishi Electric Research Laboratories, Inc. Method for registering points and planes of 3D data in multiple coordinate systems
US20170140539A1 (en) * 2015-11-16 2017-05-18 Abb Technology Ag Three-dimensional visual servoing for robot positioning

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
International Search Report dated Apr. 5, 2016 in PCT/JP2016/051598 filed Jan. 20, 2016.
Taguchi, Y. et al., "Point-Plane SLAM for Hand-Held 3D Sensors," IEEE International Conference on Robotics and Automation (ICRA), Mitsubishi Electric Research Laboratories, May 2013, 9 Pages.

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11788845B2 (en) * 2018-06-29 2023-10-17 Baidu Usa Llc Systems and methods for robust self-relocalization in a visual map

Also Published As

Publication number Publication date
HK1254590A1 (zh) 2019-07-26
JPWO2017126060A1 (ja) 2018-07-05
JP6489566B2 (ja) 2019-03-27
DE112016006262B4 (de) 2023-05-04
WO2017126060A1 (fr) 2017-07-27
KR101973917B1 (ko) 2019-04-29
CN108474653B (zh) 2020-03-20
US20190049240A1 (en) 2019-02-14
KR20180090353A (ko) 2018-08-10
DE112016006262T5 (de) 2018-10-04
CN108474653A (zh) 2018-08-31

Similar Documents

Publication Publication Date Title
US10393515B2 (en) Three-dimensional scanner and measurement assistance processing method for same
CN108527362B (zh) 机器人设置设备、机器人设置方法和计算机可读记录介质
JP5538667B2 (ja) 位置姿勢計測装置及びその制御方法
JP5248806B2 (ja) 情報処理装置、情報処理方法
JP5799273B2 (ja) 寸法計測装置、寸法計測方法、寸法計測システム、プログラム
JP5109294B2 (ja) ３次元位置補正装置
JP2018144164A (ja) ロボット設定装置、ロボット設定方法、ロボット設定プログラム及びコンピュータで読み取り可能な記録媒体並びに記録した機器
EP2551633B1 (fr) Procédé et dispositif de mesure de distance en trois dimensions
US20170109932A1 (en) Content projection apparatus, content projection method, and computer readable storage medium
JP2018144166A (ja) 画像処理装置、画像処理方法、画像処理プログラム及びコンピュータで読み取り可能な記録媒体並びに記録した機器
US9595095B2 (en) Robot system
JP2018048839A (ja) 三次元データ生成装置及び三次元データ生成方法、並びに三次元データ生成装置を備えた監視システム
JP2008309595A (ja) オブジェクト認識装置及びそれに用いられるプログラム
JP2017049035A (ja) 画像測定装置及びその制御プログラム
JP2018144162A (ja) ロボット設定装置、ロボット設定方法、ロボット設定プログラム及びコンピュータで読み取り可能な記録媒体並びに記録した機器
Gallegos et al. Appearance-based slam relying on a hybrid laser/omnidirectional sensor
CN114726978A (zh) 信息处理装置、信息处理方法以及程序
JP6132246B2 (ja) 寸法計測方法
JP2020071739A (ja) 画像処理装置
Comport et al. An asymmetric real-time dense visual localisation and mapping system
JP4002211B2 (ja) 現場作業支援装置
CN114930388A (zh) 工件图像解析装置、工件图像解析方法以及程序
JPWO2021111613A1 (ja) ３次元地図作成装置、３次元地図作成方法、及び３次元地図作成プログラム
WO2020150870A1 (fr) Procédé et appareil de surveillance de système robotisé
JP6546898B2 (ja) ３次元空間特定装置、方法、及びプログラム

Legal Events

Date Code Title Description
AS Assignment

Owner name: MITSUBISHI ELECTRIC CORPORATION, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KAWANISHI, RYOSUKE;SEKI, MAKITO;SAMBONGI, KEISUKE;AND OTHERS;SIGNING DATES FROM 20180418 TO 20180423;REEL/FRAME:046343/0757

FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4