WO2023162730A1 - 情報処理装置、情報処理方法及びプログラム - Google Patents

情報処理装置、情報処理方法及びプログラム Download PDF

Info

Publication number
WO2023162730A1
WO2023162730A1 PCT/JP2023/004645 JP2023004645W WO2023162730A1 WO 2023162730 A1 WO2023162730 A1 WO 2023162730A1 JP 2023004645 W JP2023004645 W JP 2023004645W WO 2023162730 A1 WO2023162730 A1 WO 2023162730A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
point cloud
cloud data
control unit
imaging
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2023/004645
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
晃一 星野
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Kyocera Corp
Original Assignee
Kyocera Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kyocera Corp filed Critical Kyocera Corp
Priority to JP2024503024A priority Critical patent/JPWO2023162730A1/ja
Priority to EP23759734.9A priority patent/EP4485007A4/en
Priority to CN202380023654.3A priority patent/CN118786361A/zh
Priority to US18/839,622 priority patent/US20250157079A1/en
Publication of WO2023162730A1 publication Critical patent/WO2023162730A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/86—Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G01S17/42—Simultaneous measurement of distance and other co-ordinates
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/481—Constructional features, e.g. arrangements of optical elements
    • G01S7/4816—Constructional features, e.g. arrangements of optical elements of receivers alone
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
    • G06T3/147—Transformations for image registration, e.g. adjusting or mapping for alignment of images using affine transformations
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing

Definitions

  • the present disclosure relates to an information processing device, an information processing method, and a program.
  • An information processing apparatus includes an image acquisition unit, a point cloud data acquisition unit, and a control unit.
  • the image acquisition unit acquires an image from an imaging device.
  • the point cloud data acquisition unit acquires point cloud data representing a distance distribution from a distance measuring device.
  • the control unit obtains each of the first image and the second image based on the first image obtained from the image obtaining unit and the second image including the same subject as the first image. Acquire information indicating the correspondence between locations.
  • the control unit obtains two pieces of point cloud data obtained from the point cloud data obtaining unit, the first point cloud data in which the imaging range of the first image and the measurement range of the distance distribution at least partially overlap. and second point cloud data in which the imaging range of the second image and the measurement range of the distance distribution at least partially overlap, based on the information indicating the correspondence relationship.
  • An information processing method acquires a first image and a second image including the same subject as the first image, and two point cloud data representing a distance distribution.
  • first point cloud data in which the imaging range of the first image and the measurement range of the distance distribution at least partially overlap; and the imaging range of the second image and the measurement range of the distance distribution are at least partially overlapped. and acquiring information indicating the correspondence relationship between the positions of the first image and the second image based on the first image and the second image. and associating the first point cloud data with the second point cloud data based on the information indicating the correspondence relationship.
  • a program acquires a first image and a second image including the same subject as the first image, and two point cloud data indicating a distance distribution, The imaging range of the first image and the measurement range of the distance distribution at least partially overlap the first point cloud data, and the imaging range of the second image and the measurement range of the distance distribution at least partially overlap.
  • FIG. 1 is a configuration diagram showing a basic configuration of a posture estimation system including an information processing device according to an embodiment of the present disclosure.
  • FIG. 2 is a configuration diagram showing a more detailed example of the posture estimation system of FIG.
  • FIG. 3 is a block diagram showing a schematic configuration of the information processing apparatus of FIG. 1.
  • FIG. 4 is a flowchart showing processing executed by the information processing apparatus of FIG.
  • FIG. 5 is a diagram illustrating an example of an image acquired by the image acquiring unit in FIG. 3;
  • FIG. 6 is a diagram showing an example of a distance image based on point cloud data acquired by the point cloud data acquiring unit in FIG. 3 .
  • FIG. 7 is a diagram showing an example of image composition by associating the first image and the second image.
  • FIG. 1 is a configuration diagram showing a basic configuration of a posture estimation system including an information processing device according to an embodiment of the present disclosure.
  • FIG. 2 is a configuration diagram showing a more detailed example of the posture estimation system of FIG
  • FIG. 8 is a diagram illustrating a method of using homography transformation to associate the first image and the second image.
  • FIG. 9A is a diagram showing an example of the first point cloud data after dilation and distortion correction.
  • FIG. 9B is a diagram showing an example of the second point cloud data after dilation and distortion correction.
  • FIG. 10A is a diagram showing a first three-dimensional spatial point cloud created from the first point cloud data.
  • FIG. 10B is a diagram showing a second three-dimensional spatial point cloud created from the second point cloud data.
  • FIG. 11 is a diagram showing an example of an environment map generated using the information processing apparatus according to this embodiment.
  • FIG. 12 is a diagram showing an example of an environmental map generated using the same image and point cloud data as in FIG. 11 by a conventional method that can use feature points.
  • posture estimation means estimating the amount of rotation and the amount of movement from data of different viewpoints.
  • the point cloud data showing the distribution of distances to each measurement position obtained from the ranging device is composed of point clouds from different viewpoints. It may be difficult to make correspondence between data. As a result, sufficient posture estimation accuracy cannot be obtained, and the environment map accuracy cannot be improved in some cases.
  • the ICP Interactive Closest Point
  • the ICP Interactive Closest Point
  • each point forming one point group is searched for the closest point in the other point group, and this is taken as a corresponding point.
  • the method estimates, for each point thus determined and its corresponding points, a rigid transformation that minimizes the distance between the corresponding points.
  • Rigid body transformations are represented by rotations and translations.
  • Pose estimation can be done by estimating rigid transformations.
  • this method since the corresponding points cannot be determined correctly, there are cases where posture estimation cannot be performed or estimation errors increase.
  • each point of the two point cloud data of the distance measuring device using two images corresponding to the respective point clouds of the imaging device whose imaging range is the same as the measurement range of the distance measuring device. be done. That is, it is possible to extract feature points corresponding to each other between two images, extract points located at the same positions as the feature points on the images of the two point cloud data corresponding to each image, and set them as corresponding points. can. By transforming a set of a plurality of corresponding points obtained in this way into a point group in a three-dimensional space, posture estimation becomes possible. However, in this method, the points corresponding to the feature points on the image are often not included in the point cloud data obtained from the distance measuring device.
  • the ranging device is not suitable for edge and corner ranging. Therefore, the points that can be associated with the point cloud data may be limited. Therefore, sufficient corresponding points cannot be obtained, and the accuracy of posture estimation may be lowered.
  • Embodiments of the present disclosure improve pose estimation methods as described above by facilitating correspondence between point cloud data.
  • embodiments of the present disclosure will be described with reference to the drawings.
  • the figures used in the following description are schematic.
  • the dimensions, ratios, etc. on the drawings do not necessarily match the actual ones.
  • a posture estimation system 10 is mounted on a mobile object.
  • the posture estimation system 10 includes a distance measuring device 11, an imaging device 12, and an information processing device 13, as shown in FIG.
  • the relative positions and orientations of the distance measuring device 11 and the imaging device 12 are fixed.
  • the information processing device 13 may be configured to communicate with the distance measuring device 11 and imaging device 12 that are not mounted on a moving object but are mounted on a moving object.
  • the rangefinder 11 is a scanning rangefinder such as a laser radar.
  • a laser radar measures the distance to a subject by irradiating a measurement range with laser light and measuring the time it takes for the reflected laser light to return.
  • Laser radar is also called LiDAR (Light Detection And Ranging).
  • the distance measuring device 11 outputs point cloud data indicating the distance distribution of the measurement range.
  • the distance distribution is the distribution of distances from the distance measuring device 11 to each measurement position.
  • the point cloud data includes distance information detected by the distance measuring device 11 scanning the entire measurement range.
  • the point cloud data includes, for example, one frame of distance data from the distance measuring device 11 .
  • Each point of the point cloud data includes position information when the object to be measured is viewed two-dimensionally, and information on the distance from the distance measuring device 11 corresponding to the position information to the measurement position.
  • the two-dimensional position information corresponds to the direction from the distance measuring device 11 toward each measurement position in real space.
  • the imaging device 12 has an imaging optical system and an imaging element, and acquires an image of the imaging range.
  • the imaging range of the imaging device 12 at least partially overlaps the measurement range of the distance measuring device 11 .
  • the distance measuring device 11 and the imaging device 12 can be configured so that the coordinate systems for observing the target are substantially the same or close to each other.
  • the distance measuring device 11 and the imaging device 12 may be configured such that the optical axis ax of the optical system 14 for detecting or imaging a subject is aligned. In this case, the distance measuring device 11 and the imaging device 12 have the same optical axis and a fixed positional relationship. The upper position can be associated in advance.
  • the distance measuring device 11 and the imaging device 12 may be arranged so that the optical axes of the optical systems for measuring or imaging the subject are parallel and close to each other. In this case, the optical system 14 for matching the mutual optical axes may be omitted. Between the point cloud data and the image to be detected by the distance measuring device 11 and the imaging device 12, there is no parallax or the parallax is small. The point cloud data output from the distance measuring device 11 and the image output from the imaging device 12 can be superimposed so that the positions of the same frames correspond to each other.
  • the distance measuring device 11 and the imaging device 12 may be fixed with respect to the moving body so that their relative positions and orientations are fixed.
  • Mobile objects include, for example, automobiles, robots, humans, handcarts, wheelchairs, home electric appliances such as self-propelled vacuum cleaners, unmanned aircraft such as drones, toys with mobile functions, and articles attached to these including any moving object such as
  • the information processing device 13 is a computer.
  • the information processing device 13 may be any of various computers including general-purpose computers, workstations, and PCs (Personal Computers).
  • the information processing device 13 may be a dedicated computer that performs specific processing for constructing the environment map and estimating the self-location.
  • the information processing device 13 may read a dedicated program and data and execute each function of the information processing device 13 described below.
  • the posture estimation system 10 includes a detection device 21 that includes the functions of the ranging device 11 and the imaging device 12 and an information processing device 13 .
  • the detection device 21 includes an irradiation unit 22 , a reflection unit 23 , a control unit 24 , a first optical system 25 and a detection element 26 as components corresponding to the distance measurement device 11 .
  • the detection device 21 also includes a control unit 24 , a second optical system 27 , and an imaging device 28 as components corresponding to the imaging device 12 .
  • the detection device 21 further includes a switching section 32 .
  • the irradiation unit 22 radiates at least one electromagnetic wave of infrared rays, visible rays, ultraviolet rays, and radio waves. In one embodiment, the irradiation section 22 emits infrared rays.
  • the irradiating unit 22 irradiates the radiated electromagnetic wave toward the object ob, which is a subject, directly or indirectly via the reflecting unit 23 . In the embodiment illustrated in FIG. 2 , the irradiation unit 22 indirectly irradiates the radiated electromagnetic wave toward the object ob through the reflection unit 23 .
  • the object ob includes all objects located within the measurement range of the distance measuring device 11 .
  • the irradiation unit 22 radiates a beam-shaped electromagnetic wave with a narrow width, for example, 0.5°.
  • the irradiation unit 22 can emit electromagnetic waves in pulses.
  • the irradiation unit 22 includes LEDs (Light Emitting Diodes) and LDs (Laser Diodes). The irradiation unit 22 switches between electromagnetic wave radiation and stop based on the control of the control unit 24, which will be described later.
  • the reflecting unit 23 changes the irradiation position of the electromagnetic wave irradiated to the object ob by reflecting the electromagnetic wave emitted from the irradiation unit 22 while changing the direction.
  • the irradiation position is equal to the measurement position where the distance is measured by the distance measuring device 11 .
  • the reflecting unit 23 scans the measurement range including the object ob with the electromagnetic waves emitted from the irradiating unit 22 .
  • the reflecting unit 23 can scan the irradiation position of the electromagnetic waves in two-dimensional directions.
  • the reflecting unit 23 can perform regular scanning by sequentially shifting the irradiation position detected by the distance measuring device 11 in the vertical direction while scanning in the horizontal direction.
  • Reflector 23 may scan the measurement range in any other manner.
  • the reflecting section 23 is configured such that at least part of the irradiation area of the electromagnetic wave emitted from the irradiating section 22 and reflected is included in the detection range of the detecting element 26 . Therefore, at least a part of the electromagnetic waves irradiated to the object ob through the reflector 23 can be detected by the detection element 26 .
  • the reflecting section 23 includes, for example, a MEMS (Micro Electro Mechanical Systems) mirror, a polygon mirror, a galvanomirror, and the like.
  • MEMS Micro Electro Mechanical Systems
  • the reflecting section 23 changes the direction in which the electromagnetic waves are reflected under the control of the control section 24, which will be described later.
  • the reflection unit 23 may have an angle sensor such as an encoder, and may notify the control unit 24 of the angle detected by the angle sensor as direction information for reflecting the electromagnetic waves.
  • the controller 24 can calculate the irradiation position based on the direction information acquired from the reflector 23 .
  • the control unit 24 can calculate the irradiation position based on the drive signal input to the reflecting unit 23 to change the direction in which the electromagnetic waves are reflected.
  • the reflection unit 23 is configured such that the central axis of the direction in which the electromagnetic wave is scanned is the imaging device. It can be arranged substantially parallel and close to the twelve optical axes.
  • the control unit 24 includes one or more processors and memory.
  • the processor may include at least one of a general-purpose processor that loads a specific program and executes a specific function, and a dedicated processor that specializes in specific processing.
  • a dedicated processor may include an Application Specific Integrated Circuit (ASIC).
  • the processor may include a programmable logic device (PLD).
  • the PLD may include an FPGA (Field-Programmable Gate Array).
  • the control unit 24 may include at least one of SoC (System-on-a-Chip) and SiP (System In a Package) in which one or more processors cooperate.
  • the control unit 24 is configured to be able to control the reflection unit 23 and the switching unit 32 .
  • the control unit 24 can control the switching unit 32 so that the detecting element 26 can acquire the reflected electromagnetic wave according to the irradiation position and the irradiation time of the electromagnetic wave by the reflecting unit 23 .
  • the control unit 24 can acquire detection information from the detection element 26 and calculate the distance.
  • the control unit 24 can generate point cloud data from each measurement position and the distance at the measurement position.
  • the control unit 24 can acquire an image signal from the imaging device 28 .
  • the control unit 24 can output point cloud data and image signals to the information processing device 13 .
  • the first optical system 25 is irradiated from the irradiation unit 22 and reflected by the reflection unit 23, so that the reflected wave from the object ob of the electromagnetic wave irradiated toward the measurement range is detected by the detection element 26. proceed as follows. As will be described later, the optical axis of the first optical system 25 may be configured to match the optical axis of the second optical system 27 on the object ob side.
  • the detection element 26 includes an element capable of detecting electromagnetic waves emitted from the irradiation section 22 .
  • the detection element 26 includes a single element such as an APD (Avalanche PhotoDiode) and a PD (PhotoDiode).
  • Detector elements 26 may include element arrays such as APD arrays, PD arrays, ranging imaging arrays, and ranging image sensors.
  • the detection element 26 transmits detection information indicating detection of the reflected wave from the subject to the control section 24 as a signal.
  • the detection element 26 detects, for example, electromagnetic waves in the infrared band.
  • the detection element 26 only needs to be able to detect electromagnetic waves in a configuration that is a single element that constitutes the distance measuring sensor described above, and it is not necessary for the object ob to be imaged on the detection surface. Therefore, the detection element 26 does not have to be provided at the secondary image forming position, which is the image forming position by the first post-stage optical system 30, which will be described later. That is, in this configuration, if the detecting element 26 is at a position where electromagnetic waves from all angles of view can be incident on the detecting surface, the detecting element 26 travels in the first direction d1 by the switching unit 32, and then moves toward the first post-optical system. It may be placed anywhere on the path of the electromagnetic wave traveling through 30 .
  • the control unit 24 acquires the distance based on the electromagnetic waves detected by the detection element 26. Based on the detection information detected by the detection element 26, the control unit 24 acquires the distance of the irradiation position irradiated by the irradiation unit 22 by a ToF (Time-of-Flight) method as described below.
  • ToF Time-of-Flight
  • the control unit 24 By inputting an electromagnetic wave radiation signal to the irradiation unit 22, the control unit 24 causes the irradiation unit 22 to radiate pulsed electromagnetic waves.
  • the irradiation unit 22 irradiates an electromagnetic wave based on the input electromagnetic wave radiation signal.
  • the electromagnetic waves radiated by the irradiation unit 22 and reflected by the reflection unit 23 to irradiate an arbitrary irradiation area are reflected in the irradiation area.
  • the detection element 26 detects the electromagnetic waves reflected in the irradiation area, the detection element 26 notifies the control unit 24 of detection information.
  • the control unit 24 measures the time from the time when the irradiation unit 22 emits electromagnetic waves to the time when the detection information is acquired.
  • the control unit 24 multiplies the time by the speed of light and divides by 2 to calculate the distance to the irradiation position.
  • the control unit 24 calculates the irradiation position based on the direction information acquired from the reflection unit 23 or the drive signal output to the reflection unit 23 by itself, as described above. do.
  • the control unit 24 can calculate the irradiation position based on the position at which the electromagnetic wave reflected by the object ob is detected on the element array.
  • the irradiation position is the measurement position of the distance measuring device 11 as described above.
  • the control unit 24 calculates the distance to each irradiation position while changing the irradiation position, thereby creating point cloud data including information on a plurality of measurement positions and distances.
  • the second optical system 27 forms an image of the object ob in a range overlapping the measurement range of the distance measuring device 11 on the imaging surface of the imaging device 28 .
  • the imaging device 28 converts the image formed on the imaging surface into an electrical signal to generate an image of the imaging range including the object ob.
  • the imaging device 28 may include either a CCD image sensor (Charge-Coupled Device Image Sensor) or a CMOS image sensor (Complementary MOS Image Sensor).
  • the imaging device 28 outputs the generated image to the control unit 24.
  • the control unit 24 may perform arbitrary processing such as distortion correction, brightness adjustment, contrast adjustment, gamma correction, etc. on the image.
  • the control unit 24 may correct the distortion. If there is a deviation between the optical system that detects the point cloud data and the optical system that captures the image, the control unit 24 controls at least one of the point cloud data and the image so as to reduce the influence of the deviation. can be corrected. Such point cloud data and image correction may be performed by the information processing device 13 instead of the control unit 24 .
  • the first optical system 25 includes a pre-stage optical system 29 common to the second optical system 27 and a first post-stage optical system 30 positioned downstream of the switching section 32 .
  • the second optical system 27 includes a pre-stage optical system 29 that is common to the first optical system 25 and a second post-stage optical system 31 positioned after the switching section 32 .
  • the pre-stage optical system 29 includes, for example, at least one of a lens and a mirror, and forms an image of an object ob, which is a subject.
  • the switching unit 32 is provided at or near the primary imaging position, which is the imaging position of the image of the object ob, which is separated from the preceding optical system 29 by the preceding optical system 29, by the preceding optical system 29. It is good if there is
  • the switching unit 32 has an action surface as on which the electromagnetic wave that has passed through the pre-stage optical system 29 is incident.
  • the active surface as is composed of a plurality of pixels px arranged two-dimensionally.
  • the action surface as is a surface that causes an electromagnetic wave to have an action such as reflection or transmission in at least one of a first state and a second state described later.
  • the switching unit 32 can switch for each pixel px between a first state in which an electromagnetic wave incident on the action surface as travels in a first direction d1 and a second state in which it travels in a second direction d2.
  • a first state is a first reflection state in which an electromagnetic wave incident on the working surface as is reflected in a first direction d1.
  • the second state is a second reflection state in which the electromagnetic wave incident on the action surface as is reflected in the second direction d2.
  • the switching unit 32 includes a reflecting surface that reflects electromagnetic waves for each pixel px.
  • the switching unit 32 switches between the first reflection state and the second reflection state for each pixel px by changing the orientation of the reflection surface for each pixel px.
  • the switching unit 32 includes, for example, a DMD (Digital Micro mirror Device).
  • the DMD drives a minute reflecting surface that constitutes the working surface as so that the reflecting surface is tilted at a predetermined angle, for example +12° or ⁇ 12°, with respect to the working surface as for each pixel px. can be switched to The active surface as is parallel to the surface of the substrate on which the minute reflecting surface of the DMD is placed.
  • the switching unit 32 switches between the first state and the second state for each pixel px under the control of the control unit 24 .
  • the switching unit 32 can simultaneously switch some of the pixels px1 to the first state so that the electromagnetic waves incident on the pixels px1 travel in the first direction d1, and switch some of the pixels px2 to the first state.
  • the electromagnetic waves incident on the pixel px2 can be caused to travel in the second direction d2.
  • the switching unit 32 switches the same pixel px from the first state to the second state, thereby directing the electromagnetic waves incident on the pixel px in the first direction d1 and then in the second direction d2. can proceed.
  • the first post-stage optical system 30 is provided from the switching section 32 in the first direction d1.
  • the first post-stage optical system 30 includes, for example, at least one of a lens and a mirror.
  • the first post-stage optical system 30 causes the electromagnetic wave whose traveling direction is switched by the switching unit 32 to enter the detection element 26 .
  • the second post-stage optical system 31 is provided from the switching section 32 in the second direction d2.
  • the second post-stage optical system 31 includes, for example, at least one of a lens and a mirror.
  • the second post-stage optical system 31 forms an image of the target ob as an electromagnetic wave whose traveling direction is switched by the switching unit 32 on the imaging surface of the imaging device 28 .
  • the detection device 21 aligns the optical axis ax of the front optical system 29 with the optical axis of the first rear optical system 30 in the first state, It is possible to match the optical axis of the optical system 31 . Therefore, by switching the pixel px of the switching unit 32 between the first state and the second state, the detection device 21 detects the measurement position where the detection element 26 in the case of an element array detects the distance and the image pickup element. 28 can reduce the parallax shift with the image captured.
  • the control unit 24 can switch some of the pixels px in the switching unit 32 to the first state and switch some other pixels px to the second state as the reflecting unit 23 moves the irradiation position. Therefore, the detection device 21 can cause the detection element 26 to detect electromagnetic waves in some pixels px and simultaneously cause the imaging element 28 to detect images in another part of pixels px. As a result, the detection device 21 can substantially simultaneously acquire an image of a portion excluding the distance of the irradiation position and the vicinity of the irradiation position of the electromagnetic waves in the same field of view.
  • an optical system including the switching unit 32 is used in order to make the optical axis of the distance measuring device 11 for measuring the distance and the optical axis of the imaging device 12 match or approach each other.
  • the method for making the optical axes of the distance measuring device 11 and the imaging device 12 coincide or approach each other is not limited to this.
  • the optical system 14 in FIG. 1 can be configured using the difference in wavelength between the light detected by the distance measuring device 11 and the image captured by the imaging device 12 .
  • the optical system 14 can be configured to match the optical axis of the optical system of the distance measuring device 11 and the optical axis of the imaging device 12 on the object ob side using a dichroic mirror, a dichroic prism, or the like.
  • the information processing apparatus 13 includes a point cloud data acquisition unit 41, an image acquisition unit 42, a control unit 43, a storage unit 44, and an output unit 45, as shown in FIG.
  • the point cloud data acquisition unit 41 acquires point cloud data representing the distance distribution from the distance measuring device 11 .
  • the point cloud data acquisition unit 41 may include a communication module that communicates with the distance measuring device 11 .
  • the point cloud data acquisition unit 41 acquires the point cloud data from the control unit 24 forming part of the distance measuring device 11 .
  • the point cloud data includes at least two-dimensional position information indicating the measurement position when the measurement range is viewed as a secondary plane, and the measured value of the distance to the object ob positioned at that position.
  • the image acquisition unit 42 acquires images from the imaging device 12 .
  • the image acquisition unit 42 may include a communication module that communicates with the imaging device 12 .
  • the image acquisition section 42 acquires an image from the control section 24 forming part of the imaging device 12 .
  • the time when the distance measuring device 11 acquires the point cloud data and the time when the imaging device 12 acquires the image may be synchronized.
  • the control unit 43 like the control unit 24 of the detection device 21, includes one or more processors and memory. Like the control unit 24 of the detection device 21, the control unit 43 includes at least one of a general-purpose processor that loads a specific program and executes a specific function, and a dedicated processor that specializes in specific processing. OK.
  • the control unit 43 can acquire point cloud data and images in which the measurement range and the imaging range when viewed as a two-dimensional plane at least partially overlap.
  • the control unit 43 can acquire the point cloud data and the image that are measured and captured substantially at the same time as a set of data.
  • Substantially the same time includes the time when the distance measuring device 11 measures one frame of point cloud data and the time when the imaging device 12 captures an image.
  • the distance measurement device 11 and the imaging device 12 perform distance measurement and imaging based on the same timing signal.
  • the control unit 43 can sequentially acquire two sets of point cloud data and images.
  • the control unit 43 sets the first set of point cloud data and the image as the first point cloud data and the first image, and sets the second set of the point cloud data and the image as the second point cloud data and the second image. It can be an image.
  • the control unit 43 calculates information indicating the correspondence between the positions of the first image and the second image, and generates the first point cloud data and the second point cloud data based on the information indicating this correspondence. can be mapped with
  • the correspondence between the first point cloud data and the second point cloud data is performed by associating each point included in the first point cloud data with each point data included in the second point cloud data. means.
  • each point of the first point cloud data and the second point cloud data that are associated is the first image and It is not limited to the points located at the feature points of the second image.
  • the posture estimation system 10 is configured such that the control unit 43 calculates information indicating the correspondence between the positions of the first image and the second image. may be configured such that the information is calculated by an information processing device on a network such as a server or cloud, and the information calculated by the information processing device is acquired by the control unit 43 .
  • the control unit 43 can estimate the amount of rotation and the amount of movement of the distance measuring device 11 and the imaging device 12 based on the correspondence between the first point cloud data and the second point cloud data. That is, the control unit 43 can estimate the posture of the moving object.
  • the control unit 43 can construct a map of the surrounding environment and estimate the self-position within the environmental map from the sequentially acquired point cloud data and the estimated amount of rotation and amount of movement.
  • the storage unit 44 includes semiconductor memory and/or magnetic memory.
  • the storage unit 44 may function, for example, as a main memory device, an auxiliary memory device, or a cache memory.
  • the storage unit 44 stores arbitrary information used for the operation of the information processing device 13 .
  • the storage unit 44 may store system programs, application programs, management databases, and the like.
  • the storage unit 44 can temporarily store the point cloud data acquired by the point cloud data acquisition unit 41, the image acquired by the image acquisition unit 42, and information obtained by processing these data.
  • the storage unit 44 may sequentially accumulate and store information on the coordinates of the point cloud data located in the surrounding environment and the amount of rotation and amount of movement.
  • the storage unit 44 may store environment maps sequentially constructed by the control unit 43 .
  • the output unit 45 is configured to be able to output information from the information processing device 13 to the outside.
  • the output unit 45 includes one or more of a display, a communication module that transmits information to an external computer, a device that outputs information to a storage medium, and the like.
  • the output unit 45 may output the constructed environment map and the estimated self-location information to another system of the mobile body on which the information processing device 13 is mounted, via an electric line.
  • the detection device 21 and the information processing device 13 are separated.
  • the information processing device 13 may be incorporated into the same hardware as the detection device 21 of FIG.
  • the control unit 43 of the information processing device 13 may be shared with the control unit 24 of the detection device 21 .
  • Posture estimation processing executed by the control unit 43 will be described below with reference to the flowchart of FIG.
  • the information processing device 13 may be configured to read a program recorded on a non-temporary computer-readable medium and implement the processing performed by the control unit 43, which will be described below.
  • Non-transitory computer-readable media include, but are not limited to, magnetic storage media, optical storage media, magneto-optical storage media, and semiconductor storage media.
  • the control unit 43 acquires the first image and the first point cloud data via the image acquisition unit 42 and the point cloud data acquisition unit 41 (step S101).
  • the first image and the first point cloud data have the same or close viewpoints for imaging and measurement. Note that when the measurement range of the distance measuring device 11 and the imaging range of the imaging device 12 only partially overlap, the first image and the first point cloud data for which the following processes are performed are obtained from the overlapping portion. shall be The same applies to the second image and the second point cloud data.
  • FIG. 5 shows an example of an image acquired by the image acquisition unit 42.
  • FIG. FIG. 6 shows an example of displaying the point cloud data acquired by the point cloud data acquiring unit 41 as a two-dimensional distance image.
  • the distance to each measurement position within the field of view of the distance measuring device 11 expressed in two-dimensional space is represented by the color or brightness at each point indicating each measurement position.
  • the control unit 43 acquires a second image and second point cloud data via the image acquisition unit 42 and the point cloud data acquisition unit 41 (step S102).
  • the viewpoint for capturing and measuring the second image and the second point cloud data by rotating and moving the moving body on which the distance measuring device 11 and the imaging device 12 are mounted changes the first image and the first point cloud model. It is different from the imaged and measured viewpoint.
  • the imaging range of the second image and the imaging range of the first image deviate from each other, the imaging ranges partially overlap each other.
  • the measurement range of the second point cloud data and the measurement range of the first point cloud data deviate, the measurement ranges partially overlap each other.
  • the first image and the first point cloud data are captured and measured at a first time
  • the second image and the second point cloud data are captured at a second time following the first time. and may be measured.
  • the control unit 43 may sequentially acquire images and point cloud data captured and measured at the same time.
  • a set of first image and first point cloud data and a set of second image and second point cloud data are obtained, and rotated between a first time and a second time Estimate volume and displacement.
  • the amount of rotation and the amount of movement after the second time can also be sequentially estimated in the same manner. Note that the first image and the first point cloud data do not necessarily have to be captured and measured at the same time.
  • the capturing of the first image and the measurement of the first point cloud data are sequentially performed at different times. be able to. The same applies to the second image and the second point cloud data.
  • steps S103 to S105 and steps S106 to S108 are a series of processes.
  • Steps S103 to S105 are processes for the first image and the second image.
  • Steps S106 to S108 are processes for the first point cloud data and the second point cloud data. Steps S103 to S105 and steps S106 to S108 may be interchanged or executed in parallel.
  • step S102 if there is distortion in the acquired first image and second image, the control unit 43 performs correction to reduce the distortion (step S103).
  • the first image and the second image may have barrel distortion as shown in FIG. 5 due to the lens of the imaging device 12 or the like.
  • the control unit 43 can correct this so that the image becomes a rectangle.
  • a first image and a second image as illustrated in the upper part of FIG. 7 are obtained.
  • the control unit 43 extracts feature points of the first image and the second image (step S104).
  • various methods including, for example, SHIFT (Scale-Invariant Feature Transform), SURF (Speeded Up Robust Features), and ORB (Oriented FAST and Rotated Brief) can be used.
  • SHIFT Scale-Invariant Feature Transform
  • SURF Speeded Up Robust Features
  • ORB Oriented FAST and Rotated Brief
  • RANSAC Random SAmple Consensus
  • RANSAC Random SAmple Consensus
  • the control unit 43 uses the feature points to calculate information indicating the correspondence between the first image and the second image (step S105).
  • the control unit 43 can obtain the correspondence relationship for each pixel between the first image and the second image by overlapping the images by matching feature points using a panoramic image creation technique. For example, the control unit 43 can superimpose the feature points of the second image on the feature points of the first image by performing homography transformation on the second image.
  • the control unit 43 calculates a transformation matrix for homography transformation from the feature points of the first image and the feature points of the second image.
  • the transformation matrix of homography transformation is information indicating the correspondence relationship between the first image and the second image.
  • FIG. 7 is an image of an office in which desks, chairs, and the like are arranged.
  • FIG. 8 is a simple drawing showing an image of synthesis using homographic transformation of the first image and the second image.
  • the transformation matrix of the homography transformation has a minimum of two so as to minimize the error between the feature points when the feature points of the first image and the feature points of the second image are matched using the homography transformation. It may be estimated using multiplication.
  • the number of measurement points (corresponding to points in the point cloud data) that the distance measuring device 11 can acquire in one scan is smaller than the number of pixels of the image captured by the imaging device 12 . Therefore, the control unit 43 performs processing for converting the first point cloud data and the second point cloud data into data having the same number of pixels as the number of pixels of the image (step S106). The control unit 43 converts the number of vertical and horizontal points of the first point cloud data and the second point cloud data to match the number of vertical and horizontal pixels of the first image and the second image. do. Thereby, each pixel of the first image and the second image is associated with any one of the first point cloud data and the second point cloud data, respectively.
  • step S107 if the expanded first point cloud data and second point cloud data have distortion, the control unit 43 corrects the distortion. This distortion correction may be performed before step S106 or after step S108, which will be described later, instead of after step S106.
  • FIGS. 9A and 9B show an example of displaying the first point cloud data and the second point cloud data after expansion and distortion correction as distance images.
  • FIGS. 9A and 9B it is difficult to detect edges of a subject with point cloud data, unlike images.
  • point cloud data it is easy to acquire distance data of measurement points on a flat portion such as a wall.
  • the point cloud data of FIGS. 9A and 9B are shown as examples. 9A and 9B are obtained by measuring the inside of an office with desks, chairs, etc., similar to FIG. 7, but are obtained by measuring a scene different from the first image and the second image shown in FIG. It has become.
  • the control unit 43 calculates point groups in a three-dimensional space from the expanded and distortion-corrected first point group data and second point group data (step S108). That is, the control unit 43 converts point cloud data (u, v, d) in a two-dimensional space consisting of two-dimensional coordinates (u, v) indicating the position of each point of the point cloud data and distance d between each point to Convert to coordinates (x, y, z) of a point in 3D space.
  • the coordinates (x, y, z) in the three-dimensional space are the positions measured by the distance measuring device 11 in the three-dimensional space.
  • the three-dimensional coordinates (x, y, z) are defined, for example, with the position of the distance measuring device 11 as the origin, the direction of the optical system in which the distance measuring device 11 detects the object ob as the z axis, and the direction perpendicular to the z axis.
  • a coordinate system having two axes perpendicular to each other as the x-axis and the y-axis can be used.
  • the coordinate axes of this three-dimensional space are fixed with respect to the distance measuring device 11, and rotate and move as the distance measuring device 11 rotates and moves.
  • 10A and 10B show point clouds in a three-dimensional space calculated based on the first point cloud data corresponding to FIG. 9A and the second point cloud data corresponding to FIG. 9B, respectively, viewed from a specific direction. It is represented as a diagram. 10A and 10B are observations of overlapping measurement ranges from different viewpoints.
  • control unit 43 associates the first point cloud data with the second point cloud data using the information indicating the correspondence relationship between the first image and the second image calculated in step S105. to acquire the coordinates of the corresponding points in the three-dimensional space (step S109). Step S109 will be described below in two stages.
  • the control unit 43 generates two-dimensional first point cloud data and second two-dimensional point cloud data having the same point arrays as the pixel arrays of the first image and the second image as shown in FIGS. 9A and 9B. Each point is associated with the point cloud data. Based on the information indicating the correspondence calculated in step S105, the control unit 43 extracts points of the first point cloud data and points of the second point cloud data located at mutually corresponding positions (u, v). do. For example, in step S105, when the control unit 43 calculates the transformation matrix for homographically transforming the second image into the first image, the control unit 43 adds this Apply a transformation matrix.
  • the control unit 43 extracts the points of the first point cloud data whose positions (u, v) match those of the converted second point cloud data. If the points of the extracted first point cloud data and the points of the second point cloud data have a distance d that is substantially equal to each other in consideration of the error and the range of movable distances, the two points correspond to It can be determined that there are In this method, the correspondence between the points on the first point cloud data and the points on the second point cloud data is examined based on the information indicating the correspondence. Therefore, much more points can be associated than when point cloud data are associated using only the feature points of the image.
  • the control unit 43 assigns a Obtain the coordinates (x, y, z) of the corresponding point in three-dimensional space.
  • the coordinates (x, y, z) in the three-dimensional space corresponding to the points of the two-dimensional point cloud data are calculated in step S108.
  • These coordinates (x, y, z) are the coordinates of the points of the first point group data and the points of the second point group data corresponding to each other represented by the coordinates of the three-dimensional space.
  • the control unit 43 excludes unnecessary points from the set of corresponding points represented by the coordinates (x, y, z) in the three-dimensional space (step S110). For example, points whose distance is calculated as 0 are deleted. Also, points that are determined to be abnormal values may be deleted.
  • the control unit 43 estimates the posture of the posture estimation system 10 and the moving object equipped with the posture estimation system 10 using the coordinates of the corresponding points in the three-dimensional space (step S111).
  • Posture estimation can be performed by obtaining a rotation matrix and a translation vector so as to minimize the distance between corresponding points included in the first point cloud data and the second point cloud data.
  • q i (i 1, . 2, . . . , n).
  • n is the number of corresponding points of each point cloud data.
  • R be the desired rotation matrix and t the desired translation vector.
  • the rotation matrix R and translation vector t are obtained by the following formula (1).
  • control unit 43 can calculate the amount of rotation and the amount of movement of the moving body between the first time and the second time to estimate the posture.
  • control unit 43 can sequentially acquire the point cloud data measured by the distance measuring device 11 and the image captured by the imaging device 12 while the moving body sequentially changes its position and posture. Based on the estimated amount of rotation and movement of the moving object, the control unit 43 can reconstruct the information of the measured point cloud data in the global coordinate system and construct an environment map. The control unit 43 can estimate the self-position and orientation of the mobile object on the constructed environment map of the global coordinate system.
  • the first point cloud data and the second point cloud data are obtained using information indicating the correspondence relationship between the first image and the second image. Since the corresponding points with the data are extracted, many corresponding points can be extracted. This improves the accuracy of posture estimation. Furthermore, when constructing an environment map using this attitude estimation system 10, the accuracy of attitude estimation is improved, so accumulation of errors in attitude estimation can be reduced, and a more accurate environment map can be constructed. can.
  • the present inventor prepares point cloud data and images obtained by rotating a moving body equipped with the posture estimation system 10 by 10 degrees in a horizontal plane by 10 degrees in total 360 degrees, and prepares point cloud data and images. was created. Further, as a comparative example, the present inventor extracted feature points from the same point cloud data and image by ORB from the image, and used these feature points to associate the point cloud data, thereby creating an environment map. made.
  • FIG. 11 is an environmental map constructed by the method of the present disclosure.
  • FIG. 12 is an environmental map constructed by the method of the comparative example. In either case, the angle ⁇ is the error in the amount of rotation accumulated as a result of a 360 degree rotation.
  • the inventor also performed pose estimation on data obtained by removing outliers by RANSAC from feature points by ORB.
  • the method of the present disclosure also showed significant accuracy improvements over this method.
  • the information processing device 13 of the present disclosure performs correspondence between the first point cloud data and the second point cloud data acquired from different viewpoints by the distance measuring device 11 for more points. It can be done easily. This improves the accuracy of posture estimation of the moving body. Furthermore, the improved accuracy of pose estimation improves the accuracy of environment map construction and self-location estimation.
  • attitude estimation system 11 ranging device 12 imaging device 13 information processing device 14 optical system 21 detection device 22 irradiation unit 23 reflection unit 24 control unit 25 first optical system 26 detection element 27 second optical system 28 imaging element 29 front stage Optical system 30 First post-stage optical system 31 Second post-stage optical system 32 Switching unit 41 Point group data acquisition unit 42 Image acquisition unit 43 Control unit 44 Storage unit ax Optical axis d1 First direction d2 Second direction ob Object (subject)

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Electromagnetism (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Length Measuring Devices By Optical Means (AREA)
PCT/JP2023/004645 2022-02-24 2023-02-10 情報処理装置、情報処理方法及びプログラム Ceased WO2023162730A1 (ja)

Priority Applications (4)

Application Number Priority Date Filing Date Title
JP2024503024A JPWO2023162730A1 (https=) 2022-02-24 2023-02-10
EP23759734.9A EP4485007A4 (en) 2022-02-24 2023-02-10 INFORMATION PROCESSING DEVICE, INFORMATION PROCESSING METHOD, AND PROGRAM
CN202380023654.3A CN118786361A (zh) 2022-02-24 2023-02-10 信息处理装置、信息处理方法以及程序
US18/839,622 US20250157079A1 (en) 2022-02-24 2023-02-10 Information processing device, information processing method, and program

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022-027269 2022-02-24
JP2022027269 2022-02-24

Publications (1)

Publication Number Publication Date
WO2023162730A1 true WO2023162730A1 (ja) 2023-08-31

Family

ID=87765751

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/004645 Ceased WO2023162730A1 (ja) 2022-02-24 2023-02-10 情報処理装置、情報処理方法及びプログラム

Country Status (5)

Country Link
US (1) US20250157079A1 (https=)
EP (1) EP4485007A4 (https=)
JP (1) JPWO2023162730A1 (https=)
CN (1) CN118786361A (https=)
WO (1) WO2023162730A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117646828A (zh) * 2024-01-29 2024-03-05 中国市政工程西南设计研究总院有限公司 一种用于检测顶管接口相对位移和渗漏水的装置及方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014173990A (ja) * 2013-03-08 2014-09-22 Topcon Corp 計測装置
US20200174130A1 (en) * 2017-08-04 2020-06-04 Bayerische Motoren Werke Aktiengesellschaft Method, Apparatus and Computer Program for a Vehicle
JP2020117779A (ja) 2019-01-24 2020-08-06 日本製鉄株式会社 鋼板及び鋼板の製造方法
WO2021135157A1 (zh) * 2019-12-31 2021-07-08 山东大学 岩体结构探测及危石探测系统及方法

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3333538B1 (en) * 2016-12-07 2020-09-09 Hexagon Technology Center GmbH Scanner vis

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014173990A (ja) * 2013-03-08 2014-09-22 Topcon Corp 計測装置
US20200174130A1 (en) * 2017-08-04 2020-06-04 Bayerische Motoren Werke Aktiengesellschaft Method, Apparatus and Computer Program for a Vehicle
JP2020117779A (ja) 2019-01-24 2020-08-06 日本製鉄株式会社 鋼板及び鋼板の製造方法
WO2021135157A1 (zh) * 2019-12-31 2021-07-08 山东大学 岩体结构探测及危石探测系统及方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4485007A4

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117646828A (zh) * 2024-01-29 2024-03-05 中国市政工程西南设计研究总院有限公司 一种用于检测顶管接口相对位移和渗漏水的装置及方法
CN117646828B (zh) * 2024-01-29 2024-04-05 中国市政工程西南设计研究总院有限公司 一种用于检测顶管接口相对位移和渗漏水的装置及方法

Also Published As

Publication number Publication date
US20250157079A1 (en) 2025-05-15
JPWO2023162730A1 (https=) 2023-08-31
CN118786361A (zh) 2024-10-15
EP4485007A1 (en) 2025-01-01
EP4485007A4 (en) 2025-11-05

Similar Documents

Publication Publication Date Title
JP5580164B2 (ja) 光学情報処理装置、光学情報処理方法、光学情報処理システム、光学情報処理プログラム
Kolb et al. Time‐of‐flight cameras in computer graphics
US8446492B2 (en) Image capturing device, method of searching for occlusion region, and program
CN108020825B (zh) 激光雷达、激光摄像头、视频摄像头的融合标定系统及方法
EP3438776B1 (en) Method, apparatus and computer program for a vehicle
JP5465128B2 (ja) 点群位置データ処理装置、点群位置データ処理システム、点群位置データ処理方法、および点群位置データ処理プログラム
Draelos et al. The Kinect up close: Adaptations for short-range imaging
CN106548489B (zh) 一种深度图像与彩色图像的配准方法、三维图像采集装置
EP2751521B1 (en) Method and system for alignment of a pattern on a spatial coded slide image
Martel et al. An active approach to solving the stereo matching problem using event-based sensors
US11977167B2 (en) Efficient algorithm for projecting world points to a rolling shutter image
JP2015017992A (ja) 環境を光学的に走査および測定する方法ならびにデバイス
JP6526955B2 (ja) センサ情報統合方法、及びその装置
JP2012057960A (ja) 点群位置データ処理装置、点群位置データ処理方法、点群位置データ処理システム、および点群位置データ処理プログラム
US10254402B2 (en) Stereo range with lidar correction
Bileschi Fully automatic calibration of lidar and video streams from a vehicle
JP2018155664A (ja) 撮像システム、撮像制御方法、画像処理装置および画像処理プログラム
Son et al. A lightweight and cost-effective 3D omnidirectional depth sensor based on laser triangulation
WO2023162730A1 (ja) 情報処理装置、情報処理方法及びプログラム
Chen et al. Geometric calibration of a multi-layer LiDAR system and image sensors using plane-based implicit laser parameters for textured 3-D depth reconstruction
CN112419427A (zh) 用于提高飞行时间相机精度的方法
EP4086850A1 (en) Calibrating system for colorizing point-clouds
JP7206855B2 (ja) 三次元位置検出装置、三次元位置検出システム、及び三次元位置検出方法
EP3255455A1 (en) Single pulse lidar correction to stereo imaging
KR20220153804A (ko) 라이다 장치

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23759734

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2024503024

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 18839622

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 202380023654.3

Country of ref document: CN

WWE Wipo information: entry into national phase

Ref document number: 2023759734

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2023759734

Country of ref document: EP

Effective date: 20240924

WWP Wipo information: published in national office

Ref document number: 18839622

Country of ref document: US