WO2023176621A1 - 推定装置、システム、推定方法、およびプログラム - Google Patents

推定装置、システム、推定方法、およびプログラム Download PDF

Info

Publication number
WO2023176621A1
WO2023176621A1 PCT/JP2023/008799 JP2023008799W WO2023176621A1 WO 2023176621 A1 WO2023176621 A1 WO 2023176621A1 JP 2023008799 W JP2023008799 W JP 2023008799W WO 2023176621 A1 WO2023176621 A1 WO 2023176621A1
Authority
WO
WIPO (PCT)
Prior art keywords
ratio value
region
point
unit
estimation device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2023/008799
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
正浩 加藤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Pioneer Corp
Pioneer Smart Sensing Innovations Corp
Original Assignee
Pioneer Corp
Pioneer Smart Sensing Innovations Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Pioneer Corp, Pioneer Smart Sensing Innovations Corp filed Critical Pioneer Corp
Priority to EP23770577.7A priority Critical patent/EP4495918A1/en
Priority to US18/845,364 priority patent/US20250189673A1/en
Priority to CN202380026034.5A priority patent/CN118974517A/zh
Priority to JP2024507802A priority patent/JPWO2023176621A1/ja
Publication of WO2023176621A1 publication Critical patent/WO2023176621A1/ja
Anticipated expiration legal-status Critical
Priority to JP2025122217A priority patent/JP2025160287A/ja
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B29/00—Maps; Plans; Charts; Diagrams, e.g. route diagram
    • G09B29/003—Maps
    • G09B29/006—Representation of non-cartographic information on maps, e.g. population distribution, wind direction, radiation levels, air and sea routes
    • G09B29/007—Representation of non-cartographic information on maps, e.g. population distribution, wind direction, radiation levels, air and sea routes using computer methods
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/005—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 with correlation of navigation data from several sources, e.g. map or contour matching
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/28—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network with correlation of data from several navigational instruments
    • G01C21/30—Map- or contour-matching
    • G01C21/32—Structuring or formatting of map data
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
    • G01C21/3804—Creation or updating of map data
    • G01C21/3807—Creation or updating of map data characterised by the type of data
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
    • G01C21/3804—Creation or updating of map data
    • G01C21/3833—Creation or updating of map data characterised by the source of data
    • G01C21/3837—Data obtained from a single source
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
    • G01C21/3804—Creation or updating of map data
    • G01C21/3833—Creation or updating of map data characterised by the source of data
    • G01C21/3848—Data obtained from both position sensors and additional sensors
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
    • G01C21/3863—Structures of map data
    • G01C21/387—Organisation of map data, e.g. version management or database structures
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
    • G08G1/0108—Measuring and analyzing of parameters relative to traffic conditions based on the source of data
    • G08G1/0112—Measuring and analyzing of parameters relative to traffic conditions based on the source of data from the vehicle, e.g. floating car data [FCD]
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
    • G08G1/0137—Measuring and analyzing of parameters relative to traffic conditions for specific applications
    • G08G1/0141—Measuring and analyzing of parameters relative to traffic conditions for specific applications for traffic information dissemination
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/09—Arrangements for giving variable traffic instructions
    • G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
    • G08G1/09626—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages where the origin of the information is within the own vehicle, e.g. a local storage device, digital map

Definitions

  • the present invention relates to an estimation device, a system, an estimation method, and a program.
  • Patent Document 1 describes that the position of a moving object including the lidar is estimated by associating point cloud data measured by a lidar with position information of an object for each unit area. It is also described that a reliability index of an estimated position is calculated using the ratio of the number of associated measurement points to the number of measurement points of point cloud data.
  • Patent Document 2 describes calculating an evaluation function for each voxel based on point cloud data measured by a lidar and a matching result for each voxel in a map database. It is also described that a voxel with a low evaluation function is extracted and matching deterioration information for that voxel is transmitted to a server device.
  • the accuracy of the map information is important. If map information does not accurately reflect the actual situation, the accuracy of position estimation may decrease.
  • Patent Document 1 cannot identify changing points where map information does not accurately reflect the actual situation.
  • Patent Document 2 there is room for improvement in the accuracy of estimating change points.
  • An example of the problem to be solved by the present invention is to provide a technique for accurately identifying an estimated change point where the content of map information is estimated to be different from the actual situation.
  • the invention according to claim 1 includes: a first acquisition unit that acquires point cloud data at multiple timings obtained by a sensor mounted on the moving body; a second acquisition unit that acquires map information; a dividing unit that divides each point cloud data acquired by the first acquiring unit into a plurality of predetermined regions; a ratio value calculation unit that calculates a ratio value indicating a correspondence ratio between each data point in the region and the map information for the first region and the second region different from the first region; , An estimated change in which the content of the map information is estimated to be different from the actual situation, using the strength of the correlation between the ratio value of the first area shifted in time or position and the ratio value of the second area.
  • the estimation device includes an estimation unit that specifies a point.
  • the invention according to claim 11 is The estimation device according to any one of claims 1 to 10, Equipped with a server, The estimation device transmits information indicating the identified estimated change point to a server, The server is receiving information indicating a plurality of said estimated change points; The system extracts the estimated change points where the content of the map information is likely to be different from the actual situation from the plurality of estimated change points by processing the received information indicating the plurality of estimated change points for each point.
  • the invention according to claim 13 is A computer-implemented estimation method, comprising: a first acquisition step of acquiring point cloud data at multiple timings obtained by a sensor mounted on the moving body; a second acquisition step of acquiring map information; a dividing step of dividing each point cloud data acquired in the first acquiring step into a plurality of predetermined regions; a ratio value calculation step of calculating a ratio value indicating a correspondence ratio between each data point in the region and the map information for the first region and the second region different from the first region; , An estimated change in which the content of the map information is estimated to be different from the actual situation, using the strength of the correlation between the ratio value of the first area shifted in time or position and the ratio value of the second area.
  • This estimation method includes an estimation step of specifying a point.
  • the invention according to claim 14 is A program that causes a computer to execute the estimation method according to claim 13.
  • FIG. 1 is a block diagram illustrating a functional configuration of an estimation device according to an embodiment.
  • FIG. FIG. 2 is a diagram illustrating how data obtained by a sensor mounted on a moving body is compared with an ND map.
  • FIG. 2 is a diagram illustrating how data obtained by a sensor mounted on a moving body is compared with an ND map. It is a flowchart illustrating the flow of processing performed by the estimation device according to the embodiment.
  • 1 is a block diagram illustrating a functional configuration of an estimation device according to a first embodiment.
  • FIG. 3 is a flowchart illustrating the flow of self-position estimation processing performed by the estimation device according to the first embodiment.
  • FIG. 2 is a diagram illustrating a computer for realizing an estimation device.
  • FIG. 1 is a diagram illustrating a functional configuration of a system according to a first embodiment
  • FIG. FIG. 3 is a diagram showing a plurality of regions.
  • FIG. 3 is a diagram for explaining a correspondence ratio.
  • 11 is a diagram showing changes over time from DDAR(1) to DDAR(8) in the example of FIG. 10.
  • FIG. 3 is a diagram illustrating an example of normalization of DDAR.
  • FIG. 3 is a diagram for explaining processing performed by an estimator.
  • FIG. 3 is a diagram for explaining conditions for determining correlation.
  • FIG. 3 is a diagram for explaining the meanings of the first condition to the third condition.
  • 7 is a flowchart illustrating a flow of change point detection processing performed by the estimation device according to the first embodiment.
  • FIG. 1 is a block diagram illustrating a functional configuration of a server according to a first embodiment.
  • FIG. FIG. 6 is a diagram for explaining determination based on detection time of change point information.
  • 3 is a flowchart illustrating the flow of processing performed by a server.
  • 3 is a flowchart illustrating the flow of ND map update processing performed by the server according to the first embodiment.
  • 7 is a flowchart illustrating the flow of change point detection processing performed by the estimation device according to the second embodiment. It is a graph illustrating a cross-correlation function C( ⁇ ). It is a figure which shows the original ND map and the ND map which generated the change point.
  • (a) to (k) are diagrams showing various data obtained by a vehicle traveling on the course shown in FIG. 23.
  • FIG. 3 is a diagram showing the relationship between a moving object (vehicle) and regions (1) to (8) in Experimental Example 1.
  • FIG. (a) to (c) are diagrams showing the results of performing the filter processing, normalization processing, and limit processing described in Example 1 on each DDAR, respectively.
  • (a) to (e) are diagrams showing each data in the process of change point detection processing. It is a figure which shows DDAR of each area on the left side seen from a vehicle.
  • (a) to (e) are diagrams showing each data in the process of change point detection processing.
  • FIG. 26 is a diagram showing a cross-correlation function C( ⁇ ) calculated using DDAR(5) and DDAR(8) shown in FIG. 26(c).
  • 29 is a diagram showing a cross-correlation function C( ⁇ ) calculated using DDAR(1) and DDAR(4) shown in FIG. 28.
  • FIG. FIG. 3 is a diagram for explaining the association between data points and voxels.
  • FIG. 1 is a block diagram illustrating the functional configuration of an estimation device 10 according to an embodiment.
  • the estimation device 10 includes a first acquisition section 110, a second acquisition section 130, a division section 150, a ratio value calculation section 170, and an estimation section 190.
  • the first acquisition unit 110 acquires point cloud data at multiple timings obtained by a sensor mounted on a moving object. Point cloud data at multiple timings is, for example, point cloud data at multiple times.
  • the second acquisition unit 130 acquires map information.
  • the dividing unit 150 divides each point cloud data acquired by the first acquiring unit 110 into a plurality of predetermined regions.
  • the ratio value calculation unit 170 calculates a ratio value indicating the association ratio between each data point in the area and the map information for the first area and the second area different from the first area.
  • the estimation unit 190 estimates that the content of the map information is different from the actual situation, using the strength of the correlation between the ratio value of the first region shifted in time or position and the ratio value of the second region. Identify points of estimated change.
  • a moving object such as a vehicle
  • the position of the moving object at the next point in time is estimated using the position of the moving object at a certain point in time and the moving speed and direction of movement of the moving object up to the next point in time.
  • NDT Normal Distribution Transform
  • NDT scan matching matches data detected from objects around a moving object with an ND (Normal Distributions) map that divides three-dimensional space into grids (voxels) of a predetermined size and represents a normal distribution.
  • ND Normal Distributions
  • This is a method to calculate the self-position of
  • the accuracy of the ND map is important for highly accurate self-position estimation. Since detection data of objects by moving bodies is matched with the ND map, if the accuracy of the ND map is low, the accuracy of self-position estimation will also be reduced. Therefore, the ND map needs to be as accurate as possible.
  • an ND map is created, for example, by a dedicated measurement vehicle equipped with various sensors acquiring high-density and highly accurate three-dimensional point cloud data while actually driving.
  • the estimation device 10 can identify an estimated change point where the content of the map information is estimated to be different from the actual situation. Therefore, for such points, it is possible to adjust the handling of ND map data and promote updating of the ND map.
  • map information is data that can be used as reference data for NDT scan matching.
  • the map information will also be referred to as "ND map”.
  • the ND map includes a plurality of voxel data.
  • Voxel data is data in which position information of a stationary structure, etc. is recorded in each region (also called “voxel") when a three-dimensional space is divided into a plurality of regions.
  • the voxel data includes data representing measured point cloud data of stationary structures within each voxel using a normal distribution.
  • the ND map includes at least the voxel ID, voxel coordinates, average vector, covariance matrix, and point group number information of each of the plurality of voxels.
  • Each voxel is a cube obtained by dividing space into a grid, and has a predetermined shape and size.
  • Voxel coordinates indicate the absolute three-dimensional coordinates of a reference position such as the center position of a voxel.
  • the mean vector and covariance matrix correspond to parameters when a point group within a voxel is represented by a normal distribution.
  • the point group number information is information indicating the number of point groups used to calculate the average vector and covariance matrix of the voxel.
  • FIGS. 2 and 3 are diagrams illustrating how data obtained by the sensor 210 mounted on the mobile object 20 is compared with the ND map.
  • FIG. 2 shows an example where the ND map matches the actual situation
  • FIG. 3 shows an example where the ND map does not match the actual situation.
  • rectangles labeled A to E indicate structures.
  • each square represents a voxel included in the ND map
  • the black circle represents a point group obtained by the sensor 210 of the moving body 20.
  • the sensor 210 is, for example, a sensor that measures the distance to a target object by emitting light and receiving reflected light reflected by the target object.
  • the sensor 210 is not particularly limited, but may be a radar or lidar (LIDAR: Laser Imaging Detection and Ranging, Laser Illuminated Detection and Ranging, or LiDAR: Light Detection and Ranging).
  • LIDAR Laser Imaging Detection and Ranging
  • Laser Illuminated Detection and Ranging or LiDAR: Light Detection and Ranging
  • the light emitted from the sensor 210 is not particularly limited, it is, for example, infrared light. Further, the light emitted from the sensor 210 is, for example, a laser pulse.
  • the sensor 210 calculates the distance from the sensor 210 to the object using, for example, the time from emitting the pulsed light to receiving the reflected light and the propagation speed of the pulsed light.
  • the direction in which light is emitted from the sensor 210 is variable, and by sequentially performing measurements in a plurality of directions, the measurement area is scanned. For example, by moving the emitted light vertically while reciprocating horizontally, the distance to an object in each angular direction can be measured, so three-dimensional information data within the horizontal and vertical scanning range can be measured. Obtainable.
  • the sensor 210 outputs point cloud data in which the three-dimensional position of the light reflection point is associated with the reflection intensity (that is, the light reception intensity at the sensor 210).
  • the first acquisition unit 110 of the estimation device 10 acquires the point cloud data.
  • the point cloud data output from the sensor 210 is configured in units of frames. One frame is composed of data obtained by scanning the measurement area once.
  • the sensor 210 generates a plurality of consecutive frames by repeatedly scanning the measurement area.
  • a plurality of sensors 210 may be mounted so as to scan in different directions when viewed from the moving body 20. In addition, the sensor 210 may scan the surroundings by rotating through 360 degrees.
  • point cloud data that detects structures A, B, and D near the road on which the moving object 20, which is a vehicle, is traveling is correctly associated with voxels on the ND map.
  • accurate self-position estimation is performed.
  • structure C is behind structure B. Therefore, when this ND map is generated, a point group regarding structure C is not obtained, and no voxel corresponding to structure C exists in the ND map.
  • the point cloud data that detected the structures C and E will not be used for the NDT calculation, and only the point group data that detected the structure A will be used for the calculation with the voxel data. Therefore, the number of data used for the NDT calculation decreases, and there is a possibility that the accuracy of self-position estimation decreases.
  • the voxel data of structure D and the point cloud measured for structure E may be erroneously matched, or the voxel data of structure B and the point cloud measured for structure C may be erroneously matched. There is also the possibility of relaxing. Such erroneous matching can cause the estimated self-position to deviate significantly from the actual position.
  • the ratio value calculation unit 170 calculates a ratio value indicating the correspondence ratio between each data point in the region and the map information for the first region and the second region. Calculate. Then, the estimation unit 190 uses the strength of the correlation between the ratio value of the first area shifted in time or position and the ratio value of the second area to estimate that the content of the map information is different from the actual situation. Identify estimated change points. Therefore, if a situation occurs where the actual environment and the ND map do not match, the location can be specified with high accuracy.
  • FIG. 4 is a flowchart illustrating the flow of processing performed by the estimation device 10 according to the present embodiment.
  • the estimation method according to this embodiment is executed by a computer.
  • the estimation method according to this embodiment includes a first acquisition step S101, a second acquisition step S102, a division step S103, a ratio value calculation step S104, and an estimation step S105.
  • the first acquisition step S101 point cloud data at multiple timings obtained by a sensor mounted on a moving object is acquired.
  • map information is acquired.
  • each point cloud data acquired in the first acquisition step S101 is divided into a plurality of predetermined regions.
  • ratio value calculation step S104 a ratio value indicating the correspondence ratio between each data point in the area and the map information is calculated for the first area and the second area different from the first area.
  • estimation step S105 it is estimated that the content of the map information is different from the actual situation, using the strength of the correlation between the ratio value of the first region shifted in time or position and the ratio value of the second region. Estimated change points are identified.
  • FIG. 5 is a block diagram illustrating the functional configuration of the estimation device 10 according to the first embodiment.
  • the estimation device 10 according to the present example has the configuration of the estimation device 10 according to the embodiment.
  • the estimation device 10 according to this embodiment further includes a self-position estimation section 140 and a reliability calculation section 180.
  • the estimation section 190 includes a shift section 191, a correlation determination section 192, and a change point identification section 193.
  • FIG. 6 is a flowchart illustrating the flow of self-position estimation processing performed by the estimation device 10 according to the present embodiment.
  • the self-position estimating unit 140 estimates the position of the mobile object 20.
  • the sensor 210 will be described below as a lidar, the sensor 210 may be of another type.
  • the estimation device 10 first specifies the first estimated self-position (S10). Specifically, the self-position estimating unit 140 uses the positioning result by a GNSS (Global Navigation Satellite System) provided in the mobile object 20 or the estimation device 10 as the first estimated self-position. Next, the self-position estimating unit 140 calculates the latest predicted self-position using the previous estimated self-position (S20). Specifically, the self-position estimating unit 140 acquires the speed and yaw angular velocity of the mobile body 20 from a speed sensor and a gyro sensor provided on the mobile body 20, and determines the moving direction of the mobile body 20 from the previous estimated self-position. and determine the amount of movement. Then, the self-position estimating unit 140 calculates, as the predicted self-position, the position after movement when the previous estimated self-position is moved in the specified movement direction and movement amount.
  • S10 the first estimated self-position
  • the self-position estimating unit 140 uses the positioning result by a GNSS (Global Navigation Satellite System) provided in the mobile object 20 or the
  • the self-position estimating unit 140 determines whether an ND map around the predicted self-position has already been obtained (S30). If the ND map has already been acquired (Yes in S30), the self-position estimation unit 140 performs the process in S50 without performing the process in S40. If the ND map has not been acquired yet (No in S30), the second acquisition unit 130 acquires an ND map around the predicted self-position (S40).
  • the second acquisition unit 130 may acquire the ND map by reading it from a storage unit that is accessible from the second acquisition unit 130, or may acquire it from outside via a network.
  • the storage unit accessible from the second acquisition unit 130 may be provided inside the estimation device 10 or may be provided outside the estimation device 10.
  • the second acquisition unit 130 sets voxels to which a non-recommended flag is attached in the acquired ND map so as not to be used for NDT matching.
  • the second acquisition unit 130 lowers the weight of the voxel to which the non-recommended flag is attached.
  • the weight is a value indicating the magnitude of influence that the information of the voxel has on the calculation of self-position estimation in NDT matching. The deprecation flag will be described in detail later.
  • the first acquisition unit 110 determines whether the point cloud data was acquired (S50). If the point cloud data is not acquired (No in S50), such as when the vehicle is traveling in an open area with no structures around it, the process returns to S20.
  • point cloud data is acquired by the first acquisition unit 110 (Yes in S50)
  • downsampling of the point cloud data is performed in S60. Specifically, the first acquisition unit 110 performs downsampling on the acquired point cloud data to obtain a predetermined number of point clouds.
  • the self-position estimating unit 140 performs NDT matching processing and calculates the estimated self-position. Specifically, the self-position estimating unit 140 uses the predicted self-position as an initial value and performs NDT matching processing using the ND map around the predicted self-position and the downsampled point cloud data obtained by the first acquisition unit 110. I do. NDT matching processing and self-position estimation using the result can be performed using existing methods. The self-position estimating unit 140 sets the obtained estimated self-position as the latest estimated self-position.
  • the estimated self-position may be output to a device other than the estimating device 10, or may be held in a storage device that is accessible from the self-position estimation unit 140.
  • the estimated self-position can be used for functions such as route navigation, automatic driving, or driving assistance.
  • the self-position estimating unit 140 determines whether the end condition is satisfied (S90).
  • the case where the termination condition is satisfied is, for example, a case where the operation of the moving body 20 is stopped or a case where an operation to stop the estimation process by the estimation device 10 is performed. If the termination condition is satisfied (Yes in S90), the estimation device 10 terminates the process. If the termination condition is not met (No in S90), the process returns to S20.
  • Each functional component of the estimation device 10 may be realized by hardware that implements each functional component (e.g., a hardwired electronic circuit, etc.), or by a combination of hardware and software (e.g., an electronic circuit). It may also be realized by a combination of a circuit and a program that controls it.
  • a case in which each functional component of the estimation device 10 is realized by a combination of hardware and software will be further described.
  • FIG. 7 is a diagram illustrating a computer 1000 for realizing the estimation device 10.
  • Computer 1000 is any computer.
  • the computer 1000 is a SoC (System On Chip), a Personal Computer (PC), a server machine, a tablet terminal, a smartphone, or the like.
  • the computer 1000 may be a dedicated computer designed to implement the estimation device 10, or may be a general-purpose computer.
  • the computer 1000 has a bus 1020, a processor 1040, a memory 1060, a storage device 1080, an input/output interface 1100, and a network interface 1120.
  • the bus 1020 is a data transmission path through which the processor 1040, memory 1060, storage device 1080, input/output interface 1100, and network interface 1120 exchange data with each other.
  • the processor 1040 is a variety of processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or an FPGA (Field-Programmable Gate Array).
  • the memory 1060 is a main storage device implemented using RAM (Random Access Memory) or the like.
  • the storage device 1080 is an auxiliary storage device implemented using a hard disk, an SSD (Solid State Drive), a memory card, a ROM (Read Only Memory), or the like.
  • the input/output interface 1100 is an interface for connecting the computer 1000 and an input/output device.
  • an input device such as a keyboard and an output device such as a display are connected to the input/output interface 1100.
  • the network interface 1120 is an interface for connecting the computer 1000 to a network.
  • This communication network is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network).
  • the method by which the network interface 1120 connects to the network may be a wireless connection or a wired connection.
  • the storage device 1080 stores program modules that implement each functional component of the estimation device 10.
  • Processor 1040 reads each of these program modules into memory 1060 and executes them, thereby realizing the functions corresponding to each program module.
  • FIG. 8 is a diagram illustrating the functional configuration of the system 50 according to this embodiment.
  • a system 50 according to this embodiment includes an estimation device 10 and a server 40.
  • the estimation device 10 transmits information indicating the identified estimated change point (hereinafter also referred to as "change point information") to the server 40.
  • the server 40 receives a plurality of change point information and processes the received change point information for each point to identify estimated change points (hereinafter referred to as "highly reliable change points") where the content of the map information is likely to differ from the actual situation. ) is extracted from a plurality of estimated change points indicated by a plurality of change point information.
  • the estimation device 10 and the server 40 can communicate wirelessly.
  • the estimation device 10 is provided, for example, in a moving body 20.
  • the server 40 may receive change point information from a plurality of estimation devices 10. The processing performed by the server 40 will be described in detail later.
  • the first acquisition unit 110 acquires point cloud data generated by the lidar.
  • the first acquisition unit 110 may acquire point cloud data generated by the lidar and temporarily stored in the storage unit, or may acquire point cloud data directly from the lidar.
  • the first acquisition unit 110 acquires point cloud data of a plurality of frames in the order in which they are generated. As described above, the first acquisition unit 110 performs downsampling on the acquired point cloud data.
  • FIG. 9 is a diagram showing a plurality of areas.
  • the dividing unit 150 divides the point cloud data obtained around the moving object 20 into data of a plurality of regions with the position of the moving object 20 as a reference. The position of each area with respect to the moving body 20 and the size of each area are determined in advance.
  • the moving body 20 is provided with a plurality of sensors 210, each of which repeatedly generates point cloud data within a measurement area 220 at the same timing.
  • the first acquisition unit 110 acquires point cloud data of all sensors 210.
  • the dividing unit 150 combines the point cloud data obtained at the same timing on a coordinate system based on the moving body 20 to generate point cloud data around the moving body 20.
  • the same timing is, for example, the same time. However, the same timing does not necessarily have to be the completely same timing, and there may be some error. Then, by specifying which region includes the position of each data point, each data point around the moving object 20 is distributed to a plurality of regions. By doing so, point cloud data for each of the plurality of regions can be obtained.
  • the area around the moving body 20 is divided into eight areas, area (1) to area (8).
  • the interval between adjacent regions is L.
  • the region interval L is, for example, the distance between the centers of each region, and is the distance in the traveling direction of the moving body 20.
  • the point cloud data is divided into at least a right region and a left region with respect to the moving body 20.
  • the point cloud data is divided into at least a region on the front side and a region on the rear side in the traveling direction with respect to the moving body 20. It is preferable that the point cloud data is divided into four or more regions in the traveling direction of the moving body 20. The sizes of the multiple areas do not need to be the same.
  • the ratio value calculation unit 170 calculates a ratio value indicating the association ratio between each data point in the area and the map information. Specifically, the ratio value calculation unit 170 calculates time-series ratio values of the first region and the second region. Note that the ratio value may be the matching ratio itself, a value obtained by inverting the ratio, or a value obtained by performing a predetermined calculation on the ratio.
  • the ratio value of each area will be referred to as DDAR (Divided Data Association Ratio)
  • FIG. 10 is a diagram for explaining the association ratio.
  • rectangles indicated by A and C indicate structures.
  • each square represents a voxel included in the ND map
  • the black circle represents a point group obtained by the sensor 210 of the moving body 20.
  • Correlation between the ND map and the point cloud data is performed by the self-position estimating unit 140.
  • the ND map includes voxels corresponding to positions where stationary structures and the like exist. If a data point in the point cloud is located at a position that corresponds to the position of a certain voxel, that data point is associated with that voxel.
  • FIG. 32 is a diagram for explaining the association between data points and voxels.
  • the self-position estimating unit 140 first converts the coordinates of each point included in the point cloud data into the world coordinate system based on the predicted self-position. For example, if the voxel size of the ND map is 1 m square, the x, y, and z coordinates of each point are rounded off to the nearest whole number. Then, the self-position estimating unit 140 compares each point with the ND map shown in the same world coordinate system, identifies which voxel each point is in, and performs correspondence.
  • This figure shows voxels in two-dimensional form for explanation.
  • each solid rectangle indicates a voxel.
  • the voxel number is shown at the upper left of each voxel.
  • Black circles indicate data points in the point cloud.
  • the voxel coordinates are the coordinates of the center position of the voxel, but the voxel coordinates are not limited to this, and may be, for example, the coordinates of any vertex of the voxel.
  • the self-position estimating unit 140 performs NDT matching processing using the associated data points and voxels.
  • the mobile object 20 travels near a point where there is a difference between the actual environment and the ND map as shown in FIG. 10, the data point where the structure C was detected is not correlated, and the correlation ratio decreases.
  • region division shown in FIG. 9 it is expected that the DDAR of each region will decrease in the order of DDAR (5), DDAR (6), DDAR (7), and DDAR (8). .
  • DDAR may decrease from region (7) to region (8). Highly sexual.
  • FIG. 11 is a diagram showing changes over time from DDAR(1) to DDAR(8) in the example of FIG.
  • Each of DDAR(1) to DDAR(4) corresponding to the left side of the moving body 20 where there is no change point maintains a high value.
  • DDAR(5) to DDAR(8) corresponding to the right side of the moving body 20 where there is a change point there is a timing at which the value decreases. Such timing becomes later from DDAR(5) to DDAR(8). Therefore, by understanding how the timing at which the DDAR decreases changes in a plurality of regions, the point of change can be estimated with high accuracy.
  • the first region and the second region are two different regions among the plurality of regions obtained by division.
  • the relationship between the first region and the second region is not particularly limited, it is preferable that the first region and the second region are regions shifted in the traveling direction of the moving body 20. In this case, it is preferable that the amount of deviation in the traveling direction is large.
  • the first region is one of the frontmost region and the rearmost region in the traveling direction among the plurality of regions, and the second region is the other of the frontmost region and the rearmost region. It is preferable that there be.
  • the first region and the second region are both regions on the right side in the direction of travel with respect to the moving body 20, or both on the left side in the direction of travel with respect to the mobile body 20.
  • the region (5) is the first region and the region (8) is the second region, but the first region and the second region are not limited to this example.
  • the estimation device 10 does not need to use the DDAR of all regions obtained by division to estimate the estimated change point.
  • the processing performed by the ratio value calculation section 170 and the estimation section 190 may be performed on at least the first region and the second region.
  • the estimation device 10 may provide a plurality of sets of the first region and the second region to estimate the estimated change point. For example, a first area and a second area on the right side of the moving direction of the moving body 20 and a first area and a second area on the left side of the moving direction of the moving body 20 are set, and estimation for the left and right sides of the moving body 20 is performed.
  • the change points may be estimated individually.
  • the ratio value calculation unit 170 acquires information indicating the result of associating the point cloud data of the entire area with the ND map from the self-position estimation unit 140. Further, the ratio value calculation unit 170 counts the number of data points included in each divided area. Then, for each region, the number of data points associated with voxels is counted. Then, the DDAR for each area is calculated by dividing the number of associated data points by the number of data points included in the area.
  • the ratio value calculation unit 170 performs several processes on the DDAR in order to improve the accuracy of estimating the change point.
  • filter processing and normalization processing will be explained. Although not all of these processes necessarily need to be performed, it is preferable that all of these processes be performed in order to improve estimation accuracy. Note that these processes may be performed on the ratio value before the shift, which will be described later, or may be performed on the ratio value after the shift.
  • the ratio value calculation unit 170 performs first-order lag filter processing on the DDAR of each region. Specifically, it is preferable that the ratio value calculation unit 170 performs first-order lag filter processing on the DDAR of each region using a time constant of a value that is 10 times or more and 20 times or less the period of the NDT processing. By doing so, it becomes possible to grasp the tendency of change while suppressing instantaneous fluctuations in DDAR. In turn, the stability of correlation determination can be improved.
  • FIG. 12 is a diagram illustrating an example of normalization of DDAR. If an error occurs due to some influence in self-position estimation by NDT and the estimated self-position deviates from the correct position, the correspondence ratio between point cloud data and voxels decreases overall. If such a situation exists, the DDAR of each region will decrease in value at the same time. Since this is not due to the change point, it becomes an error in the correlation determination for detecting the change point.
  • the ratio value calculation unit 170 calculates an overall ratio value DAR (Data Association Ratio) indicating the association ratio between each data point of the point cloud data before division and map information, and calculates the overall ratio value DAR. normalize the ratio value of the first region and the ratio value of the second region.
  • the estimation unit 190 then identifies the estimated change point using the normalized ratio value. By doing so, it is possible to reduce the influence of deviations in the estimated self-position.
  • DDAR since DDAR may be larger than DAR, it may exceed 1 when normalized. In the correlation determination described below, it is desirable that the upper limit of DDAR is 1, so the ratio value calculation unit 170 may further perform limit processing to limit the value of DDAR to a range of 0 to 1. That is, if the value of DDAR after normalization exceeds 1, the ratio value calculation unit 170 replaces the value with 1.
  • the estimation unit 190 identifies the estimated change point using the strength of the correlation between the shifted ratio value obtained by shifting the time of the ratio value of the first region and the ratio value of the second region. do. Specifically, the estimation unit 190 generates a shifted ratio value by shifting the time axis of the ratio value in the first region by time ⁇ so as to match the time axis of the ratio value in the second region. The estimation unit 190 then identifies the estimated change point using the generated post-shift ratio value of the first region and the ratio value of the second region.
  • the time ⁇ is a time calculated using the speed of the moving object.
  • the estimation unit 190 calculates a first index by multiplying the post-shift ratio value of the first region by the ratio value of the second region. Furthermore, the estimation unit 190 calculates the second index by subtracting one of the post-shift ratio value of the first region and the ratio value of the second region from the other. Then, the estimated change point is identified using the first index and the second index. Note that identification of the estimated change point will be explained in detail below.
  • FIG. 13 is a diagram for explaining the processing performed by the estimation unit 190.
  • a portion where the value decreases occurs in the time-series DDAR, and the decreasing portion appears slightly shifted in multiple regions. That is, it is presumed that there is a strong correlation between the DDARs of multiple areas shifted in time according to the movement of the mobile object 20. Therefore, by evaluating the strength of the correlation, the estimated change point can be identified.
  • the estimation unit 190 divides the area interval 3L by the speed of the moving body 20 to obtain the time ⁇ for the moving body 20 to travel a distance of 3L.
  • the estimation unit 190 shifts the time axis of DDAR(5) by time ⁇ . In this way, the shifted DDAR(5), ie, the shifted ratio value, is obtained.
  • the correlation determination unit 192 determines the correlation using the shifted ratio value of area (5) and the ratio value of area (8). Specifically, first, the correlation determination unit 192 inverts each of the shifted DDAR(5) and DDAR(8). That is, the value obtained by subtracting the DDAR(k) before inversion from 1 is set as the DDAR(k) after inversion. Then, the first index M is obtained by multiplying the inverted DDAR(5) and the inverted DDAR(8). Further, the correlation determination unit 192 obtains the second index S by subtracting DDAR(8) from the shifted DDAR(5). Using the time series DDAR (5) and the time series DDAR (8), a time series first index M and a second index S are obtained.
  • FIG. 14 is a diagram for explaining conditions for determining correlation.
  • the correlation determining unit 192 determines whether the following first to third conditions are satisfied for the first index M and the second index S, respectively. If all of the first to third conditions are satisfied, the correlation determination unit 192 determines that the correlation between DDAR(5) and DDAR(8) is strong.
  • Wt is the time width of the period during which the first index M is continuously equal to or greater than the threshold value Th1.
  • the threshold Th1 is, for example, 0.01 or more and 0.1 or less, and preferably 0.04.
  • v is the speed of the moving body 20.
  • the threshold value Th2 is, for example, 1 m or more and 8 m or less, and preferably 5 m.
  • the second condition is expressed by the following equation (1). That is, the second condition is that the average value M ave of the first index M (M(k)) within a period in which the first index M is continuously equal to or greater than the threshold Th1 is greater than the threshold Th3.
  • N is the number of data within a period in which the first index M is continuously equal to or greater than the threshold value Th1.
  • the threshold Th3 is, for example, 0.01 or more and 0.1 or less, and preferably 0.05.
  • the third condition is expressed by the following equation (2). That is, the third condition is that the deviation S dev of the second index S (S(k)) is smaller than the threshold Th4 during a period in which the first index M is continuously equal to or greater than the threshold Th1.
  • the threshold Th4 is, for example, 0.1 or more and 0.5 or less, and preferably 0.2.
  • FIG. 15 is a diagram for explaining the meanings of the first to third conditions. As shown in the leftmost column of this figure, when two DDARs are large to some extent, have similar sizes, and match positions, the multiplication result becomes large and the subtraction result becomes small. In such a case, it can be said that there is a strong correlation between the two DDARs.
  • the multiplication result will be small even if the peak positions match.
  • the third column from the left if there is a difference in peak position, even if the two DDARs are large to some extent, the multiplication result will be small and the deviation of the subtraction result will be large.
  • the fourth column from the left when one DDAR is small, the multiplication result is large, but the deviation of the subtraction result also becomes large.
  • the correlation determination unit 192 may evaluate the correlation between the two DDARs using a method other than the above method. Further, the correlation determination unit 192 may evaluate the correlation between the two DDARs using only one of the first index and the second index.
  • the correlation determination unit 192 determines that the first index M is continuously equal to or higher than the threshold Th1, and the period in which the change point is reflected. . Then, the change point specifying unit 193 specifies the timing at which the data for this period was obtained in the DDAR (8), and also specifies the direction of the DDAR (8) as seen from the mobile object 20 as the change point direction. Then, the change point specifying unit 193 specifies a position in the direction of the change point as the position of the estimated change point based on the estimated self-position at that timing.
  • the estimation device 10 further calculates the reliability regarding the estimated change point. Then, the calculated reliability is transmitted to the server 40. Using the received reliability, the server 40 can extract an estimated change point whose map information is likely to differ from the actual situation from among a plurality of estimated change points.
  • the reliability calculation unit 180 calculates the reliability of the estimated change point. Specifically, the larger W d , the larger M ave , and the smaller S dev calculated in the judgments from the first condition to the third condition described above, the higher the reliability of the change point (or the importance of the change point). ) to be higher.
  • the change point identification unit 193 can calculate the reliability R cp using the following equations (3) to (6).
  • a, b, and c are each predetermined coefficients.
  • R cp1 , R cp2 , and R cp3 which are the calculation results of equations (3), (4), and (5), all have values within the range of 0 to 1. Therefore, as each of R cp1 , R cp2 , and R cp3 becomes larger, the reliability R cp also becomes larger and approaches 1. Moreover, if any one of R cp1 , R cp2 , and R cp3 is close to 0, the reliability R cp becomes small.
  • the estimation device 10 associates information indicating the position of the estimated change point with the time at which the estimated change point was identified and the reliability, and transmits the information to the server 40.
  • the server 40 receives the information transmitted from the estimation device 10.
  • FIG. 16 is a flowchart illustrating the flow of the change point detection process performed by the estimation device 10 according to this embodiment.
  • the dividing unit 150 divides the point cloud data into data of a plurality of regions.
  • the ratio value calculation unit 170 calculates the DDAR of each area. Further, the ratio value calculation unit 170 performs the above-described filter processing and normalization on the DDAR.
  • the shift unit 191 calculates a shift amount based on the speed of the moving object 20, and shifts the DDAR of the first area. Then, in S803, the correlation determination unit 192 inverts the shifted DDAR of the first area and the DDAR of the second area, and calculates the first index and the second index using both DDARs after inversion. Further, in S804, the correlation determining unit 192 calculates W d , M ave , and S dev using the first index and the second index.
  • the correlation determining unit 192 determines whether the calculated W d , M ave , and S dev satisfy the first to third conditions, respectively (S805). If at least one of the first to third conditions is not satisfied (No in S805), no change point is detected and the change point detection process ends. On the other hand, if all of the first to third conditions are satisfied (Yes in S805), the change point specifying unit 193 uses the estimated self-position specified by the self-position estimating unit 140 to specify the position of the estimated change point. Further, the reliability calculation unit 180 uses W d , M ave , and S dev to generate reliability regarding the estimated change point. (S806). Then, in S807, information indicating the position, detection time, and reliability of the estimated change point is transmitted from the estimation device 10 to the server 40. Then, the change point detection process ends.
  • FIG. 17 is a block diagram illustrating the functional configuration of the server 40 according to this embodiment.
  • the server 40 according to this embodiment includes a change point information acquisition section 410, an information expansion section 420, an evaluation section 430, a change point extraction section 440, a maintenance required point registration section 450, and a flag assignment section 460.
  • the change point information is transmitted from the estimation device 10 to the server 40.
  • the change point information acquisition unit 410 acquires change point information from the estimation device 10.
  • the estimation device 10 transmits change point information each time an estimated change point is specified.
  • the server 40 may acquire change point information from a plurality of estimation devices 10. By doing so, the change point information acquisition unit 410 acquires a plurality of change point information.
  • the information development unit 420 develops the change point information acquired by the change point information acquisition unit 410 into information for each point. For example, when the positions indicated by a plurality of pieces of change point information are within a predetermined area on the map, the information expansion unit 420 groups the pieces of change point information as information regarding the same point. Information development by the information development unit 420 may be performed each time the change point information acquisition unit 410 acquires change point information, or may be performed at a predetermined cycle.
  • the evaluation unit 430 evaluates the reliability of the estimated change point for each point, that is, for each of the above-mentioned grouped information. For example, the evaluation unit 430 makes a determination for each point using the detection time linked to the change point information, the number of times the change point information has been acquired, and the reliability R cp linked to the change point information. , to extract highly reliable change points where the content of map information is likely to differ from the actual situation. However, the evaluation unit 430 may make the determination using only one or two of the detection time, the number of cases, and the reliability R cp . Note that if the detection time is not linked to the change point information, the time at which the change point information acquisition unit 410 acquires the change point information may be used instead of the detection time.
  • the evaluation unit 430 determines that the point is a highly reliable change point.
  • the change point extraction unit 440 can extract highly reliable change points from the plurality of estimated change points indicated by the plurality of change point information acquired by the change point information acquisition unit 410.
  • FIG. 18 is a diagram for explaining determination based on the detection time of change point information.
  • the change point follows a process in which it is detected at a certain time and then is no longer detected again. This is because if the parked vehicle moves and no longer exists, it will no longer be detected as a change point. Additionally, the number of detected cases changes rapidly.
  • the detection time is temporary in this way, it can be determined that the content of the map information is unlikely to be different from the actual situation.
  • the number of detections increases slowly, and the number of cases continues to be high. In this case, it can be determined that the content of the map information is likely to be different from the actual situation.
  • the evaluation unit 430 identifies the number of detections for each detection time (that is, the number of change point information acquired by the change point information acquisition unit 410). Then, it is determined whether the state in which the number of detections is greater than or equal to a predetermined number continues for a predetermined time or longer.
  • the fourth condition is a condition regarding duration, and is that the state in which the number of detections is equal to or greater than a predetermined number N1 continues for a predetermined time or longer.
  • the evaluation unit 430 counts the number of pieces of acquired change point information for each point. The evaluation unit 430 then determines whether the number of acquired change point information is equal to or greater than a predetermined number N2 .
  • the fifth condition is a condition regarding the number of detections, and is that the number of acquired change point information is equal to or greater than a predetermined number N2 .
  • the evaluation unit 430 calculates the average value for each point of reliability R cp linked to the plurality of acquired change point information. Then, it is determined whether the calculated average value is greater than or equal to a predetermined value.
  • the sixth condition is a condition regarding reliability, and is that the average value of reliability R cp is equal to or greater than a predetermined value.
  • the change point extraction unit 440 extracts highly reliable change points from the plurality of estimated change points based on the evaluation result by the evaluation unit 430. Then, the maintenance point registration unit 450 registers the extracted highly reliable change points in the database as points that require maintenance of the ND map. At this time, the maintenance priority may be set higher for a point with a larger number of acquired change point information. By checking this database when updating the ND map, it is possible to understand the points that need to be re-measured on a priority basis, leading to the realization of an accurate ND map.
  • the database is stored in a storage section 470 that is accessible from the maintenance point registration section 450.
  • the flag assigning unit 460 assigns a non-recommended flag to the voxel corresponding to the highly reliable change point extracted by the change point extracting unit 440 in the ND map.
  • the hardware configuration of the computer that implements the server 40 is represented, for example, in FIG. 7, similarly to the estimation device 10.
  • the storage device 1080 of the computer 1000 that implements the server 40 stores program modules that implement each functional component of the server 40 according to this embodiment.
  • the storage unit 470 is realized by a storage device 1080.
  • FIG. 19 is a flowchart illustrating the flow of processing performed by the server 40.
  • the change point information acquisition unit 410 acquires the change point information in S410
  • the information development unit 420 develops the change point information into information for each point (S420).
  • the evaluation unit 430 determines whether the fourth condition is satisfied for the point to which the change point information belongs (S430). If the fourth condition is not satisfied (No in S430), the point is not extracted as a highly reliable change point, and the ND map update process (S490) is started.
  • the evaluation unit 430 determines whether the fifth condition is satisfied for the same point (S440). If the fifth condition is not satisfied (No in S440), the point is not extracted as a highly reliable change point, and the ND map update process (S490) is started.
  • the evaluation unit 430 determines whether the sixth condition is satisfied for the same point (S450). If the sixth condition is not satisfied (No in S450), the point is not extracted as a highly reliable change point, and the ND map update process (S490) is started.
  • the change point extraction unit 440 extracts the point as a highly reliable change point (S460).
  • the maintenance point registration unit 450 registers the extracted point in the database as a maintenance point (S470).
  • the flag assigning unit 460 determines that a non-recommended flag should be assigned to the voxel corresponding to the highly reliable change point extracted by the change point extracting unit 440 in the ND map (S480).
  • the server 40 then performs ND map update processing (S490).
  • FIG. 20 is a flowchart illustrating the flow of the ND map update process performed by the server 40 according to this embodiment.
  • the ND map update process is started, it is determined whether there is a voxel to which a non-recommended flag should be added (S491). If it is determined in S480 that a non-recommended flag should be added, it is determined that there is a voxel to which a non-recommended flag should be added (Yes in S491). Then, the flag assigning unit 460 reads the ND map held in the storage unit 470 and assigns a non-recommended flag to the corresponding voxel (S492). Then, the flag adding unit 460 updates the ND map held in the storage unit 470 (S493).
  • S494 it is determined whether new point cloud data for updating the ND map has been acquired. For example, when new point cloud data is obtained by a measurement maintenance vehicle or the like, the data is stored in the storage unit 470. Therefore, it can be determined whether new point cloud data has been acquired based on whether new data is stored in the storage unit 470. If new data has been acquired (Yes in S494), the server 40 uses the acquired point cloud data to create voxel data within the ND map. Then, the deprecation flag of the newly created voxel is cleared (S495). The flag adding unit 460 updates the ND map held in the storage unit 470 (S496), and the ND map update process ends. If new data has not been acquired (No in S494), no new voxel data is created and the ND map update process ends.
  • the ND map held in the storage unit 470 is acquired by the estimation device 10 and used for estimating the self-position of the mobile object 20.
  • the present embodiment shows an example in which the estimating device 10 identifies the estimated change point and transmits the information to the server 40
  • the distribution of processing between the estimating device 10 side and the server 40 side is not particularly limited.
  • the point cloud data obtained by the sensor 210 of the moving object 20 and the movement information of the moving object 20 may be transmitted from the estimation device 10 to the server 40, and the estimated change point may be specified by the server 40.
  • the server 40 will also function as the estimation device 10.
  • the estimation device 10 may perform up to any stage of the change point detection processing, the necessary information may be transmitted from the estimation device 10 to the server 40, and the remaining processing may be performed by the server 40.
  • the change point detection process is performed together with the self position estimation by the self position estimation unit 140, but the change point detection process may be performed separately from the self position estimation process and after the fact. good.
  • the first acquisition unit 110 may acquire the point cloud data by reading point cloud data that has been acquired in advance by the sensor 210 and held in the storage unit. Further, information indicating the speed of the moving object 20 at each time, etc. may also be once held in the storage unit, and the estimation device 10 may read and use it.
  • FIG. 21 is a flowchart illustrating the flow of change point detection processing performed by the estimation device 10 according to the second embodiment.
  • the estimation device 10 according to the present embodiment is the same as the estimation device 10 according to the first embodiment except for the points described below.
  • the estimating unit 190 calculates the strength of correlation between the ratio value of the first region after shifting and the ratio value of the second region with respect to a plurality of shift amounts, and calculates the strength of the correlation. Identify the maximum shift amount.
  • the estimation unit 190 then identifies the estimated change point by comparing the identified shift amount and the reference value.
  • the reference value is determined based on the distance between the first region and the second region.
  • the estimation unit 190 uses a cross-correlation function to identify the estimated change point instead of using the first index and second index described in the first embodiment.
  • the dividing unit 150 divides the point cloud data into data of a plurality of regions, similarly to S801 described in the first embodiment. Then, the ratio value calculation unit 170 calculates the DDAR of each area (S811). Further, the ratio value calculation unit 170 performs the above-described filter processing and normalization on the DDAR.
  • the shift unit 191 calculates a cross-correlation function between the DDAR of the first region and the DDAR of the second region.
  • the shift unit 191 may or may not invert the DDAR of the first region and the DDAR of the second region before calculating the cross-correlation function.
  • the shift unit 191 calculates the cross-correlation function C( ⁇ ) using equation (7).
  • f(t) is the DDAR of the first region
  • g(t) is the DDAR of the second region
  • ⁇ is the amount of time shift.
  • the time shift amount ⁇ can be converted into the position shift amount D by multiplying by the speed v of the moving body 20.
  • the shift unit 191 can acquire the speed v at the timing (for example, time) at which the point group data that is the source of the DDAR is obtained from a speed sensor provided in the moving body 20.
  • FIG. 22 is a graph illustrating the cross-correlation function C( ⁇ ). In this figure, high correlation values are obtained at the points marked with circles. If the shift amount at this time is appropriate in light of the area interval between the first area and the second area, it can be determined that the result is due to a change point.
  • the correlation determination unit 192 specifies the maximum correlation value Cmax in the cross-correlation function, and further shifts the correlation value when the correlation value reaches the maximum correlation value Cmax . Determine the amount.
  • the correlation determining unit 192 determines whether the seventh condition and the eighth condition described below are satisfied.
  • the seventh condition is that the maximum correlation value C max is greater than the threshold Th5. Further, the eighth condition is that the following
  • D is the above-mentioned positional shift amount D
  • L is the area interval between the first area and the second area.
  • the seventh condition indicates that the magnitude of the correlation is large enough to estimate the existence of a change point.
  • the eighth condition indicates that the difference between the characteristics of the DDAR of the first region and the DDAR of the second region is appropriate in light of the actual positional relationship of the regions.
  • a region interval L between the first region and the second region is predetermined.
  • the change point detection process ends.
  • the change point specifying unit 193 specifies the position of the estimated change point.
  • the reliability calculation unit 180 calculates the reliability R cp of the estimated change point (S815).
  • the change point identification unit 193 identifies the peak timing of DDAR in the first region or the second region. Then, the change point specifying unit 193 can specify a position in the direction of the area from the mobile body 20 as the position of the estimated change point, based on the estimated self-position of the mobile body 20 at that timing.
  • the reliability calculation unit 180 can calculate the reliability R cp using the maximum correlation value C max and the value of
  • d and e are predetermined coefficients. d is, for example, 0.1 or more and 1.0 or less, and e is, for example, 5.0 or more and 10.0 or less. That is, the reliability calculation unit 180 increases the reliability R cp as the maximum correlation value C max is larger, and increases the reliability R cp as the value of
  • R cp4 and R cp5 which are the calculation results of equations (8) and (9), both have values within the range of 0 to 1. Therefore, as both R cp4 and R cp5 are larger, the reliability R cp also becomes larger and approaches 1. Further, if either R cp4 or R cp5 is close to 0, the reliability R cp becomes small.
  • the method according to this embodiment also allows for accurate estimation of change points.
  • Example 1 In order to confirm the effect of change point detection according to Example 1, voxel data of a certain building was deleted from the ND map to generate an ND map that did not match the actual environment.
  • FIG. 23 is a diagram showing the original ND map and the ND map with change points generated.
  • Information on buildings that existed in the area surrounded by a white oval on the original ND map was deleted. Since a building actually exists, data obtained by measuring that building is acquired. However, since the ND map does not have voxel data of buildings, the correspondence ratio decreases. This situation is equivalent to a situation where a new building has been constructed but is not reflected on the ND map.
  • FIGS. 24(a) to 24(k) are diagrams showing various data obtained when a vehicle travels counterclockwise on a course that goes around the road shown in FIG. 23.
  • NDT scan matching was performed using the vehicle position and orientation obtained using RTK-GPS (Real Time Kinematic Global Positioning System) as a reference for evaluation. It is a graph showing the difference value from the estimated self-position which is the result. From these results, it can be seen that although the voxel data of buildings was deleted in the ND map, the accuracy of self-position estimation has hardly deteriorated.
  • RTK-GPS Real Time Kinematic Global Positioning System
  • Figures 24(e) to 24(g) show the total number of data points generated by the lidar mounted on the vehicle after downsampling the point cloud data, and the data points associated with the voxels of the ND map, respectively. , and the overall ratio value DAR. It can be seen that the DAR decreases slightly in the 30 to 50 [s] portion.
  • FIG. 25 is a diagram showing the relationship between the moving object 20 (vehicle) and regions (1) to (8) in this experimental example.
  • FIGS. 24(h) to 24(k) each show the DDAR of each area on the right side when viewed from the vehicle. It can be seen that the part where the value is decreasing is gradually moving.
  • FIGS. 26(a) to 26(c) are diagrams showing the results of performing the filter processing, normalization processing, and limit processing described in Example 1 on each DDAR, respectively. That is, FIG. 26(a) shows the data obtained by performing filter processing on each of FIG. 24(h) to FIG. 24(k), and FIG. 26(b) shows the data obtained by performing filter processing on each of the data in FIG. 26(a). 26(c) shows data obtained by performing limit processing on each data in FIG. 26(b). It can be seen that these processes relatively increase the contrast in the portion where the DDAR decreases due to the difference between the point cloud data and the voxel data. Of these, DDAR (5) and DDAR (8), which have a large number of data points and clear change characteristics, were used for subsequent processing.
  • FIGS. 27(a) to 27(e) are diagrams showing each data in the process of change point detection processing.
  • FIG. 27(a) is a graph in which the time axis of DDAR (5) in FIG. 26(c) is shifted based on the vehicle speed and the area interval (30 m).
  • FIG. 27(b) is an inverted version of the graph of FIG. 27(a).
  • FIG. 27(c) is an inverted version of the graph of DDAR(8) in FIG. 26(c).
  • FIGS. 27(d) and 27(e) are graphs showing the first index M and second index S calculated using the data shown in FIGS. 27(b) and 27(c), respectively.
  • W t calculated based on the first index M was 10.792 [s], and W d was 66.714 [m].
  • the threshold Th1 was set to 0.04.
  • N was 131
  • M ave was calculated to be 0.128.
  • S dev calculated based on the second index S was 0.071.
  • FIG. 28 is a diagram showing the DDAR of each area on the left side when viewed from the vehicle. Both show data after filtering, normalization, and limit processing. Using these data, we also examined the area on the left side as seen from the vehicle. On the left side, the actual environment matches the ND map, so the correspondence ratio between areas (1) to (4) is high.
  • FIGS. 29(a) to 29(e) are diagrams showing each data in the process of change point detection processing.
  • FIG. 29(a) is a graph in which the time axis of DDAR (1) in FIG. 28 is shifted based on the speed of the vehicle and the area interval (30 m).
  • FIG. 29(b) is an inverted version of the graph of FIG. 29(a).
  • FIG. 29(c) is an inverted version of the graph of DDAR(4) in FIG. 28.
  • FIGS. 29(d) and 29(e) are graphs showing the first index M and second index S calculated using the data shown in FIGS. 29(b) and 29(c), respectively.
  • Example 2 The change point detection process of Example 2 was performed using the same ND map and point cloud data as in Experimental Example 1.
  • FIG. 30 is a diagram showing the cross-correlation function C( ⁇ ) calculated using DDAR(5) and DDAR(8) shown in FIG. 26(c). This figure shows the DDAR(5) and DDAR(8) together inverted.
  • the maximum correlation value C max is marked with a circle.
  • the area distance L between area (5) and area (8) was 30 m, and
  • R cp was calculated using the calculated C max and
  • FIG. 31 is a diagram showing the cross-correlation function C( ⁇ ) calculated using DDAR(1) and DDAR(4) shown in FIG. 28. This figure shows DDAR(1) and DDAR(4) together inverted.
  • the maximum correlation value C max is marked with a circle.
  • the area distance L between area (1) and area (4) was 30 m, and
  • Estimation device 20 Mobile object 40 Server 50 System 110 First acquisition section 130 Second acquisition section 140 Self-position estimation section 150 Division section 170 Ratio value calculation section 180 Reliability calculation section 190 Estimation section 410 Change point information acquisition section 420 Information development Section 430 Evaluation section 440 Change point extraction section 450 Maintenance required point registration section 460 Flag assignment section 470 Storage section 1000 Computer

Landscapes

  • Engineering & Computer Science (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Analytical Chemistry (AREA)
  • Chemical & Material Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Electromagnetism (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Ecology (AREA)
  • Mathematical Physics (AREA)
  • Business, Economics & Management (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • Databases & Information Systems (AREA)
  • Navigation (AREA)
  • Image Analysis (AREA)
PCT/JP2023/008799 2022-03-14 2023-03-08 推定装置、システム、推定方法、およびプログラム Ceased WO2023176621A1 (ja)

Priority Applications (5)

Application Number Priority Date Filing Date Title
EP23770577.7A EP4495918A1 (en) 2022-03-14 2023-03-08 Estimation device, system, estimation method, and program
US18/845,364 US20250189673A1 (en) 2022-03-14 2023-03-08 Estimation apparatus, estimation system, estimation method, and non-transitory storage medium
CN202380026034.5A CN118974517A (zh) 2022-03-14 2023-03-08 推定装置、系统、推定方法及程序
JP2024507802A JPWO2023176621A1 (https=) 2022-03-14 2023-03-08
JP2025122217A JP2025160287A (ja) 2022-03-14 2025-07-22 推定装置、システム、推定方法、およびプログラム

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022-038992 2022-03-14
JP2022038992 2022-03-14

Publications (1)

Publication Number Publication Date
WO2023176621A1 true WO2023176621A1 (ja) 2023-09-21

Family

ID=88023135

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/008799 Ceased WO2023176621A1 (ja) 2022-03-14 2023-03-08 推定装置、システム、推定方法、およびプログラム

Country Status (5)

Country Link
US (1) US20250189673A1 (https=)
EP (1) EP4495918A1 (https=)
JP (2) JPWO2023176621A1 (https=)
CN (1) CN118974517A (https=)
WO (1) WO2023176621A1 (https=)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012118666A (ja) * 2010-11-30 2012-06-21 Iwane Laboratories Ltd 三次元地図自動生成装置
WO2018180338A1 (ja) 2017-03-30 2018-10-04 パイオニア株式会社 情報処理装置、サーバ装置、制御方法、プログラム及び記憶媒体
WO2021084896A1 (ja) * 2019-11-01 2021-05-06 株式会社デンソー 同期装置、同期方法、および同期プログラム
WO2021112177A1 (ja) 2019-12-04 2021-06-10 パイオニア株式会社 情報処理装置、制御方法、プログラム及び記憶媒体
JP2022016908A (ja) * 2020-07-13 2022-01-25 フォルシアクラリオン・エレクトロニクス株式会社 俯瞰画像生成装置、俯瞰画像生成システム及び自動駐車装置
JP2022022287A (ja) * 2017-05-31 2022-02-03 パイオニア株式会社 地図作成装置、制御方法、プログラム及び記憶媒体
JP2022038992A (ja) 2020-08-27 2022-03-10 住友電気工業株式会社 光トランシーバおよびその制御方法

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022009602A1 (ja) * 2020-07-08 2022-01-13 ソニーグループ株式会社 情報処理装置、情報処理方法、及びプログラム
CN116508084A (zh) * 2020-10-28 2023-07-28 三菱电机株式会社 道路壁形状推测装置及道路壁形状推测方法
US11608084B1 (en) * 2021-08-27 2023-03-21 Motional Ad Llc Navigation with drivable area detection
US12145617B2 (en) * 2021-10-28 2024-11-19 Nvidia Corporation 3D surface reconstruction with point cloud densification using artificial intelligence for autonomous systems and applications

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012118666A (ja) * 2010-11-30 2012-06-21 Iwane Laboratories Ltd 三次元地図自動生成装置
WO2018180338A1 (ja) 2017-03-30 2018-10-04 パイオニア株式会社 情報処理装置、サーバ装置、制御方法、プログラム及び記憶媒体
JP2022022287A (ja) * 2017-05-31 2022-02-03 パイオニア株式会社 地図作成装置、制御方法、プログラム及び記憶媒体
WO2021084896A1 (ja) * 2019-11-01 2021-05-06 株式会社デンソー 同期装置、同期方法、および同期プログラム
WO2021112177A1 (ja) 2019-12-04 2021-06-10 パイオニア株式会社 情報処理装置、制御方法、プログラム及び記憶媒体
JP2022016908A (ja) * 2020-07-13 2022-01-25 フォルシアクラリオン・エレクトロニクス株式会社 俯瞰画像生成装置、俯瞰画像生成システム及び自動駐車装置
JP2022038992A (ja) 2020-08-27 2022-03-10 住友電気工業株式会社 光トランシーバおよびその制御方法

Also Published As

Publication number Publication date
EP4495918A1 (en) 2025-01-22
US20250189673A1 (en) 2025-06-12
JPWO2023176621A1 (https=) 2023-09-21
CN118974517A (zh) 2024-11-15
JP2025160287A (ja) 2025-10-22

Similar Documents

Publication Publication Date Title
US11719788B2 (en) Signal processing apparatus, signal processing method, and program
Kellner et al. Instantaneous lateral velocity estimation of a vehicle using Doppler radar
US10260892B2 (en) Data structure of environment map, environment map preparing system and method, and environment map updating system and method
AU2018282302B2 (en) Integrated sensor calibration in natural scenes
US10024965B2 (en) Generating 3-dimensional maps of a scene using passive and active measurements
CN112513679B (zh) 一种目标识别的方法和装置
US12298141B2 (en) Efficient map matching method for autonomous driving and apparatus thereof
KR101632486B1 (ko) 레이저 거리 센서를 이용한 도로의 연석 추출 방법 및 도로의 연석 정보를 이용한 이동 로봇의 위치 측정 방법
CN114266859B (zh) 数据处理方法、装置、设备和存储介质
CN112731337B (zh) 地图构建方法、装置和设备
Konrad et al. Localization in digital maps for road course estimation using grid maps
CN116263492A (zh) 用于识别由基于互易性的传感器欺骗产生的幽灵对象的雷达反欺骗系统
CN113971697A (zh) 一种空地协同车辆定位定向方法
US11933884B2 (en) Radar image processing device, radar image processing method, and storage medium
EP4296615A1 (en) Distance measuring method and device
WO2023176621A1 (ja) 推定装置、システム、推定方法、およびプログラム
Lee et al. Fast point cloud feature extraction for real-time slam
CN118463965B (zh) 定位精度评价方法、装置以及车辆
CN116359928A (zh) 基于预置地图的终端定位方法和智能汽车
KR102019594B1 (ko) 저해상도 카메라를 위한 보정 기법을 이용한 차량 간 거리 추정 방법 및 그 장치
CN116380004B (zh) 一种物体测距的方法、装置、设备及存储介质
US12469111B2 (en) Method and system for using multi-sensor consensus to filter-out artifacts from point cloud data
RU2798739C1 (ru) Способ трекинга объектов на этапе распознавания для беспилотных автомобилей
CN118942068B (zh) 一种道路坡度确定方法、装置、车辆以及可读存储介质
CN114332185B (zh) 一种路沿检测方法、装置、智能车和计算机可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23770577

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202380026034.5

Country of ref document: CN

WWE Wipo information: entry into national phase

Ref document number: 18845364

Country of ref document: US

ENP Entry into the national phase

Ref document number: 2024507802

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 2023770577

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2023770577

Country of ref document: EP

Effective date: 20241014

WWP Wipo information: published in national office

Ref document number: 18845364

Country of ref document: US