WO2024084601A1 - 変化検出方法、変化検出システム及び変化検出装置 - Google Patents

変化検出方法、変化検出システム及び変化検出装置 Download PDF

Info

Publication number
WO2024084601A1
WO2024084601A1 PCT/JP2022/038830 JP2022038830W WO2024084601A1 WO 2024084601 A1 WO2024084601 A1 WO 2024084601A1 JP 2022038830 W JP2022038830 W JP 2022038830W WO 2024084601 A1 WO2024084601 A1 WO 2024084601A1
Authority
WO
WIPO (PCT)
Prior art keywords
point
coordinate
feature amount
point cloud
coordinates
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/038830
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
雅也 藤若
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Priority to JP2024551110A priority Critical patent/JPWO2024084601A1/ja
Priority to PCT/JP2022/038830 priority patent/WO2024084601A1/ja
Publication of WO2024084601A1 publication Critical patent/WO2024084601A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01V—GEOPHYSICS; GRAVITATIONAL MEASUREMENTS; DETECTING MASSES OR OBJECTS; TAGS
    • G01V8/00—Prospecting or detecting by optical means
    • G01V8/10—Detecting, e.g. by using light barriers
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Definitions

  • the present invention relates to a change detection method, a change detection system, and a change detection device.
  • Patent Document 1 describes a technology that compares previously captured images with currently captured images to detect whether any changes have occurred in the target area. At this time, the target data for the changed area caused by the work is excluded from the output.
  • Patent Document 2 describes a technology that acquires three-dimensional distance data and uses a three-dimensional polar coordinate grid map to detect the presence or absence of an object based on the acquired three-dimensional distance data.
  • Patent Documents 1 and 2 disclose how to compare images and detect the presence or absence of objects, but do not mention this problem and are not able to solve it.
  • the objective of this disclosure is to provide a change detection method, a change detection system, and a change detection device that can improve the accuracy of detecting changes between point clouds, even when the point clouds being compared do not represent accurate information.
  • a change detection method is executed by a computer, and calculates a first feature of a first coordinate in a first point cloud using information about the presence of a point in each of the first small regions in a first region that is composed of a plurality of first small regions and includes the first coordinate, calculates a second feature of a second coordinate in a second point cloud that corresponds to the first coordinate using information about the presence of a point in each of the second small regions in a second region that is composed of a plurality of second small regions and includes the second coordinate, and calculates a second feature of a second coordinate in a second point cloud that corresponds to the first coordinate using information that indicates that a point is present, that a point is not present, or that the presence of a point is unknown in each of the second small regions in a second region that is composed of a plurality of second small regions and includes the second coordinate, and determines whether a change in the presence or absence of a point has occurred between the first coordinate and the second coordinate using the first feature and the
  • the change detection system includes a first feature calculation means for calculating a first feature of a first coordinate in a first point cloud using information on the presence of a point in each of the first small regions in a first region that is composed of a plurality of first small regions and includes the first coordinates; a second feature calculation means for calculating a second feature of a second coordinate in a second point cloud that corresponds to the first coordinate using information on the presence of a point in each of the second small regions in a second region that is composed of a plurality of second small regions and includes the second coordinates, indicating that a point is present, that a point is not present, or that the presence of a point is unknown in each of the second small regions; and a determination means for determining whether a change in the presence or absence of a point has occurred between the first coordinates and the second coordinates using the first feature and the second feature.
  • a change detection device includes a first feature calculation means for calculating a first feature of a first coordinate in a first point cloud using information on the presence of a point in each of the first small regions in a first region that is composed of a plurality of first small regions and includes the first coordinates; a second feature calculation means for calculating a second feature of a second coordinate in a second point cloud that corresponds to the first coordinate using information indicating that a point is present, that a point is not present, or that the presence of a point is unknown in a second region that is composed of a plurality of second small regions and includes the second coordinates; and a determination means for determining whether a change in the presence or absence of a point has occurred between the first coordinates and the second coordinates using the first feature and the second feature.
  • the present disclosure provides a change detection method, a change detection system, and a change detection device that can improve the accuracy of detecting changes between point clouds, even when the point clouds being compared do not show accurate information.
  • FIG. 1 is a block diagram showing an example of a change detection device according to a first embodiment
  • 4 is a diagram for explaining calculations performed by a feature amount calculation unit according to the first embodiment
  • FIG. 4 is a flowchart showing an example of a representative process of the change detection device according to the first embodiment
  • 1 is a block diagram showing an example of a change detection system according to a first embodiment
  • FIG. 11 is a block diagram showing an example of a monitoring system according to a second embodiment.
  • FIG. 11 is a block diagram showing an example of a center server according to a second embodiment. 13 shows an example of a reference point group according to the second embodiment. 13 shows an example of an input point group according to the second embodiment.
  • FIG. 11 is a diagram showing a polar coordinate system centered on the coordinates that are the target of calculation of the reference point group in the second embodiment.
  • 13 is an example of a spatial feature amount calculated for the coordinates of a reference point group according to the second embodiment.
  • 13 is an example of a spatial feature calculated for coordinates of an input point group according to the second embodiment.
  • FIG. 9C is a diagram for explaining equation (5) in the example of FIGS. 9B and 9C.
  • 13 is a flowchart showing an outline of an example of processing by a center server according to the second embodiment; 13 is a flowchart illustrating an example of detailed processing of the center server. 13 is a flowchart showing another example of detailed processing of the center server.
  • 1 shows an example of an image where changes are detected using the direct comparison method. 13 shows an example of an image in which a change is detected using the technique of the present disclosure.
  • FIG. 11 is a block diagram showing another example of the center server according to the second embodiment.
  • 13 is a flowchart showing an outline of an example of processing by a center server according to the second embodiment; 13 is a flowchart illustrating an example of detailed processing of the center server.
  • FIG. 13 is a flowchart showing another example of detailed processing of the center server.
  • FIG. 11 is a block diagram showing another example of the center server according to the second embodiment.
  • FIG. 11 is a block diagram showing another example of the center server according to the second embodiment.
  • 13 is a flowchart showing an outline of an example of processing by a center server according to the second embodiment;
  • 13 is a flowchart showing an outline of an example of processing by a center server according to the second embodiment;
  • FIG. 2 is a block diagram showing an example of a hardware configuration of an apparatus according to each embodiment.
  • FIG. 1 is a block diagram showing an example of a change detection device.
  • the change detection device 10 includes a first feature amount calculation unit 11, a second feature amount calculation unit 12, and a determination unit 13. Each unit (means) of the change detection device 10 is controlled by a control unit (controller) (not shown). Each unit will be described below.
  • the first feature amount calculation unit 11 calculates the first feature amount of the first coordinate in the first point cloud.
  • the first point cloud is first data indicating the presence or absence of a point at each coordinate in a predetermined area.
  • the first point cloud represents, for example, the shape of an object in a three-dimensional space, and an example thereof is data obtained by using a sensor and visualizing an object at a predetermined location.
  • the sensor used may be, for example, a range sensor or an imaging element such as a camera. When a range sensor is used, the first point cloud becomes mapping data obtained by measuring and visualizing an object at a predetermined location.
  • a specific example of the range sensor is LiDAR (Light Detection And Ranging) using light detection and ranging.
  • Both 3DLiDAR and 2DLiDAR can be used as LiDAR.
  • An example of the case where 3DLiDAR is used will be described later in the second embodiment.
  • the first point cloud becomes mapping data generated based on a two-dimensional image of a predetermined location.
  • the above-mentioned measurements or photographs may be actual measurements or photographs, or may be virtual measurements or photographs performed by a computer.
  • the objects indicated by the first point cloud are not limited to these.
  • the change detection device 10 may obtain the first point cloud data from outside the change detection device 10, or may generate the first point cloud data within the change detection device 10.
  • the first feature amount calculation unit 11 calculates the first feature amount as follows.
  • the first feature amount calculation unit 11 defines a first area that is composed of a plurality of first small areas and includes first coordinates for the first point group.
  • the first feature amount calculation unit 11 then calculates the first feature amount by using information about the presence of a point in each first small area.
  • the information about the presence of a point may be, for example, information indicating whether a point is present or not present in each first small area.
  • the information about the presence of a point may be information indicating whether a point is present, not present, or the presence of a point is unknown in each first small area. The definition of the presence of a point being unknown will be described later.
  • the information about the presence of a point may be generated by the first feature amount calculation unit 11 by analyzing the acquired first point group, or may be included in information acquired by the first feature amount calculation unit 11 from outside.
  • FIG. 2 is a diagram for explaining the calculations of the first feature amount calculation unit 11.
  • the first point group is G1
  • the first coordinate is FC
  • the first region including FC is R1.
  • Region R1 can be divided into small regions SR1.
  • the presence of points in point group G1 is indicated by black circles, and the absence of points is indicated by white circles.
  • the first feature amount calculation unit 11 defines a region R1 that is composed of multiple small regions SR1 and that includes the coordinates FC, for which the feature amount is to be calculated.
  • the first feature amount calculation unit 11 then calculates the feature amount for the coordinates FC by using information about the presence of points in each small region SR1 of the region R1.
  • the calculated first feature amount may be expressed as a scalar amount or a vector amount.
  • the second feature amount (described below) corresponding to the first feature amount is also expressed as a scalar amount. Then, in the judgment process of the judgment unit 13 described below, the first feature amount is compared with the corresponding second feature amount to execute the judgment process described below.
  • a first feature amount is expressed as a vector amount
  • the corresponding second feature amount is also expressed as a vector amount. Then, in the judgment process of the judgment unit 13 described below, each element in the first feature amount is compared with each element in the second feature amount corresponding to each element of the first feature amount, thereby executing the judgment process described below.
  • a specific example in which a feature amount is expressed as a vector amount will be described in detail in the second embodiment.
  • FIG. 2 shows an example in which information on the presence or absence of one point is associated with one small region SR1.
  • information on the presence or absence of multiple points may be associated with one small region SR1.
  • the number of small regions SR1 contained in region R1 and the shapes of region R1 and small regions SR1 are arbitrary.
  • the second feature calculation unit 12 calculates a second feature of a second coordinate in the second point cloud, which is different from the first point cloud.
  • the second point cloud is second data indicating the presence or absence of a point at each coordinate in a specified region, and an example thereof is similar to that of the first point cloud.
  • the change detection device 10 may obtain the second point cloud data from outside the change detection device 10, or may generate the second point cloud data within the change detection device 10.
  • first point cloud and the second point cloud are point clouds to be compared, and are, for example, mapping data in which the same location is measured. Also, the first coordinates and the second coordinates are compared, and, for example, the first coordinates and the second coordinates may indicate the same position, but the relationship between the first coordinates and the second coordinates is not limited to this.
  • the change detection device 10 calculates feature amounts for a first coordinate in a first point cloud and a second coordinate in a second point cloud that corresponds to the first coordinate by using a first feature amount calculation unit 11 and a second feature amount calculation unit 12. By using these feature amounts, it is possible to determine whether or not a change in the presence or absence of a point has occurred between the first coordinate and the second coordinate.
  • the second feature amount calculation unit 12 calculates the second feature amount as follows.
  • the second feature amount calculation unit 12 defines a second area, which is composed of a plurality of second small areas and includes second coordinates, for the second point group.
  • the definition of this second area and the second small areas is the same as the method executed by the first feature amount calculation unit 11, and is as described in the example of FIG. 2.
  • the second feature calculation unit 12 calculates the second feature using information indicating that in each second sub-region in this second region, a point is present, a point is not present, or the presence of a point is unknown. "Presence of a point is unknown" indicates that although the presence or absence of a point is defined in the second sub-region in the second point cloud acquired by the second feature calculation unit 12, it is considered that it is unknown whether a point actually exists.
  • the second feature calculation unit 12 calculates the feature for that sub-region to be different from either the case where a point exists or the case where a point does not exist.
  • the existence of a point, the absence of a point, or the unknown existence of a point can be defined, for example, as follows:
  • the second point cloud is mapping data acquired by measurement using a range sensor. If there is a point in the second small region, the second small region is defined as having a point present. On the other hand, if there is no point in the second small region, it is determined whether the second small region is located between the position where the point exists in the second point cloud at the time of measurement and the position of the range sensor. If the second small region is located between the position where the point exists in the second point cloud at the time of measurement and the position of the range sensor, it is defined as not having a point in the second small region. On the other hand, if the second small region is not located between the position where the point exists in the second point cloud and the position of the range sensor, the presence of the point in the second small region is defined as unknown.
  • This definition is based on the assumption that when a range sensor acquires mapping data, light should be incident on the range sensor from an object shown in the mapping data, and no object should exist between the object and the range sensor. Ray tracing technology can be applied to this definition.
  • This is a particularly effective definition for improving detection accuracy, for example, when the density of points in the second point cloud is sparser than the density of points in the first point cloud.
  • a location where there is no point in the second point cloud may not only be a location where there is no point in reality, but also a location where an object exists in the measurement but is not recorded as data. In this case, when it is not possible to determine that there is no point in reality, it is preferable to define the location where there is no point as a location where the presence of a point is unknown.
  • N the number of points in the second small region.
  • M the number of times the light passes through the second small region.
  • the presence, absence, or unknownness of a point can be defined according to the values of N-M and N+M.
  • Th1 is an integer equal to or greater than 0
  • the presence of a point in the second small region is defined as being unknown. This is because the number of points in the second small region in the second point group is small, and the number of cases in which light from other points in the second point group passes through the second small region is also small, making it difficult to determine whether or not a point exists in the second small region.
  • the presence or absence of a point in the second small region is defined depending on whether or not the value of N-M is equal to or greater than a threshold Th2 (Th2 is an integer, and is 0 as an example, but is not limited to this).
  • Th2 is an integer, and is 0 as an example, but is not limited to this.
  • the information shown above about whether a point exists, does not exist, or is unknown may be generated by the second feature calculation unit 12 by analyzing the acquired second point cloud, or may be included in information acquired by the second feature calculation unit 12 from outside.
  • the determination unit 13 uses the first feature calculated by the first feature calculation unit 11 and the second feature calculated by the second feature calculation unit 12 to determine whether or not a change in the presence or absence of a point has occurred between the first coordinate and the corresponding second coordinate.
  • the determination method of the determination unit 13 may be performed by any calculation process, such as arithmetic operations, using the first feature amount and the second feature amount, or may be performed by an algorithm based on a predefined rule base. For example, when the first feature amount and the second feature amount are scalar amounts, the determination unit 13 may calculate the difference between the first feature amount and the second feature amount and determine whether the difference is equal to or greater than a threshold value.
  • the determination unit 13 determines that a change in the presence or absence of a point has occurred between the first coordinates and the second coordinates, and when the difference is less than the threshold value, the determination unit 13 determines that a change in the presence or absence of a point has occurred between the first coordinates and the second coordinates.
  • the determination unit 13 may compare corresponding elements of each vector amount and determine whether a change in the presence or absence of a point has occurred between the first coordinate and the second coordinate based on the comparison results for all elements. For example, when comparing elements, the determination unit 13 may determine whether the elements have the same value or not, or whether the difference value of the elements is larger or smaller than a threshold, and calculate a similarity as a comparison result for all elements based on the determination. If this similarity is equal to or greater than a predetermined threshold, it is determined that there is no change in the presence or absence of a point between the first coordinate and the second coordinate.
  • the judgment method of the judgment unit 13 may be performed using an AI (Artificial Intelligence) model that has been trained in advance, such as a neural network.
  • This learning is performed by inputting teacher data, including information on the first and second feature amounts as samples and information (correct answer label) corresponding to the information, indicating whether a change in the presence or absence of a point has occurred between the first coordinate and the second coordinate.
  • the judgment unit 13 inputs the first feature amount calculated by the first feature amount calculation unit 11 and the second feature amount calculated by the second feature amount calculation unit 12 to the AI model. Based on this input information, the AI model outputs information indicating whether a change in the presence or absence of a point has occurred between the first coordinate and the second coordinate. Even in this way, the judgment unit 13 can execute the judgment process.
  • any technology such as logistic regression or neural network, can be used for training the learning model.
  • the first feature amount calculation unit 11 calculates a first feature amount of a first coordinate in the first point cloud (step S11; first feature amount calculation step).
  • the second feature amount calculation unit 12 calculates a second feature amount of a second coordinate in the second point cloud (step S12; second feature amount calculation step).
  • the determination unit 13 uses the first feature amount and the second feature amount to determine whether or not a change in the presence or absence of a point has occurred between the first coordinate and the second coordinate (step S13; determination step). Note that either the process of step S11 or the process of S12 may be executed first, or both processes may be executed in parallel.
  • the second feature calculation unit 12 calculates the second feature to reflect that state. Then, the determination unit 13 makes a determination that reflects the first feature and the second feature. Therefore, for a location where erroneous point information is indicated, such as a point not existing (or existing) in the second point cloud even though a point actually exists (or does not exist) in the second point cloud, the determination unit 13 can determine that the presence of that location is unknown. It is estimated that this determination result has a higher accuracy than when the erroneous point information is used as is. Therefore, even if the second point cloud to be compared with the first point cloud does not indicate accurate information, the change detection device 10 can improve the accuracy of detecting changes between the point clouds.
  • the determination unit 13 may perform the above determination at multiple corresponding coordinates in the first point cloud and the second point cloud to detect a change in the presence or absence of points in a predetermined area of the point cloud. For example, the determination unit 13 may perform the above determination for all coordinates of the first point cloud or all coordinates of the second point cloud. This makes it possible to detect a change in the presence or absence of points in the entire first point cloud or the entire second point cloud. Therefore, for example, when the first point cloud and the second point cloud are point cloud data measured at the same location, the change detection device 10 can identify a location that has changed in the two point clouds. The changed location is, for example, a location where an object that existed in one of the two point clouds no longer exists in the other point cloud.
  • the change detection device 10 may further include a detection unit that detects a change in the presence or absence of an object between the first point cloud and the second point cloud based on the above-mentioned determination result of the determination unit 13. A specific detection method will be described later in the second embodiment.
  • the first feature calculation unit 11 may also calculate the first feature using information indicating that in each first small region in the first region, a point is present, a point is not present, or the presence of a point is unknown.
  • the definition of the presence of a point being unknown is as described above. This allows a state in which the presence of a point in the first point cloud is unknown to be reflected in change detection, thereby further improving the change detection accuracy in the change detection device 10.
  • the change detection device 10 may further include an output unit that outputs the determination result of the determination unit 13 to the inside or outside of the change detection device 10.
  • the output unit may visually emphasize the location where the determination unit 13 has determined that a change in the presence or absence of a point has occurred, and output the data of the determination result to the outside of the change detection device 10 (e.g., a monitor) in a visible format such as an image or a point cloud. This process may be performed for all locations where the determination unit 13 has determined that a change in the presence or absence of a point has occurred, thereby making it possible to present to the user the locations where the presence or absence of an object has changed in the two point clouds.
  • Examples of "visual emphasis” include surrounding the location where the presence or absence of a point has changed (or the location where the presence or absence of an object has changed) with a frame, displaying the outline of the location in a color (e.g., red) different from the color of the outline of other objects, blinking the location, filling the location and displaying it as a shadow, and the like, but are not limited to these.
  • the output unit may also output the determination result of the determination unit 13 by sound or the like via a speaker. In this way, the output unit can output an alert by image or sound.
  • the output unit may also output the determination result of the determination unit 13 to another device. Furthermore, the output unit may output the detection result of detecting a change in the presence or absence of an object between the first point cloud and the second point cloud as described above.
  • FIG. 4 is a block diagram showing an example of a change detection system.
  • the change detection system 20 includes a feature amount calculation device 21 and a judgment device 22.
  • the feature amount calculation device 21 includes a first feature amount calculation unit 11 and a second feature amount calculation unit 12, and the judgment device 22 includes a judgment unit 13 and an output unit 14.
  • the first feature amount calculation unit 11 to the judgment unit 13 execute the same processing as that shown in (1A).
  • the judgment unit 13 of the judgment device 22 executes the processing shown in (1A) using the information on the feature amounts.
  • the output unit 14 of the judgment device 22 is the output unit described in (1A), and outputs the judgment result of the judgment unit 13 to the inside or outside of the judgment device 22.
  • the change detection process according to the present disclosure may be realized by a single device as shown in (1A), or may be realized as a system in which the processes to be executed are distributed among multiple devices as shown in (1B).
  • the device configuration shown in (1B) is merely an example.
  • the first device may have a first feature amount calculation unit 11, and the second device may have a second feature amount calculation unit 12 and a determination unit 13.
  • the first device may have an acquisition unit that acquires the first point cloud.
  • three different devices may be provided, each having the first feature amount calculation unit 11, the second feature amount calculation unit 12, and the determination unit 13.
  • each device may further have an acquisition unit that acquires the first point cloud, an acquisition unit that acquires the second point cloud, and an output unit 14.
  • the change detection system 20 may be provided in part or in its entirety in a cloud server built on the cloud, or in other types of virtualized servers generated using virtualization technology, etc. Functions other than those provided in such servers are placed at the edge.
  • the edge is a device placed at or near the site, and is also a device that is close to the terminal in terms of the network hierarchy.
  • Embodiment 2 In the following embodiment 2, a specific example of the change detection method described in embodiment 1 will be disclosed. However, the specific example of the change detection method described in embodiment 1 is not limited to the one shown below. Furthermore, the configurations and processes described below are merely examples, and are not limited to these.
  • Fig. 5 is a block diagram showing an example of a monitoring system.
  • the monitoring system 100 includes a plurality of robots 101A, 101B, and 101C (hereinafter collectively referred to as robots 101), a base station 110, a center server 120, and a reference point group DB 130.
  • robots 101 are provided on the edge side (on-site side) of the monitoring system 100, and the center server 120 is disposed at a position away from the site (on the cloud side).
  • the center server 120 is disposed at a position away from the site (on the cloud side).
  • the robot 101 functions as a terminal that measures a specific location while moving through a site to be monitored in order to inspect infrastructure equipment for failures or abnormalities.
  • the robot 101 is an edge device connected to a network, has a LiDAR 102, and can measure any location.
  • the robot 101 transmits the measured point cloud data to the center server 120 via the base station 110.
  • the robot 101 transmits the point cloud data via a wireless line.
  • the point cloud data may also be transmitted via a wired line.
  • the robot 101 may transmit the point cloud data acquired by measurement using the LiDAR 102 directly to the center server 120, or may perform appropriate preprocessing on the acquired point cloud data before transmitting it to the center server 120.
  • the robot 101 may transmit information indicating the measurement position to the center server 120 together with the point cloud data acquired at that position.
  • the robot 101 has functions such as AMCL (Adaptive Monte Carlo Localization) or SLAM (Simultaneous Localization and Mapping), and estimates its own position using these functions and transmits the information to the center server 120.
  • the robot 101 may obtain information indicating the measurement position using a satellite positioning system function such as GPS (Global Positioning System), and transmit the information to the center server 120.
  • GPS Global Positioning System
  • the movement route and measurement points of the robot 101 may be determined in advance, and the robot 101 and the center server 120 may share the information in advance, so that the center server 120 is aware of the position of the robot 101.
  • the robot 101 may be, for example, an AGV (Automatic Guided Vehicle) that runs under the control of the center server 120, an AMR (Autonomous Mobile Robot) that is capable of autonomous movement, a drone, etc., but is not limited to these.
  • AGV Automatic Guided Vehicle
  • AMR Automatic Mobile Robot
  • the base station 110 transfers the point clouds transmitted from each robot 101 to the center server 120 via the network.
  • the base station 110 is a local 5G (5th Generation) base station, a 5G gNB (next Generation Node B), an LTE eNB (evolved Node B), a wireless LAN access point, etc., but may also be other relay devices.
  • the network is, for example, a core network such as 5GC (5th Generation Core network) or EPC (Evolved Packet Core), the Internet, etc.
  • a server other than the center server 120 may be connected to the base station 110.
  • a MEC (Multi-access Edge Computing) server may be connected to the base station 110.
  • the MEC server can, for example, control the bit rate of the data transmitted by each robot 101 by assigning a bit rate for the data transmitted by each robot 101 to the base station 110 and transmitting this information to each robot 101.
  • the MEC server can transmit information on the bit rate of each robot 101 to the center server 120, allowing the center server 120 to grasp the bit rate information.
  • FIG. 6 is a block diagram showing an example of the center server 120.
  • the center server 120 includes a reference point cloud acquisition unit 121, an input point cloud acquisition unit 122, a reference feature amount calculation unit 123, an input feature amount calculation unit 124, a change detection unit 125, and a detection result generation unit 126.
  • the center server 120 can detect changes and generate comparison results by comparing each piece of point cloud data measured and acquired by the robot 101 with the reference data. Each part of the center server 120 will be described below.
  • the reference point cloud acquisition unit 121 acquires a reference point cloud for comparison with the point cloud acquired by the robot 101 during measurement.
  • the reference point cloud is a point cloud acquired by previously measuring the location measured by the robot 101, and is stored in the reference point cloud DB 130.
  • the reference point cloud is data that represents the shape of an object in three-dimensional space.
  • the center server 120 when the input point cloud acquisition unit 122, which will be described later, acquires the point cloud data transmitted by the robot 101 (hereinafter also referred to as the input point cloud), the center server 120 also acquires information on the measurement positions of the input point cloud. Furthermore, the reference point cloud DB 130 stores the reference point cloud in association with the position information at which the reference point cloud was measured. The reference point cloud acquisition unit 121 compares the measurement position information of the input point cloud with the measurement position information stored in the reference point cloud DB 130. If the comparison results in a match in the measurement position information stored in the reference point cloud DB 130, the reference point cloud acquisition unit 121 acquires the reference point cloud associated with the matching measurement position information. In this way, the reference point cloud acquisition unit 121 acquires the reference point cloud data to be compared with the point cloud measured and acquired by the robot 101 by searching the reference point cloud DB 130.
  • data previously measured and acquired may be stored in the reference point cloud DB 130 as an image rather than as a point cloud.
  • the reference point cloud acquisition unit 121 identifies matching measurement position information by searching the reference point cloud DB 130 described above, it can acquire the reference point cloud by acquiring an image associated with the matching measurement position information and converting the data format of the image into a point cloud.
  • the input point cloud acquisition unit 122 acquires the input point cloud acquired by the robot 101 through measurement via the communication unit of the center server 120. This allows the input point cloud acquisition unit 122 to acquire point cloud data measured in real time. However, if the robot 101 takes an image and transmits the image data to the center server 120, the input point cloud acquisition unit 122 can acquire the input point cloud by converting the data format of the image into a point cloud. Like the reference point cloud, the input point cloud is data that represents the shape of an object in three-dimensional space. However, there are differences between the input point cloud and the reference point cloud, as described below.
  • the first issue is that the point densities in the reference point cloud and the input point cloud may differ, and the second issue is that the measurement position of the robot 101 may deviate from the measurement position of the reference point cloud.
  • Figures 7A and 7B show examples of a reference point cloud and an input point cloud.
  • the reference point cloud in Figure 7A is 3D data measured using a sensor before the robot 101 performs measurements related to the input point cloud. "Before performing measurements related to the input point cloud” may be when the robot is deployed at the site, when the robot is inspected at the site before operating at the site, or before work begins, such as on the morning of an operating day.
  • Figure 7B is 3D data acquired by measurement by the robot 101.
  • Figures 7A and 7B are data measured inside a warehouse, and racks L inside the warehouse are captured as a point cloud.
  • Figure 7B also captures an object OB not in Figure 7A as a point cloud.
  • the reference point cloud T1 has a higher density of points, whereas the input point cloud T2 has a lower density of points. This is because the input point cloud T2 is data measured in real time, and the amount of data is smaller. In other words, compared to the reference point cloud T1, the input point cloud T2 may not record point information at coordinates where an object exists and a point should be located. In addition, the point density may change between the reference point cloud T1 and the input point cloud T2 depending on the measurement environment of the reference point cloud T1 and the input point cloud T2. Thus, in actual use, it is expected that the point density of the reference point cloud and the input point cloud will differ greatly. If these two point clouds are directly compared, it may be difficult to accurately detect changes in the two point clouds. Therefore, it is preferable to be able to improve the accuracy of detecting changes between the point clouds, even if the input point cloud does not show accurate information due to changes in the point cloud density compared to the reference point cloud.
  • FIG. 8A shows an example of a situation in which a reference point cloud is obtained by measurement
  • FIG. 8B shows an example of a situation in which an input point cloud is obtained by measurement
  • a location I1 where factory equipment is located is measured by a dedicated sensor S.
  • This dedicated sensor S obtains a point cloud by using LiDAR.
  • the measurement range of the dedicated sensor S is indicated by G11.
  • Gas tanks T1 and T2 are included within the range of G11.
  • the same location I1 is measured by a LiDAR 102 of a robot 101.
  • the measurement range of the dedicated sensor S is indicated by G12.
  • object L1 which was not included in FIG.
  • the robot 101 is controlled during measurement so that the estimated position of the robot 101 is the same as the measured position of the dedicated sensor S. Ideally, therefore, G11 and G12 are in the same range.
  • the error in the estimation of the robot 101's own position can become large depending on the environment in which the robot 101 is placed. Also, the error can become large when the position of the robot 101 cannot be corrected by the control of the center server 120. In such a case, as shown in FIG. 8B, a deviation occurs between G11 and G12.
  • a deviation in the measured position itself hereinafter also referred to as a deviation in the translation direction
  • a deviation in the measured direction hereinafter also referred to as a deviation in the rotation direction
  • Figure 8C shows the ideal comparison result between the reference point cloud and the input point cloud
  • Figure 8D shows the actual comparison result between the reference point cloud and the input point cloud.
  • the change detection method proposed in this disclosure makes it possible to solve these problems through the process described below.
  • the reference feature calculation unit 123 corresponds to the first feature calculation unit 11 in the first embodiment, and calculates the feature of each coordinate in the reference point group. This feature is expressed as vectorized information on the presence or absence of points at the coordinates to be calculated and at the surrounding coordinates, and will hereafter also be referred to as a spatial feature. Details of the method of calculation of the spatial feature by the reference feature calculation unit 123 will be described below.
  • FIG. 9A shows a polar coordinate system centered on the coordinates to be calculated for the reference point group.
  • the reference feature calculation unit 123 acquires data on a group of points in a spherical region S1 of radius ⁇ centered on coordinate F1 in the reference point group. Then, it calculates and finds parameters (r, ⁇ , ⁇ ) in the polar coordinate system shown in FIG. 9A for each coordinate in the spherical region S1.
  • the reference feature calculation unit 123 divides the spherical region S1 into a plurality of small regions SR1 (first small regions in the first embodiment) so that each coordinate is included within the small region SR1.
  • the spherical region S1 is divided so that the distance r and angle ( ⁇ , ⁇ ) in each small region SR1 are discretized values.
  • the angle ⁇ is divided by 2 ⁇ (rad)
  • the angle ⁇ is divided by ⁇ (rad)
  • the distance r is divided by d.
  • 2 ⁇ is a value equal to or less than ⁇
  • d is a value equal to or less than ⁇ /2.
  • the spherical region S1 is
  • the image is divided into a number of small regions SR1.
  • the number of small regions SR1 is expressed by using a floor function and a ceiling function, respectively.
  • the reference feature calculation unit 123 calculates a vectorized spatial feature by defining the presence or absence of a point in each divided small region SR1 as an element of the spatial feature. For example, a vector representation in which 1 is indicated when a point exists at each coordinate and 2 is indicated when a point does not exist at each coordinate is assumed, but the example of the vector representation is not limited to this.
  • FIG. 9B is an example of spatial features calculated for the coordinates of the reference point group.
  • FIG. 9B in a spherical region S1 with coordinate F1 at center O, the presence of a point in small region SR1 is indicated by a black circle, and the absence of a point is indicated by a white circle.
  • FIG. 9B also shows coordinate H1 on spherical region S1, which will be used as an example when explaining the calculation of spatial features below.
  • the reference feature calculation unit 123 sets a neighborhood area ⁇ 1 of the coordinate F1 in the reference point group.
  • the neighborhood area ⁇ 1 is an area including at least one coordinate other than the coordinate F1, and is set as an area of radius ⁇ 1 in a polar coordinate system centered on the coordinate F1, for example, but the setting of the neighborhood area ⁇ 1 is not limited to this.
  • the reference feature calculation unit 123 then calculates spatial features for each coordinate other than the coordinate F1 included in the neighborhood area ⁇ 1 in the same manner as the calculation of spatial features at the coordinate F1 described above.
  • the reference feature calculation unit 123 defines a spherical area (third area) including the coordinates, which is composed of multiple small areas (third small areas), for each coordinate other than the coordinate F1 included in the neighborhood area ⁇ 1, in the same manner as the spherical area S1 and the small area SR1. Then, the spatial features are calculated using information on the presence or absence of a point in each small area.
  • the input feature calculation unit 124 corresponds to the second feature calculation unit 12 in the first embodiment, and calculates the spatial feature of each coordinate in the input point cloud.
  • This spatial feature is expressed as a vectorized information indicating whether a point exists, does not exist, or whether the existence of a point is unknown at the coordinate to be calculated and its surrounding coordinates.
  • the input feature calculation unit 124 uses the distance r from the center O and the angle ( ⁇ , ⁇ ) as parameters to identify coordinates included in a predetermined region (the second region in the first embodiment) that includes the coordinate F2.
  • the coordinate F2 is a coordinate that is to be compared with the coordinate F1 in terms of spatial features, and here indicates the same coordinate in the reference point cloud and the input point cloud.
  • the input feature calculation unit 124 acquires data on the point cloud in a spherical region S2 of radius ⁇ centered on coordinate F2 in the input point cloud.
  • the size of the spherical region S2 is the same as that of the spherical region S1. Then, it calculates and finds the parameters (r, ⁇ , ⁇ ) in the polar coordinate system shown in Figure 9A for each coordinate in the spherical region.
  • the input feature calculation unit 124 divides the spherical region S2 into a plurality of small regions SR2 (second small regions in the first embodiment) so that each coordinate is included within the small region SR2.
  • the spherical region S1 is divided in the same way as the spherical region S2. That is, the angle ⁇ is divided by 2 ⁇ (rad), the angle ⁇ is divided by ⁇ (rad), and the distance r is divided by d. Note that 2 ⁇ is a value equal to or less than ⁇ , and d is a value equal to or less than ⁇ /2. Therefore, the spherical region S2 is,
  • the input feature amount calculation unit 124 calculates a vectorized spatial feature amount by defining, as an element of the spatial feature amount, information indicating whether a point exists, whether a point does not exist, or whether the existence of a point is unknown in each divided small region SR2. For example, a vector notation is assumed in which 1 is indicated when a point exists in each coordinate, 2 is indicated when a point does not exist, and 0 is indicated when the existence of a point is unknown, but examples of the vector notation are not limited to this.
  • the input feature calculation unit 124 determines information indicating whether a point exists in each small region SR2, whether a point does not exist, or whether the existence of a point is unknown, as follows. If a point exists in a small region SR2, the input feature calculation unit 124 defines the small region SR2 as having a point. On the other hand, if there is no point in the small region SR2, the input feature calculation unit 124 determines whether the small region SR2 is located between the position where the point exists in the input point cloud at the time of measurement and the position of LiDAR 102.
  • the input feature calculation unit 124 defines the small region SR2 as having no point. On the other hand, if the small region SR2 is not located between the position where the point exists in the input point cloud at the time of measurement and the position of LiDAR 102, the input feature calculation unit 124 defines the existence of a point in the small region SR2 as being unknown. This type of ray tracing technique can be applied to the definition.
  • the density of points in the input point cloud is sparser than the density of points in the reference point cloud. Therefore, it is possible that an object that is captured in the reference point cloud will not be captured accurately in the input point cloud, and some points of the object will not be recorded in the input point cloud. Therefore, when it is not possible to be certain that a point does not actually exist, it is preferable to define the presence of a point as unknown for a small region SR2 that does not contain a point.
  • Figure 9C is an example of spatial features calculated for the coordinates of the input point cloud.
  • Figure 9C in a spherical region S2 with coordinate F2 at center O, the presence of a point in small region SR2 is indicated by a black circle, the absence of a point is indicated by a white circle, and the presence of a point is unknown is indicated by a triangle.
  • Figure 9C also shows coordinate H2 on spherical region S2, which will be used as an example when explaining the calculation of spatial features below. Note that coordinates H1 and H2 indicate the same coordinates in the reference point cloud and the input point cloud.
  • the input feature amount calculation unit 124 sets a neighborhood area ⁇ 2 of the coordinate F2 in the input point cloud.
  • the neighborhood area ⁇ 2 is an area including at least one coordinate other than the coordinate F2, and is set as an area of radius ⁇ 2 in a polar coordinate system centered on the coordinate F2, for example, but the setting of the neighborhood area ⁇ 2 is not limited to this.
  • the input feature amount calculation unit 124 calculates the spatial feature amount for each coordinate other than the coordinate F2 included in the neighborhood area ⁇ 2 in the same manner as the calculation of the spatial feature amount for the coordinate F2 shown above.
  • the input feature amount calculation unit 124 defines a spherical area (fourth area) including the coordinates, which is composed of multiple small areas (fourth small areas) in the same manner as the spherical area S2 and the small area SR2. Then, the spatial feature amount is calculated by using information indicating that a point exists in each small area, that a point does not exist, or that the existence of a point is unknown.
  • the size of the neighborhood region ⁇ 2 in the input point cloud may be the same as or different from the size of the neighborhood region ⁇ 1 in the reference point cloud. That is, in this example, the radius ⁇ 1 may be the same length as the radius ⁇ 2, or may be a different length.
  • the reference feature calculation unit 123 and the input feature calculation unit 124 calculate the above-mentioned spatial features at each coordinate of the reference point group and the input point group.
  • the change detection unit 125 corresponds to the determination unit 13 in the first embodiment.
  • the change detection unit 125 calculates the similarity by comparing the spatial feature amounts between the coordinate F1 in the reference point group and the coordinate F2 in the input point group. Then, using the similarity, it determines whether or not the presence or absence of a point at the coordinate F1 has changed at the coordinate F2 when changing from the reference point group to the input point group.
  • the change detection unit 125 can execute: (I) Determine whether a point that did not exist in the coordinate F1 of the reference point group now exists in the coordinate F2 of the input point group. (II) Determine whether a point that existed in the coordinate F1 of the reference point group no longer exists in the coordinate F2 of the input point group. Details of processes (I) and (II) are explained below.
  • the change detection unit 125 calculates the similarity between the feature of coordinate F1 in the reference point group calculated by the reference feature calculation unit 123 and the feature of coordinate F2 in the input point group calculated by the input feature calculation unit 124.
  • the feature of coordinate F1 is P1
  • the feature of coordinate F2 is P2
  • the similarity between the two is S(P1, P2).
  • the change detection unit 125 calculates S(P1, P2) as follows:
  • ⁇ , ⁇ , and r in (5) to (7) are divided into units of 2 ⁇ , ⁇ , and d, respectively.
  • P1 ⁇ r on the right side of (5) indicates an element of feature amount P1, which is an element of the spatial feature amount defined in small region SR1 in spherical region S1.
  • P2 ⁇ r indicates an element of feature amount P2, which is an element of the spatial feature amount defined in small region SR2 in spherical region S2.
  • P1 ⁇ r and P2 ⁇ r are the elements of the feature amounts in small regions SR1 and SR2 that are in the same position when the reference point group and the input point group are compared.
  • ValidNum in (5) is the number of small regions SR2 in the spherical region S2 other than the small regions SR2 defined as "where the presence of a point is unknown.” Therefore, (5) indicates that Score (P1 ⁇ r , P2 ⁇ r ) is summed over all small regions SR2 in the spherical region S2 (or all small regions SR1 in the spherical region S1), and the sum is normalized by ValidNum. At this time, the small regions SR2 defined as "where the presence of a point is unknown" are not evaluated in the similarity calculation (i.e., they are ignored in the calculation).
  • Score(P1 ⁇ r , P2 ⁇ r ) is 1 when Same around (P1 ⁇ r , P2 ⁇ r ) exists for P1 ⁇ r and P2 ⁇ r , and is 0 otherwise.
  • FIG. 9D is a diagram for explaining the above formula (7) in the examples of FIG. 9B and FIG. 9C.
  • a part of the spherical region S1 near the coordinate H1 in FIG. 9B and a part of the spherical region S2 near the coordinate H2 in FIG. 9C are shown.
  • the element in the spatial feature of the small region SR1 including the coordinate H1 is P1 ⁇ r
  • the element in the spatial feature of the small region SR2 including the coordinate H2 is P2 ⁇ r .
  • FIG. 9D is a diagram for explaining the above formula (7) in the examples of FIG. 9B and FIG. 9C.
  • a part of the spherical region S1 near the coordinate H1 in FIG. 9B and a part of the spherical region S2 near the coordinate H2 in FIG. 9C are shown.
  • the element in the spatial feature of the small region SR1 including the coordinate H1 is P1 ⁇ r
  • the elements in the spatial feature of the small region SR1 adjacent in the ⁇ direction to the small region SR1 including the coordinate H1 in the ⁇ direction are P1 ( ⁇ +1) ⁇ r and P1 ( ⁇ -1) ⁇ r .
  • the small region SR1 adjacent in the r direction is defined in each of the small region SR1 including the coordinate H1, the small region SR1 at P1 ( ⁇ +1) ⁇ r , and the small region SR1 at P1 ( ⁇ -1) ⁇ r.
  • the elements in the spatial feature amount of these small regions SR1 are P1 ⁇ (r ⁇ 1) , P1 ( ⁇ +1) ⁇ (r ⁇ 1) , and P1 ( ⁇ 1) ⁇ (r ⁇ 1), respectively.
  • P1 ⁇ r indicates that a point does not exist
  • P2 ⁇ r indicates that a point exists, so they are not the same element.
  • the range of ⁇ indicated by formula (7) includes P1 ( ⁇ +1) ⁇ r and P1 ( ⁇ -1) ⁇ r .
  • P1 ( ⁇ -1) ⁇ r indicates that a point exists, so P1 ( ⁇ -1) ⁇ r and P2 ⁇ r are the same element. Therefore, in the example of FIG. 9D, Same around (P1 ⁇ r , P2 ⁇ r ) shown in (7) exists, and Score (P1 ⁇ r , P2 ⁇ r ) shown in (6) is 1.
  • the change detection unit 125 executes a matching process to calculate the similarity S(P1, P2) shown in equation (5) by calculating Score (P1 ⁇ r , P2 ⁇ r ) for all ⁇ , ⁇ , and r.
  • the change detection unit 125 also calculates the similarity S(Pn, P2) for each coordinate other than the coordinate F1 included in the neighborhood region ⁇ 1 set by the reference feature calculation unit 123, using the same calculation method as for S(P1, P2).
  • Pn is the feature at each coordinate other than the coordinate F1, and is calculated by the reference feature calculation unit 123 in the above process.
  • S(PN, P2) is defined as the similarity S within the neighborhood region ⁇ 1 including S(P1, P2) and S(Pn, P2).
  • the change detection unit 125 After calculating all similarities S(PN,P2) in the neighborhood region ⁇ 1 in this way, the change detection unit 125 identifies Smax (PN,P2) which is the maximum value among S(PN,P2).
  • the coordinates of the reference point group corresponding to this Smax (PN,P2) are the coordinates most similar to the coordinate F2 in the input point group in terms of the state indicating whether or not a point exists among the coordinates included in the neighborhood region ⁇ 1 in the reference point group.
  • the change detection unit 125 compares Smax (PN,P2) with a predetermined threshold value ThS1 to determine which is larger.
  • Smax (PN,P2) is equal to or smaller than ThS1
  • the change detection unit 125 determines that the coordinates included in the neighborhood region ⁇ 1 have a low similarity to the coordinate F2.
  • the change detection unit 125 determines that a point that did not exist in the coordinate F1 of the reference point group now exists in the coordinate F2 of the input point group.
  • Smax (PN,P2) is greater than ThS1
  • the change detection unit 125 does not make the above determination.
  • the change detection unit 125 executes a matching process to calculate S(P1, P2), which is the similarity between the feature amount of the coordinate F1 and the feature amount of the coordinate F2. This calculation method is the same as in (I), so the explanation will be omitted.
  • the change detection unit 125 also calculates the similarity S(P1, Pm) for each coordinate other than the coordinate F2 included in the neighborhood region ⁇ 2 set by the input feature calculation unit 124, using the same calculation method as for S(P1, P2). Note that Pm is the feature at each coordinate other than the coordinate F2, and is calculated by the input feature calculation unit 124 in the above process.
  • S(P1, PM) is defined as the similarity S within the neighborhood region ⁇ 2 including S(P1, P2) and S(P1, Pm).
  • the change detection unit 125 After calculating all similarities SS(P1,PM) in the neighborhood region ⁇ 2 in this manner, the change detection unit 125 identifies Smax (P1,PM) which is the maximum value among S(P1,PM).
  • the coordinates of the reference point group corresponding to this Smax (P1,PM) are the coordinates which, among the coordinates included in the neighborhood region ⁇ 2 in the input point group, are most similar in terms of the state indicating whether or not a point exists to the coordinate F1 in the reference point group.
  • the change detection unit 125 compares Smax (P1,PM) with a predetermined threshold value ThS2.
  • the change detection unit 125 determines that the coordinates included in the neighborhood region ⁇ 2 have a low similarity to the coordinate F1. The change detection unit 125 then determines that a point that existed at the coordinate F1 of the reference point group no longer exists at the coordinate F2 of the input point group. On the other hand, if Smax (P1,PM) is greater than ThS2, the change detection unit 125 does not make the above determination.
  • the change detection unit 125 determines that the presence or absence of a point at the coordinate F1 has not changed at the coordinate F2, i.e., that there is no change between the coordinate F1 and the coordinate F2.
  • the reference feature calculation unit 123 and the input feature calculation unit 124 can change the size of the neighborhood regions ⁇ 1 and ⁇ 2 depending on the situation.
  • the reference feature calculation unit 123 may increase the size of the neighborhood region ⁇ 1 (e.g., the size of the radius ⁇ 1) as the distance between the position indicated by the coordinate F1 when the reference point group is measured and the position of the dedicated sensor S at the time of measurement increases. This is because the influence of the above-mentioned deviation occurs over a wider range as the coordinate F1 of the reference point group is farther away from the dedicated sensor S at the time of measurement, so it is preferable to set the coordinates of the reference point group to be compared over a wider range.
  • the input feature calculation unit 124 may increase the size of the neighborhood region ⁇ 2 (e.g., the size of the radius ⁇ 2) as the distance between the position indicated by the coordinate F2 when the input point group is measured and the position of the LiDAR 102 at the time of measurement increases.
  • the change detection unit 125 executes the above processes (I) and (II) for each coordinate in the reference point cloud and each coordinate in the input point cloud that corresponds to each of those coordinates. This enables the change detection unit 125 to detect changes in the presence or absence of points in all coordinates of the reference point cloud and the input point cloud.
  • the coordinates of the input point cloud are selected as the starting point for detecting changes, and the presence or absence of a change is determined for each coordinate of the input point cloud, thereby detecting whether a point that is not in the reference point cloud has been newly added to the input point cloud.
  • the coordinates of the reference point cloud are selected as the starting point for detecting changes, and the presence or absence of a change is determined for each coordinate of the reference point cloud, thereby detecting whether a point in the reference point cloud is no longer in the input point cloud.
  • the detection result generating unit 126 generates an image showing the change detected by the change detection unit 125 as a result of the change detection unit 125 executing the process shown in (I). For example, when the reference point group and the input point group are acquired in the situation shown in FIG. 8A and 8B, the detection result generating unit 126 can generate the image shown in FIG. 8C as a result of the change detection unit 125 executing the process shown in (I). The detection result generating unit 126 can also generate an image showing the change detected by the change detection unit 125 as a result of the change detection unit 125 executing the process shown in (II).
  • the center server 120 may have an interface such as a display that shows the image generated by the detection result generating unit 126 to the user.
  • the detection result generating unit 126 may generate a point group of 3D data showing the change detected by the change detection unit 125 instead of an image.
  • the detection result generating unit 126 may perform processing to visually emphasize locations where a change in the presence or absence of a point has occurred (or locations where a change in the presence or absence of an object has occurred) on the generated screen or point cloud, similar to the output unit described in embodiment 1.
  • FIG. 10A to 10C are flowcharts showing an example of a representative process of the center server 120, and an overview of the process of the center server 120 is explained using these flowcharts. The details of each process are as described above, and therefore will not be explained as appropriate.
  • Fig. 10A is a flowchart showing an overview of an example of the process of the center server 120, and the process flow will be explained first using Fig. 10A.
  • the reference point cloud acquisition unit 121 acquires reference point cloud data from the reference point cloud DB 130 (step S21; acquisition step).
  • the input point cloud acquisition unit 122 acquires input point cloud data by acquiring data transmitted by the robot 101 (step S22; acquisition step). Note that either step S21 or S22 may be performed first, or both steps may be performed in parallel.
  • Process (A) indicates the above process (I) and processes related to it, and the details thereof will be described using FIG. 10B.
  • the reference feature calculation unit 123, the input feature calculation unit 124, and the change detection unit 125 also execute process (B) (step S24; process (B) step).
  • Process (B) indicates the above process (II) and processes related thereto, and the details thereof will be explained using FIG. 10C. Note that either of the processes in steps S23 and S24 may be performed first, or both processes may be performed in parallel.
  • the detection result generating unit 126 generates an image showing the change detected in the process (A) based on the information indicating the presence or absence of a change at each coordinate generated by the process (A). Similarly, the detection result generating unit 126 generates an image showing the change detected in the process (B) based on the information indicating the presence or absence of a change at each coordinate generated by the process (B) (step S25; detection result image generating step).
  • the input feature calculation unit 124 calculates spatial features for coordinates in the input point cloud for which features have not yet been calculated (step S31; feature calculation step).
  • the input feature calculation unit 124 outputs information on the calculated coordinates to the reference feature calculation unit 123.
  • the reference feature calculation unit 123 Based on the output coordinate information, the reference feature calculation unit 123 identifies coordinates in the reference point group that correspond to the coordinates.
  • the coordinate information output by the input feature calculation unit 124 is the coordinate F2 information in the above example
  • the coordinate information identified by the reference feature calculation unit 123 is the coordinate F1 information in the above example.
  • the reference feature calculation unit 123 sets a neighborhood region ⁇ 1 that includes the coordinate F1, and calculates the spatial features of each coordinate included in the neighborhood region ⁇ 1 (step S32; feature calculation step). Note that in the above, the process of step S31 is performed before the process of step S32, but the process of step S32 may be performed before the process of step S31, or both processes may be performed in parallel.
  • the change detection unit 125 calculates the similarity between coordinates F1 and F2 using the spatial features calculated in steps S31 and S32 (step S33; similarity calculation step).
  • the change detection unit 125 compares the maximum value of the calculated similarity with a predetermined threshold value to determine whether or not a change has occurred in the presence or absence of a point at coordinate F2 (step S34; change detection step).
  • the change detection unit 125 determines whether or not the similarity calculation and change detection determination have been completed for all coordinates in the input point cloud (step S35; completion determination step). If the similarity calculation and change detection determination have not been completed for all coordinates in the input point cloud (No in step S35), the process returns to step S31 and is repeated for coordinates in the input point cloud for which similarity has not been calculated. If the similarity calculation and change detection determination have been completed for all coordinates in the input point cloud (Yes in step S35), process (A) ends. Then, as described above, the detection result generation unit 126 generates a change detection image based on the information generated by process (A) indicating the presence or absence of a change at each coordinate.
  • the reference feature calculation unit 123 calculates the spatial feature for coordinates in the reference point group for which the feature has not yet been calculated (step S41; feature calculation step).
  • the reference feature calculation unit 123 outputs information on the calculated coordinates to the input feature calculation unit 124.
  • the input feature calculation unit 124 Based on the output coordinate information, the input feature calculation unit 124 identifies coordinates in the input point cloud that correspond to the coordinates.
  • the coordinate information output by the reference feature calculation unit 123 is the coordinate F1 information in the above example
  • the coordinate information identified by the input feature calculation unit 124 is the coordinate F2 information in the above example.
  • the input feature calculation unit 124 sets a neighborhood region ⁇ 2 that includes the coordinate F2, and calculates the spatial features of each coordinate included in the neighborhood region ⁇ 2 (step S42; feature calculation step). Note that in the above, the process of step S41 is performed before the process of step S42, but the process of step S42 may be performed before the process of step S41, or both processes may be performed in parallel.
  • the change detection unit 125 calculates the similarity between coordinates F1 and F2 using the spatial features calculated in steps S41 and S42 (step S43; similarity calculation step). The change detection unit 125 compares the maximum value of the calculated similarity with a predetermined threshold value to determine whether or not a change has occurred in the presence or absence of a point at coordinates F1 (step S44; change detection step).
  • the change detection unit 125 determines whether or not the similarity calculation and change detection determination have been completed for all coordinates in the reference point group (step S45; completion determination step). If the similarity calculation and change detection determination have not been completed for all coordinates in the reference point group (No in step S45), the process returns to step S41 and is repeated for coordinates in the reference point group for which similarity has not been calculated. If the similarity calculation and change detection determination have been completed for all coordinates in the reference point group (Yes in step S45), process (B) ends. Then, as described above, the detection result generation unit 126 generates an image in which changes have been detected, based on the information generated by process (B) indicating the presence or absence of changes at each coordinate.
  • the similarity for all coordinates in the input point cloud may be calculated first, and then a change detection determination may be performed for each coordinate in the input point cloud.
  • the similarity for all coordinates in the reference point cloud may be calculated first, and then a change detection determination may be performed for each coordinate in the reference point cloud.
  • both processes (A) and (B) are executed, but only one of processes (A) and (B) may be executed, and the change detection unit 125 may generate an image for only the executed process.
  • the change detection unit 125 may generate an image for only the executed process.
  • the center server 120 can generate a detection image for each input point cloud by executing the comparison process described above between each input point cloud and the reference point cloud.
  • the input feature calculation unit 124 calculates the spatial feature of the input point cloud by vectorizing information indicating the presence or absence of a point, or the presence or absence of a point is unknown.
  • the change detection unit 125 can determine whether a change in the presence or absence of a point has occurred between the reference point cloud and the input point cloud by using the spatial feature of the calculated reference point cloud and the spatial feature of the input point cloud.
  • the density of the points of the input point cloud is sparser than the density of the points of the reference point cloud, and there are cases where an object exists and a point is not recorded at a coordinate where the point should be. Even in such a case, it is estimated that the accuracy of the determination is higher by performing the above process in the center server 120 compared to when the data of the input point cloud is used as is. This is particularly effective when performing a point cloud comparison process in real time.
  • the reference feature calculation unit 123 may also calculate the feature of each coordinate in the neighborhood region ⁇ 1 of the calculation target coordinate.
  • the change detection unit 125 can determine whether or not a point that does not exist in the calculation target coordinate exists in the corresponding coordinate in the input point group, using the spatial feature of the calculation target coordinate, the spatial feature of each coordinate in the neighborhood region ⁇ 1 of the calculation target coordinate, and the spatial feature of the calculation target coordinate and the corresponding coordinate in the input point group.
  • the input feature calculation unit 124 may also calculate the feature of each coordinate in the neighborhood ⁇ 2 of the calculation target coordinate.
  • the change detection unit 125 can determine whether a point that exists in the calculation target coordinate does not exist in the corresponding coordinate in the reference point cloud, using the spatial feature of the calculation target coordinate, the spatial feature of each coordinate in the neighborhood ⁇ 2 of the calculation target coordinate, and the spatial feature of the calculation target coordinate and the corresponding coordinate in the reference point cloud. This makes it possible to suppress deterioration of change detection accuracy even if a translational deviation occurs between the measurement of the reference point cloud and the measurement of the input point cloud.
  • Figures 11A and 11B show images in which changes between the reference point cloud and input point cloud shown in Figures 7A and 7B have been detected by directly comparing the presence or absence of points, and images in which changes between the two have been detected using the method of this disclosure.
  • Figures 7A and 7B show measurements of a rack L in a warehouse using a sensor. At this time, it is assumed that the measurement positions of the reference point cloud and the input point cloud are shifted by a predetermined position in the translation direction. Generally, the measurement conditions of the reference point cloud and the input point cloud are not exactly the same, but are often different, and this situation reflects this. Furthermore, when the input point cloud is measured, an object OB is present that was not present when the reference point cloud was measured.
  • the points represented by dots are the points detected as changes between the reference point cloud and the input point cloud.
  • image C1 of FIG. 11A not only is object OB, which should be detected as a change, detected as a change is rack L, whose appearance from the sensor changes between the reference point cloud and the input point cloud.
  • image C2 of FIG. 11B detection of rack L as a change is suppressed, and object OB is detected as a clear change. In this way, the method disclosed herein can suppress deterioration of change detection accuracy even when a translational deviation occurs between the measurement of the reference point cloud and the measurement of the input point cloud.
  • the reference point cloud is data acquired using a sensor, and the reference point cloud acquisition unit 121 may increase the size of the neighborhood region ⁇ 1 as the distance between the position indicated by the coordinates subject to change detection at the time of measurement and the position of the sensor increases. This makes it possible to measure an area farther away from the sensor, and set the neighborhood region to cover the effects of the deviation even if the effect of the deviation is greater, thereby preventing deterioration in change detection accuracy.
  • the input point cloud is data acquired using a sensor
  • the input point cloud acquisition unit 122 may increase the size of the neighborhood region ⁇ 2 as the distance between the position indicated by the coordinates subject to change detection at the time of measurement and the position of the sensor increases. This makes it possible to measure an area farther away from the sensor, and even if the effect of the deviation is greater, set the neighborhood region so as to cover the effect of the deviation, thereby suppressing deterioration in change detection accuracy.
  • the change detection unit 125 may calculate the similarity between the spatial feature of the calculation target coordinates of the reference point group and the spatial feature of the corresponding coordinates of the input point group that correspond to the calculation target coordinates, and the similarity between the spatial feature of each coordinate in the neighboring region ⁇ 1 and the spatial feature of the corresponding coordinates of the input point group.
  • the similarity between the spatial features of the calculation target coordinates and the spatial features of the corresponding coordinates is calculated using the elements of the spatial features of the calculation target coordinates in a specific small region SR1 in the spherical region S1 and a small region SR1 included in the peripheral region of that small region SR1, and the elements of a small region SR2 in the spherical region S2 that corresponds to the specific small region SR1.
  • the similarity between the spatial features of each coordinate in the neighboring region ⁇ 1 and the spatial features of the corresponding coordinate is calculated using the elements of the spatial features of a specific small region SR1 in the spherical region S1 of each coordinate and a small region SR1 included in the peripheral region of that small region SR1, and the elements of the small region SR2 in the spherical region S2 that corresponds to the specific small region SR1.
  • the change detection unit 125 may calculate the similarity between the spatial feature of the calculation target coordinates of the input point group and the spatial feature of the corresponding coordinates of the reference point group that correspond to the calculation target coordinates, and the similarity between the spatial feature of each coordinate in the neighboring region ⁇ 2 and the spatial feature of the corresponding coordinates of the reference point group.
  • the similarity between the spatial features of the calculation target coordinates and the spatial features of the corresponding coordinates is calculated using the elements of the spatial features of the calculation target coordinates in each of a specific small region SR2 in the spherical region S2 and a small region SR2 included in the peripheral region of that small region SR2, and the elements of the small region SR1 in the spherical region S1 that corresponds to the specific small region SR2.
  • the similarity between the spatial features of each coordinate in the neighboring region ⁇ 2 and the spatial features of the corresponding coordinate is calculated using the elements of the spatial features of a specific small region SR2 in the spherical region S2 of each coordinate and a small region SR2 included in the peripheral region of that small region SR2, and the elements of the small region SR1 in the spherical region S1 that corresponds to the specific small region SR2.
  • the change detection unit 125 may appropriately change the size of at least one of the surrounding areas of small region SR1 or the surrounding areas of small region SR2. This makes it possible to change the tolerance for rotational misalignment.
  • the reference feature calculation unit 123 may also calculate spatial features in the input point cloud by vectorizing information indicating whether a point exists, whether a point does not exist, or whether the existence of a point is unknown, in a manner similar to that of the input feature calculation unit 124.
  • the ray tracing technology described in the explanation of the input feature calculation unit 124 can be applied to the definition of information indicating whether the existence of a point is unknown.
  • the change detection unit 125 calculates the similarity using the spatial feature calculated in this way and the spatial feature calculated by the input feature calculation unit 124 in the above-mentioned method.
  • ValidNum in formula (5) is the number of small areas SR1 in the spherical area S1 other than the small area SR1 defined as "where the presence of a point is unknown". Therefore, the small areas SR1 defined as "where the presence of a point is unknown" are not evaluated in the similarity calculation. This makes it possible to reflect a state in which the presence of a point in the reference point group is unknown in the change detection, thereby making it possible to further improve the accuracy of change detection between the reference point group and the input point group.
  • the reference point cloud and the input point cloud may be mapping data generated based on two-dimensional images of a specific location taken by a camera or a positioning sensor.
  • the following cases are also considered as cases in which a misalignment occurs between the images related to the reference point cloud and the images related to the input point cloud.
  • a misalignment may occur when images of the same location are taken at different times using a movable camera carried by a person and the reference point cloud and the input point cloud are generated based on the images.
  • the change detection unit 125 may perform process (I) for a number of coordinates of less than all of the input point cloud and a number of coordinates of the reference point cloud corresponding to those coordinates. Similarly, the change detection unit 125 may perform process (II) for a number of coordinates of less than all of the reference point cloud and a number of coordinates of the input point cloud corresponding to those coordinates.
  • the detection result generation unit 126 generates an image showing the changes detected by the change detection unit 125 based on this determination result. In this way, if there is an area in the point cloud that does not require determination, the center server 120 can detect changes in the coordinates excluding that area.
  • FIG. 12 is a block diagram showing another example of the center server according to the second embodiment.
  • the center server 120 further includes an extraction unit 127 in addition to the components shown in Fig. 6. Below, the processing of the center server 120 will be described with reference to (2A) omitted, and only the points specific to this example will be described.
  • the extraction unit 127 directly compares the presence or absence of points at each corresponding coordinate in the reference point group acquired by the reference point group acquisition unit 121 and the input point group acquired by the input point group acquisition unit 122. This extracts information on coordinates where the presence or absence of points at corresponding coordinates differs between the reference point group and the input point group (information on changed coordinates), and does not extract other coordinates.
  • the reference feature calculation unit 123 and the input feature calculation unit 124 execute the process shown in (2A) for the coordinates of the reference point group and the coordinates of the input point group extracted in this manner, and calculate the spatial feature.
  • the change detection unit 125 and the detection result generation unit 126 execute the process shown in (2A) using the calculated spatial feature.
  • FIGS. 13A to 13C are flowcharts showing an example of a representative process of the center server 120 in (2B), which corresponds to Figures 10A to 10C, and explain an overview of the process of the center server 120. Note that descriptions of the same points as in (2A) will be omitted as appropriate.
  • the reference point cloud acquisition unit 121 acquires reference point cloud data from the reference point cloud DB 130 (step S21; acquisition step). Also, the input point cloud acquisition unit 122 acquires input point cloud data by acquiring data transmitted by the robot 101 (step S22; acquisition step).
  • the extraction unit 127 directly compares the presence or absence of points at corresponding coordinates between the reference point group acquired by the reference point group acquisition unit 121 and the input point group acquired by the input point group acquisition unit 122. This detects information on changed coordinates (step S26; change detection step).
  • the reference feature amount calculation unit 123, the input feature amount calculation unit 124, and the change detection unit 125 execute process (A') (step S23'; process (A') step). Details of process (A') will be explained using FIG. 13B.
  • process (B') indicates the above process (II) and processes related thereto, and the details thereof will be explained using FIG. 13C. Note that either of the processes in steps S23' and S24' may be performed first, or both may be performed in parallel.
  • the detection result generating unit 126 generates an image showing the change detected with respect to process (A') based on the information indicating the presence or absence of a change at each coordinate generated by process (A'). Similarly, the detection result generating unit 126 generates an image showing the change detected with respect to process (B') based on the information indicating the presence or absence of a change at each coordinate generated by process (B') (step S25; detection result image generating step).
  • the input feature calculation unit 124 calculates spatial features for coordinates in the input point cloud extracted by the extraction unit 127 for which features have not yet been calculated (step S31; feature calculation step).
  • the input feature calculation unit 124 outputs information on the calculated coordinates to the reference feature calculation unit 123.
  • the reference feature calculation unit 123 Based on the output coordinate information, the reference feature calculation unit 123 identifies the coordinate F1 in the reference point group that corresponds to that coordinate. The reference feature calculation unit 123 sets a neighborhood region ⁇ 1 that includes the coordinate F1, and calculates the spatial feature of each coordinate included in the neighborhood region ⁇ 1 (step S32; feature calculation step). As described above, the process of step S31 or the process of step S32 may be performed in any order, or both processes may be performed in parallel.
  • the change detection unit 125 calculates the similarity between coordinates F1 and F2 using the spatial features calculated in steps S31 and S32 (step S33; similarity calculation step).
  • the change detection unit 125 compares the maximum value of the calculated similarity with a predetermined threshold value to determine whether or not a change has occurred in the presence or absence of a point at coordinate F2 (step S34; change detection step).
  • the change detection unit 125 determines whether or not the similarity calculation and change detection determination have been completed for all coordinates extracted in the input point cloud (step S36; completion determination step). If the similarity calculation and change detection determination have not been completed for all extracted coordinates (No in step S36), the process returns to step S31 and repeats the process for the coordinates for which the similarity has not been calculated. If the similarity calculation and change detection determination have been completed for all extracted coordinates (Yes in step S36), process (A') ends. Then, as described above, the detection result generation unit 126 generates a change detection image based on the information generated by process (A') that indicates the presence or absence of a change at each coordinate.
  • the reference feature calculation unit 123 calculates spatial features for coordinates in the reference point group extracted by the extraction unit 127 for which features have not yet been calculated (step S41; feature calculation step).
  • the reference feature calculation unit 123 outputs information on the calculated coordinates to the input feature calculation unit 124.
  • the input feature calculation unit 124 Based on the output coordinate information, the input feature calculation unit 124 identifies coordinate F2 in the input point cloud that corresponds to that coordinate.
  • the input feature calculation unit 124 sets a neighborhood region ⁇ 2 that includes coordinate F2, and calculates the spatial feature of each coordinate included in neighborhood region ⁇ 2 (step S42; feature calculation step).
  • step S42 feature calculation step
  • the process of step S41 or the process of step S42 may be performed in any order, or both processes may be performed in parallel.
  • the change detection unit 125 calculates the similarity between coordinates F1 and F2 using the spatial features calculated in steps S41 and S42 (step S43; similarity calculation step). The change detection unit 125 compares the maximum value of the calculated similarity with a predetermined threshold value to determine whether or not a change has occurred in the presence or absence of a point at coordinates F1 (step S44; change detection step).
  • the change detection unit 125 determines whether or not the similarity calculation and change detection determination have been completed for all coordinates extracted in the reference point group (step S46; completion determination step). If the similarity calculation and change detection determination have not been completed for all extracted coordinates (No in step S46), the process returns to step S41 and repeats the process for the coordinates for which the similarity has not been calculated. If the similarity calculation and change detection determination have been completed for all extracted coordinates (Yes in step S46), process (B') ends. Then, as described above, the detection result generation unit 126 generates an image in which changes have been detected, based on the information indicating the presence or absence of changes at each coordinate, generated by process (B').
  • the center server 120 can extract information on coordinates that have changed using the extraction unit 127, and execute the change detection process shown in (2A) for the extracted coordinates. This can reduce the calculation costs required for all steps of the process, compared to executing the change detection process shown in (2A) for the coordinates of all input point groups or reference point groups.
  • FIG. 14A is a block diagram showing another example of the center server according to the embodiment 2.
  • the center server 120 further includes an object identification unit 128 in addition to the components shown in Fig. 12.
  • object identification unit 128 in addition to the components shown in Fig. 12.
  • the object identification unit 128 determines whether the change in the presence or absence of a point shown in the image generated by the detection result generation unit 126 as a result of executing the process shown in (I) or the image generated as a result of executing the process shown in (II) corresponds to any object (e.g., a container, a vehicle, construction materials, etc.).
  • the object identification unit 128 determines whether the changed point cloud portion is equal to or larger than a predetermined size, and if it is equal to or larger than the predetermined size, it determines that the changed point cloud portion corresponds to some kind of object. If the changed point cloud portion is smaller than the predetermined size, the object identification unit 128 determines that the changed point cloud portion is noise.
  • the object identification unit 128 may compare the location of the changed point cloud with pre-stored point cloud data of various objects for determination, such as containers, vehicles, construction materials, etc. If the location of the changed point cloud matches the point cloud data of any object, or matches except for an error of, for example, a few percent, the object identification unit 128 identifies the matching object as an object whose presence or absence has changed between the input point cloud and the reference point cloud.
  • the object identification unit 128 may perform object determination using a pre-trained AI model. This training is performed by inputting training data, including image information showing the changed point cloud as a sample and information (correct label) showing various objects corresponding to that information, into the AI model. After the AI model has been trained using the training data, the object identification unit 128 inputs the image generated by the detection result generation unit 126 into the AI model. Based on this input image, the AI model outputs information showing the object shown in the input image. In this way, the object identification unit 128 can also perform object identification processing. Note that any technology, such as logistic regression or neural network, can be used to train the learning model.
  • the object identifying unit 128 determines whether the change in the presence or absence of points shown in the point cloud, which is the detection result, corresponds to some object.
  • the detection result generating unit 126 can perform object determination using an AI model that has been trained in advance. This learning is performed by inputting training data including a changed point cloud that serves as a sample and information (correct answer label) indicating various objects corresponding to that information into the AI model. After the AI model has been trained using the training data, the object identifying unit 128 inputs the point cloud generated by the detection result generating unit 126 into the AI model. Based on this point cloud, the AI model can output information indicating the object indicated by the point cloud.
  • Other determination methods that the detection result generating unit 126 can execute are as described above.
  • the center server 120 can detect whether there has been a change in the presence or absence of an object between the reference point group and the input point group. More preferably, the center server 120 can also identify an object whose presence or absence has changed between the reference point group and the input point group.
  • the object identification unit 128 may generate and output a screen on which a process for visually highlighting the identified object has been performed. The process for visually highlighting the object is as described in the first embodiment.
  • the object identification unit 128 may output an alert by voice via a speaker.
  • the object identification unit 128 may output the determination result of the determination unit 13 to another device.
  • FIG. 14B is a block diagram showing another example of the center server according to the embodiment 2.
  • the center server 120 further includes a mobility control unit 129.
  • the points explained in (2A) to (2C) are omitted, and only points unique to this example are explained.
  • the movement control unit 129 controls the movement of the robot 101. For example, when the object identification unit 128 determines that the location of the changed point cloud corresponds to some kind of object, the movement control unit 129 can instruct the robot 101 to approach the location of the changed point cloud and then measure the location further. This instruction is given in order to obtain more detailed point cloud data of the location and analyze it. Alternatively, the movement control unit 129 may control the robot 101 as described above when the object identification unit 128 identifies a specific type of object.
  • This instruction is output in at least one of the following cases: as a result of the process shown in (I), it is determined that an object does not exist in the reference point cloud and an object has become present in the input point cloud, or as a result of the process shown in (II), it is determined that an object exists in the reference point cloud and that the object no longer exists in the input point cloud.
  • the instruction is output at least when an object does not exist in the reference point cloud and an object has become present in the input point cloud.
  • FIG. 15A and 15B are flowcharts showing an example of a representative process of the center server 120 in (2D), which corresponds to Fig. 13A, and explain an overview of the process of the center server 120. Note that descriptions of the same points as in (2A) and (2B) will be omitted as appropriate.
  • Steps S21 to S25 are the same as those in FIG. 13A, and therefore will not be described here.
  • the object identification unit 128 executes the process shown in (2C) and determines whether or not the changed point cloud portion corresponds to any object in the image generated by the detection result generation unit 126 through the process of (I) (step S27; object detection determination step). If the changed point cloud portion does not correspond to an object (No in step S27), the movement control unit 129 does not execute any special control. In this case, the robot 101 moves along, for example, a previously set movement route.
  • step S27 the movement control unit 129 controls the robot 101 to approach the changed point cloud portion and then further measure the portion (step S28; robot movement step). In this case, the robot 101 moves away from the previously set movement route.
  • steps S27 and S28 can also be performed on the images generated by the detection result generating unit 126 through the process of (II).
  • the center server 120 can also perform the processes shown in (2A) to (2D) on the images acquired by the robot approaching and measuring as a result of the process of step S28.
  • the center server 120 controls the robot 101 to acquire detailed point cloud data. This enables more detailed inspection of failures or abnormalities in infrastructure facilities. Note that even when the detection result generating unit 126 generates a point cloud, the movement control unit 129 can execute similar control based on the object determination result of the object identifying unit 128.
  • the movement control unit 129 may set the movement route to a route that avoids the location where the object is present.
  • the movement control unit 129 controls the movement of the robot 101 so that the robot 101 moves along the newly set route.
  • steps S31 to S34 are repeatedly executed for each point in the input point cloud.
  • a process of calculating spatial features for all coordinates to be processed in the input point cloud and a process of calculating spatial features for an area including the coordinates of the reference point cloud that correspond to each coordinate of the input point cloud may be executed first.
  • the processes of steps S33 to S34 are executed for all coordinates, making it possible to detect a change in the presence or absence of points in the area to be processed.
  • the order of the processes may be similarly changed.
  • polar coordinates are used to define the spherical region S1 surrounding the coordinate F1 and the spherical region S2 surrounding the coordinate F2.
  • the use of polar coordinates has the effect of making the calculations of the reference feature calculation unit 123 and the input feature calculation unit 124 easier.
  • the change detection unit 125 calculates S(P1, P2) by simply adding Score(P1 ⁇ r , P2 ⁇ r ) shown in (6).
  • S(P1, P2) may be calculated by other methods so that S(P1, P2) increases monotonically with respect to the number of Same around (P1 ⁇ r , P2 ⁇ r ) present in the entire range of ⁇ , ⁇ , and r.
  • the change detection unit 125 calculates S(P1, P2) as being proportional to the inverse of ValidNum.
  • S(P1, P2) may be calculated by other methods so that S(P1, P2) decreases monotonically with respect to ValidNum.
  • any part of the above-mentioned processing executed by each unit of the center server 120 may be executed by at least one of the robot 101 and a different server.
  • the processing of the center server 120 may be realized by a distributed system.
  • the center server 120 may not be provided, and the robot 101 may execute the above-mentioned processing of the center server 120 in a stand-alone manner.
  • the reference point cloud and the position information at which the reference point cloud was measured are stored in association with each other in the storage unit of the robot 101.
  • the robot 101 performs measurement with its own LiDAR 102 and acquires the input point cloud.
  • the robot 101 searches the storage unit using the position information at which the input point cloud was measured, and acquires the data of the reference point cloud to be compared with the input point cloud.
  • the details are the same as those of the processing of the reference point cloud acquisition unit 121 described above.
  • the robot 101 can execute the processing related to the reference feature amount calculation unit 123 to the detection result generation unit 126 described above.
  • the robot 101 may also execute processing related to the object identification unit 128.
  • the robot 101 detects an object with the object identification unit 128, it can control its own movement unit to approach the location of the changed point cloud and then measure the location.
  • this disclosure has been described as a hardware configuration, but this disclosure is not limited to this.
  • This disclosure can also be realized by having a processor in a computer execute a computer program to execute the processes (steps) of the change detection device, each device in the change detection system, or the center server described in the above embodiment.
  • FIG. 16 is a block diagram showing an example of the hardware configuration of an information processing device in which the processes of the above-described embodiments are executed.
  • this information processing device 90 includes a signal processing circuit 91, a processor 92, and a memory 93.
  • the signal processing circuit 91 is a circuit for processing signals according to the control of the processor 92.
  • the signal processing circuit 91 may also include a communication circuit for receiving signals from a transmitting device.
  • the processor 92 is connected (coupled) to the memory 93, and performs the processing of the device described in the above embodiment by reading and executing software (computer programs) from the memory 93.
  • Examples of the processor 92 include a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an FPGA (Field-Programmable Gate Array), a DSP (Demand-Side Platform), and an ASIC (Application Specific Integrated Circuit).
  • a single processor may be used as the processor 92, or multiple processors may be used in cooperation with each other.
  • Memory 93 may be composed of volatile memory, non-volatile memory, or a combination of both.
  • the volatile memory may be, for example, a RAM (Random Access Memory) such as DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory).
  • the non-volatile memory may be, for example, a ROM (Read Only Memory) such as PROM (Programmable Random Only Memory) or EPROM (Erasable Programmable Read Only Memory), flash memory, or an SSD (Solid State Drive).
  • a single memory may be used as memory 93, or multiple memories may be used in cooperation with each other.
  • the memory 93 is used to store one or more instructions.
  • the one or more instructions are stored in the memory 93 as a group of software modules.
  • the processor 92 can perform the processing described in the above embodiment by reading and executing these groups of software modules from the memory 93.
  • the memory 93 may include memory built into the processor 92 in addition to memory provided outside the processor 92.
  • the memory 93 may also include storage located away from the processors that make up the processor 92.
  • the processor 92 can access the memory 93 via an I/O (Input/Output) interface.
  • processors in each device in the above-mentioned embodiments execute one or more programs including a set of instructions for causing a computer to execute the algorithm described in the drawings. This process realizes the information processing described in each embodiment.
  • the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments.
  • the program may be stored on a non-transitory computer-readable medium or tangible storage medium.
  • computer-readable medium or tangible storage medium may include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disk (DVD), Blu-ray® disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device.
  • the program may be transmitted on a transitory computer-readable medium or communication medium.
  • transitory computer-readable medium or communication medium may include electrical, optical, acoustic, or other forms of propagated signals.
  • the first feature amount is calculated using information indicating that a point is present in each of the first small regions, that a point is not present, or that the presence of a point is unknown. 2.
  • the change detection method of claim 1. (Appendix 3) Further calculating a feature amount of each coordinate in a first neighborhood of the first coordinate in the first point cloud, in a third neighborhood including the coordinate, the third neighborhood being composed of a plurality of third small regions, using information regarding the presence of a point in each of the third small regions; determining whether or not a point that does not exist in the first coordinates is present in the second coordinates by using a feature amount of each coordinate in the first neighboring region, the first feature amount, and the second feature amount; 3.
  • the first point cloud data is data acquired using a first sensor, the size of the first neighborhood area is increased as the distance between the position indicated by the first coordinates and the position of the first sensor increases when the first point cloud data is acquired; 4.
  • the second point cloud data is data acquired by a second sensor, Regarding the second small region in which there is no point in the second point cloud, if the second small region is located between a position where a point exists in the second point cloud at the time of acquiring the data and the position of the second sensor, it is defined that there is no point in the second small region, and if the second small region is not located between a position where a point exists in the second point cloud and the position of the second sensor, the presence of a point in the second small region is defined as unknown. 7.
  • a change detection method according to any one of claims 1 to 6.
  • the first point cloud data is data acquired by a first sensor, Regarding the first small region in which there is no point in the first point cloud, if the first small region is located between a position where a point exists in the first point cloud at the time of acquiring the data and the position of the first sensor, it is defined that there is no point in the first small region, and if the first small region is not located between a position where a point exists in the first point cloud and the position of the first sensor, the presence of a point in the first small region is defined as unknown. 3.
  • the second point cloud data is data acquired using a second sensor, the size of the second neighborhood area is increased as the distance between the position indicated by the second coordinates and the position of the second sensor increases when the second point cloud data is acquired. 6.
  • (Appendix 12) determining whether a point that exists at the first coordinates does not exist at the second coordinates by calculating a similarity between the first feature amount and the second feature amount and, for each coordinate in the second neighboring region, a similarity between the feature amount of the coordinate and the first feature amount; a similarity between the first feature amount and the second feature amount is calculated using elements of the second feature amount in each of the second small region in the second region and a second small region included in a peripheral region of the second small region, and elements of the first small region in the first region corresponding to the second small region; a similarity between the feature amount of each coordinate in the second neighboring region and the first feature amount is calculated using an element of the fourth feature amount in each of the fourth small region in the fourth region and a small region included in a peripheral region of the fourth small region, and an element of the first small region in the first region corresponding to the fourth small region; 12.
  • the change detection method according to claim 5 or 11.
  • Appendix 13 a first feature amount calculation means for calculating a first feature amount of a first coordinate in a first point cloud using information on the presence of a point in each of the first small regions, the first region being composed of a plurality of first small regions and including the first coordinate; a second feature amount calculation means for calculating a second feature amount of a second coordinate in a second point group corresponding to the first coordinate, using information indicating that a point exists, that a point does not exist, or that the existence of a point is unknown in a second area including the second coordinate, the second area being composed of a plurality of second small areas; a determination means for determining whether or not a change in the presence or absence of a point has occurred between the first coordinates and the second coordinates by using the first feature amount and the second feature amount;
  • a change detection system comprising: (Appendix 14) the first feature amount calculation means calculates the first feature amount using information indicating that a point is present in each of the first small
  • the first feature amount calculation means further calculates feature amounts of each coordinate in a first neighborhood area of the first coordinate in the first point cloud, in a third area including the coordinate, the third area being composed of a plurality of third small areas, using information regarding the presence of points in each of the third small areas; the determining means determines whether or not a point that does not exist in the first coordinates is present in the second coordinates by using a feature amount of each coordinate in the first neighboring region, the first feature amount, and the second feature amount. 15.
  • the first point cloud data is data acquired using a first sensor
  • the first feature amount calculation means increases a size of the first neighboring region as a distance between a position indicated by the first coordinates and a position of the first sensor increases when the first point cloud data is acquired; 16.
  • the second feature amount calculation means further calculates feature amounts of each coordinate in a second neighboring region of the second coordinate in the second point cloud, the fourth region being composed of a plurality of fourth small regions and including the coordinate, using information indicating that a point exists in each of the fourth small regions, that a point does not exist, or that the existence of a point is unknown; the determining means determines whether or not a point existing at the first coordinates does not exist at the second coordinates by using a feature amount of each coordinate in the second neighboring region, the first feature amount, and the second feature amount. 17.
  • a change detection system according to any one of claims 13 to 16.
  • (Appendix 18) a detection unit that detects a change in the presence or absence of an object between the first point cloud and the second point cloud by performing the determination at a plurality of corresponding coordinates in the first point cloud and the second point cloud; An output unit that outputs the result of the detection, 18.
  • a change detection system according to any one of claims 13 to 17.
  • the second point cloud data is data acquired by a second sensor, Regarding the second small region in which there is no point in the second point cloud, if the second small region is located between a position where a point exists in the second point cloud at the time of acquiring the data and the position of the second sensor, it is defined that there is no point in the second small region, and if the second small region is not located between a position where a point exists in the second point cloud and the position of the second sensor, the presence of a point in the second small region is defined as unknown. 19.
  • a change detection system according to any one of claims 13 to 18.
  • Appendix 20 An extraction unit that compares the first point cloud with the second point cloud and extracts coordinates where a point is present or absent in the first point cloud and the second point cloud, At least one of the first coordinates and the second coordinates is a coordinate extracted by the extraction unit. 20.
  • a change detection system according to any one of claims 13 to 19.
  • the first point cloud data is data acquired by a first sensor, Regarding the first small region in which there is no point in the first point cloud, if the first small region is located between a position where a point exists in the first point cloud at the time of acquiring the data and the position of the first sensor, it is defined that there is no point in the first small region, and if the first small region is not located between a position where a point exists in the first point cloud and the position of the first sensor, the presence of a point in the first small region is defined as unknown. 15.
  • the determining means determines whether or not a point that does not exist in the first coordinates is present in the second coordinates by calculating a similarity between the first feature amount and the second feature amount and, for each coordinate in the first neighboring region, a similarity between the feature amount of the coordinate and the second feature amount; a similarity between the first feature amount and the second feature amount is calculated using elements of the first feature amount in each of the first small region in the first region and a first small region included in a peripheral region of the first small region, and elements of the second small region in the second region corresponding to the first small region; a similarity between the feature amount of each coordinate in the first neighboring region and the second feature amount is calculated using elements of the first feature amount in each of the third small region in the third region and a small region included in a peripheral region of the third small region, and elements of the second small region in the second region corresponding to the third small region; 17.
  • the second point cloud data is data acquired using a second sensor
  • the second feature amount calculation means increases a size of the second neighborhood area as a distance between a position indicated by the second coordinates and a position of the second sensor increases when the second point cloud data is acquired; 18.
  • the determining means determines whether or not a point existing at the first coordinates does not exist at the second coordinates by calculating a similarity between the first feature amount and the second feature amount and, for each coordinate in the second neighboring region, a similarity between the feature amount of the coordinate and the first feature amount; a similarity between the first feature amount and the second feature amount is calculated using elements of the second feature amount in each of the second small region in the second region and a second small region included in a peripheral region of the second small region, and elements of the first small region in the first region corresponding to the second small region; a similarity between the feature amount of each coordinate in the second neighboring region and the first feature amount is calculated using an element of the fourth feature amount in each of the fourth small region in the fourth region and a small region included in a peripheral region of the fourth small region, and an element of the first small region in the first region corresponding to the fourth small region; 24.
  • the change detection system of claim 17 or 23 (Appendix 25) a first feature amount calculation means for calculating a first feature amount of a first coordinate in a first point cloud using information on the presence of a point in each of the first small regions, the first region being composed of a plurality of first small regions and including the first coordinate; a second feature amount calculation means for calculating a second feature amount of a second coordinate in a second point group corresponding to the first coordinate, using information indicating that a point exists, that a point does not exist, or that the existence of a point is unknown in a second area including the second coordinate, the second area being composed of a plurality of second small areas; a determination means for determining whether or not a change in the presence or absence of a point has occurred between the first coordinates and the second coordinates by using the first feature amount and the second feature amount;
  • a change detection device comprising: (Appendix 26) the first feature amount calculation means calculates the first feature amount using information indicating that a point is present in each of the first small regions
  • the change detection apparatus of claim 25 (Appendix 27) the first feature amount calculation means further calculates feature amounts of each coordinate in a first neighborhood area of the first coordinate in the first point cloud, in a third area including the coordinate, the third area being composed of a plurality of third small areas, using information regarding the presence of points in each of the third small areas; the determining means determines whether or not a point that does not exist in the first coordinates is present in the second coordinates by using a feature amount of each coordinate in the first neighboring region, the first feature amount, and the second feature amount. 27.
  • the change detection device of claim 25 or 26 The change detection device of claim 25 or 26.
  • the first point cloud data is data captured by a first sensor
  • the first feature amount calculation means increases a size of the first neighboring region as a distance between a position indicated by the first coordinates and a position of the first sensor at the time of acquiring the data increases; 28.
  • the second feature amount calculation means further calculates feature amounts of each coordinate in a second neighboring region of the second coordinate in the second point cloud, the fourth region being composed of a plurality of fourth small regions and including the coordinate, using information indicating that a point exists in each of the fourth small regions, that a point does not exist, or that the existence of a point is unknown; the determining means determines whether or not a point existing at the first coordinates does not exist at the second coordinates by using a feature amount of each coordinate in the second neighboring region, the first feature amount, and the second feature amount. 29.
  • a change detection device according to any one of claims 25 to 28.
  • Appendix 30 An extraction unit that compares the first point cloud with the second point cloud and extracts coordinates where a point is present or absent in the first point cloud and the second point cloud, At least one of the first coordinates and the second coordinates is a coordinate extracted by the extraction unit. 30.
  • a change detection device according to any one of claims 25 to 29.
  • the second point cloud data is data acquired by a second sensor, Regarding the second small region in which there is no point in the second point cloud, if the second small region is located between a position where a point exists in the second point cloud at the time of acquiring the data and the position of the second sensor, it is defined that there is no point in the second small region, and if the second small region is not located between a position where a point exists in the second point cloud and the position of the second sensor, the presence of a point in the second small region is defined as unknown. 31.
  • a change detection device according to any one of claims 25 to 30.
  • the first point cloud data is data acquired by a first sensor, Regarding the first small region in which there is no point in the first point cloud, if the first small region is located between a position where a point exists in the first point cloud at the time of acquiring the data and the position of the first sensor, it is defined that there is no point in the first small region, and if the first small region is not located between a position where a point exists in the first point cloud and the position of the first sensor, the presence of a point in the first small region is defined as unknown.
  • the change detection apparatus of claim 26 is if the first small region is located between a position where a point exists in the first point cloud at the time of acquiring the data and the position of the first sensor, it is defined that there is no point in the first small region, and if the first small region is not located between a position where a point exists in the first point cloud and the position of the first sensor, the presence of a point in the first small region is defined as unknown.
  • the determining means determines whether or not a point that does not exist in the first coordinates is present in the second coordinates by calculating a similarity between the first feature amount and the second feature amount and, for each coordinate in the first neighboring region, a similarity between the feature amount of the coordinate and the second feature amount; a similarity between the first feature amount and the second feature amount is calculated using elements of the first feature amount in each of the first small region in the first region and a first small region included in a peripheral region of the first small region, and elements of the second small region in the second region corresponding to the first small region; a similarity between the feature amount of each coordinate in the first neighboring region and the second feature amount is calculated using elements of the first feature amount in each of the third small region in the third region and a small region included in a peripheral region of the third small region, and elements of the second small region in the second region corresponding to the third small region; 29.
  • the change detection apparatus of claim 27 or 28 (Appendix 34) the second point cloud data is data acquired using a second sensor, the second feature amount calculation means increases a size of the second neighborhood area as a distance between a position indicated by the second coordinates and a position of the second sensor increases when the second point cloud data is acquired; 30.
  • the change detection apparatus of claim 29 (Appendix 34) the second point cloud data is data acquired using a second sensor, the second feature amount calculation means increases a size of the second neighborhood area as a distance between a position indicated by the second coordinates and a position of the second sensor increases when the second point cloud data is acquired; 30.
  • the change detection apparatus of claim 29 The change detection apparatus of claim 29.
  • the determining means determines whether or not a point existing at the first coordinates does not exist at the second coordinates by calculating a similarity between the first feature amount and the second feature amount and, for each coordinate in the second neighboring region, a similarity between the feature amount of the coordinate and the first feature amount; a similarity between the first feature amount and the second feature amount is calculated using elements of the second feature amount in each of the second small region in the second region and a second small region included in a peripheral region of the second small region, and elements of the first small region in the first region corresponding to the second small region; a similarity between the feature amount of each coordinate in the second neighboring region and the first feature amount is calculated using an element of the fourth feature amount in each of the fourth small region in the fourth region and a small region included in a peripheral region of the fourth small region, and an element of the first small region in the first region corresponding to the fourth small region; 35.
  • the change detection apparatus of claim 29 or 34 (Appendix 36) Calculating a first feature amount of a first coordinate in a first point cloud using information about the presence of a point in each of the first small regions, the first region being composed of a plurality of first small regions and including the first coordinate; calculating a second feature amount of a second coordinate in a second point cloud corresponding to the first coordinate, using information indicating that a point exists, that a point does not exist, or that the existence of a point is unknown in a second region that is composed of a plurality of second small regions and includes the second coordinate; determining whether or not a change in the presence or absence of a point has occurred between the first coordinates and the second coordinates by using the first feature amount and the second feature amount; A non-transitory computer-readable medium on which a program for causing a computer to execute a process is stored.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Electromagnetism (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Life Sciences & Earth Sciences (AREA)
  • Geophysics (AREA)
  • Image Analysis (AREA)
PCT/JP2022/038830 2022-10-18 2022-10-18 変化検出方法、変化検出システム及び変化検出装置 Ceased WO2024084601A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2024551110A JPWO2024084601A1 (https=) 2022-10-18 2022-10-18
PCT/JP2022/038830 WO2024084601A1 (ja) 2022-10-18 2022-10-18 変化検出方法、変化検出システム及び変化検出装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/038830 WO2024084601A1 (ja) 2022-10-18 2022-10-18 変化検出方法、変化検出システム及び変化検出装置

Publications (1)

Publication Number Publication Date
WO2024084601A1 true WO2024084601A1 (ja) 2024-04-25

Family

ID=90737075

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/038830 Ceased WO2024084601A1 (ja) 2022-10-18 2022-10-18 変化検出方法、変化検出システム及び変化検出装置

Country Status (2)

Country Link
JP (1) JPWO2024084601A1 (https=)
WO (1) WO2024084601A1 (https=)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009064287A (ja) * 2007-09-07 2009-03-26 Meidensha Corp 侵入者検知装置
JP2016217941A (ja) * 2015-05-22 2016-12-22 株式会社東芝 ３次元データ評価装置、３次元データ測定システム、および３次元計測方法
JP2018181056A (ja) * 2017-04-17 2018-11-15 富士通株式会社 差分検知プログラム、差分検知装置、差分検知方法
JP2019046295A (ja) * 2017-09-05 2019-03-22 三菱電機株式会社 監視装置
JP2020166516A (ja) * 2019-03-29 2020-10-08 田中　成典 点群データ管理システム
JP2022098432A (ja) * 2020-12-21 2022-07-01 コモンウェルス サイエンティフィック アンド インダストリアル リサーチ オーガナイゼーション 車両ナビゲーション

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009064287A (ja) * 2007-09-07 2009-03-26 Meidensha Corp 侵入者検知装置
JP2016217941A (ja) * 2015-05-22 2016-12-22 株式会社東芝 ３次元データ評価装置、３次元データ測定システム、および３次元計測方法
JP2018181056A (ja) * 2017-04-17 2018-11-15 富士通株式会社 差分検知プログラム、差分検知装置、差分検知方法
JP2019046295A (ja) * 2017-09-05 2019-03-22 三菱電機株式会社 監視装置
JP2020166516A (ja) * 2019-03-29 2020-10-08 田中　成典 点群データ管理システム
JP2022098432A (ja) * 2020-12-21 2022-07-01 コモンウェルス サイエンティフィック アンド インダストリアル リサーチ オーガナイゼーション 車両ナビゲーション

Also Published As

Publication number Publication date
JPWO2024084601A1 (https=) 2024-04-25

Similar Documents

Publication Publication Date Title
US12455367B2 (en) Ego motion estimation based on consecutive time frames input to machine learning model
JP5671281B2 (ja) 位置姿勢計測装置、位置姿勢計測装置の制御方法及びプログラム
US12175699B2 (en) Confidence determination of 3D point cloud data acquired by LIDAR sensor
CN111079619A (zh) 用于检测图像中的目标对象的方法和装置
KR20200040665A (ko) 컨볼루션 신경망을 이용하여 poi 변화를 검출하기 위한 시스템 및 방법
JP7427615B2 (ja) 情報処理装置、情報処理方法およびプログラム
JP2015127664A (ja) キャリブレーション装置、キャリブレーション方法およびキャリブレーションプログラム
JP2009237845A (ja) 情報処理装置、および情報処理方法、並びにコンピュータ・プログラム
US11004211B2 (en) Imaging object tracking system and imaging object tracking method
JP5976089B2 (ja) 位置姿勢計測装置、位置姿勢計測方法、およびプログラム
US11922667B2 (en) Object region identification device, object region identification method, and object region identification program
US12002193B2 (en) Inspection device for inspecting a building or structure
Shao et al. DIMMC: A 3D vision approach for structural displacement measurement using a moving camera
Das et al. Camera motion compensation and person detection in construction site using yolo-bayes model
WO2024084601A1 (ja) 変化検出方法、変化検出システム及び変化検出装置
Vaida et al. Automatic extrinsic calibration of LiDAR and monocular camera images
WO2024224647A1 (ja) 変化検出方法、変化検出システム及び変化検出装置
WO2021186640A1 (ja) 劣化検出装置、劣化検出システム、劣化検出方法、およびプログラム
CN117606500A (zh) 减速带检测方法、通过方法、网络训练方法及相关装置
JP2023107676A (ja) 情報処理装置、情報処理方法及びプログラム
KR102139667B1 (ko) 정보 획득 방법 및 디바이스.
US20240144624A1 (en) Measurement apparatus, measurement system, and measurement method
US20240127480A1 (en) Information processing apparatus, information processing method, and program
JP7443303B2 (ja) 計測装置、計測方法及びプログラム
US20250095190A1 (en) Method, computing device, and recording medium for detecting object in real time based on lidar point cloud

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22962713

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024551110

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22962713

Country of ref document: EP

Kind code of ref document: A1