US20220170739A1 - Surface abnormality detection device and system - Google Patents

Surface abnormality detection device and system Download PDF

Info

Publication number
US20220170739A1
US20220170739A1 US17/599,747 US201917599747A US2022170739A1 US 20220170739 A1 US20220170739 A1 US 20220170739A1 US 201917599747 A US201917599747 A US 201917599747A US 2022170739 A1 US2022170739 A1 US 2022170739A1
Authority
US
United States
Prior art keywords
reflection brightness
distance measurement
cluster
value
points
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
US17/599,747
Other languages
English (en)
Inventor
Yoshimasa Ono
Akira Tsuji
Hidemi Noguchi
Junichi Abe
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Assigned to NEC CORPORATION reassignment NEC CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: ONO, YOSHIMASA, NOGUCHI, HIDEMI, ABE, JUNICHI, TSUJI, AKIRA
Publication of US20220170739A1 publication Critical patent/US20220170739A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/30—Measuring arrangements characterised by the use of optical techniques for measuring roughness or irregularity of surfaces
    • G01B11/303—Measuring arrangements characterised by the use of optical techniques for measuring roughness or irregularity of surfaces using photoelectric detection means
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/30—Measuring arrangements characterised by the use of optical techniques for measuring roughness or irregularity of surfaces
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C3/00—Measuring distances in line of sight; Optical rangefinders
    • G01C3/02—Details
    • G01C3/06—Use of electric means to obtain final indication
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
    • G01N21/9515—Objects of complex shape, e.g. examined with use of a surface follower device
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
    • G01N21/956—Inspecting patterns on the surface of objects
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/4802—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00 using analysis of echo signal for target characterisation; Target signature; Target cross-section
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8854—Grading and classifying of flaws
    • G01N2021/8861—Determining coordinates of flaws
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds

Definitions

  • the present disclosure relates to a surface abnormality detection device and a system, and more particularly, to a surface abnormality detection device, and a system, capable of detecting an abnormal portion having a displacement below the distance measurement accuracy when detecting the abnormal portion on the surface of a structure.
  • a laser distance measurement device is a device capable of acquiring a three-dimensional structure of an object in the surroundings of the device, and often has a function of measuring received light brightness of laser light in addition to the information of a three-dimensional object point.
  • the received light brightness i.e., reflection brightness from the object depends on the state of the object surface to which the laser light is emitted. This enables a portion of an abnormality such as rust or peeling of coating on the surface to be detected through the processing of the received light brightness information acquired by the laser distance measurement device.
  • the received light brightness acquired by the laser distance measurement device is referred to as “reflection brightness.”
  • Patent Literature 1 discloses that a minimum curvature direction estimation unit estimates a minimum curvature direction for each region, an autocorrelation value calculation unit calculates an autocorrelation value of a feature amount of a partial region for each region, a sweep shape candidate region determination unit determines that each region in which the autocorrelation value is larger than a threshold is a sweep shape candidate region, a region integration processing unit integrates the regions determined to be the sweep shape candidate regions, and a sweep shape determination unit determines whether the integrated region has a sweep shape.
  • Non Patent Literature 1 the roughness of a surface to be observed is measured based on position information of a point cloud. To obtain the surface roughness, an average curved surface is locally calculated, so that the displacement of the point cloud from the plane can be calculated as roughness.
  • a roughness value of the ground surface observed from the distance measurement device of an aircraft is measured with centimeter (cm) scale accuracy.
  • Non Patent Literature 2 and Non Patent Literature 3 the recognition of an object under measurement and the determination of materials are attempted by focusing on the information of reflection brightness of the point cloud. There is adopted a method of correcting the reflection brightness acquired by the laser distance measurement device by assuming modeling based on the radar equation and modeling of the bidirectional reflectance distribution function for the reflection brightness of the acquired point cloud.
  • Patent Literature 1 does not disclose that the abnormal portion on the surface of the structure is detected.
  • the deterioration such as rust or peeling of coating on the surface is a displacement that is below approximately 1 millimeter (mm), and is finer than the accuracy of the laser distance measurement device which is widely used at present. Therefore, it is difficult to identify these abnormal portions on the surface from the roughness based on the position information of the point cloud.
  • the absorption property and reflection anisotropy of the laser light are different from surface to surface of the equipment, it is difficult to determine the abnormal portion based on the information of only the reflection brightness. For example, when the abnormal portion is determined based on a uniform threshold for the reflection brightness, all of the used surface materials having a strong absorption of the laser light wavelength are determined as the abnormal portions.
  • the modeling of the reflection and absorption properties with respect to the all the surfaces of the equipment in the facility is not a realistic method.
  • an object of the present disclosure is to provide a surface abnormality detection device, a system, and a method.
  • a surface abnormality detection device includes:
  • a classification means for classifying an object under measurement into one or more clusters having the same structure, based on position information at a plurality of points on a surface of the object under measurement;
  • a determination means for determining a reflection brightness normal value of the cluster based on a distribution of reflection brightness values at a plurality of points on a surface of the cluster
  • an identification means for identifying an abnormal portion on the surface of the cluster based on a difference between the reflection brightness normal value and the reflection brightness value at each of the plurality of points on the surface of the cluster.
  • a surface abnormality detection device includes:
  • a first calculation means for calculating a first incident angle of a laser for each of a plurality of distance measurement points based on position information of a first observation point, and position information, included in first point cloud data, of the plurality of distance measurement points of a surface of an object under measurement;
  • a second calculation means for calculating a second incident angle of a laser for each of the plurality of distance measurement points based on position information of a second observation point, and position information of the plurality of distance measurement points included in second point cloud data;
  • a position control means for making an adjustment to match positions for each of the plurality of distance measurement points based on the position information of the plurality of distance measurement points in the first point cloud data and the position information of the plurality of distance measurement points in the second point cloud data;
  • a brightness difference calculation means for calculating, for each of the plurality of distance measurement points, a reflection brightness difference value which is a difference between a first reflection brightness value at each of the plurality of distance measurement points in the first point cloud data after the position adjustment and a second reflection brightness value at each of the plurality of distance measurement points in the second point cloud data after the position adjustment;
  • a correction means for calculating, for each of the plurality of distance measurement points, an incident angle difference which is a difference between the first incident angle at each of the plurality of distance measurement points in the first point cloud data after the position adjustment and the second incident angle at each of the plurality of distance measurement points in the second point cloud data after the position adjustment, and correcting, for each of the plurality of distance measurement points, the reflection brightness difference value based on the incident angle difference;
  • an identification means for identifying an abnormal portion of the object under measurement based on the reflection brightness difference value after the correction.
  • a surface abnormality detection device includes:
  • a position control means for making an adjustment to match positions for each of a plurality of distance measurement points based on position information of the plurality of distance measurement points on a surface of an object under measurement, the position information being included in cloud data for evaluation and position information of the plurality of distance measurement points included in cloud data for comparison;
  • a brightness difference calculation means for calculating, for each of the plurality of distance measurement points, a reflection brightness difference value which is a difference between a reflection brightness value for evaluation at each of the plurality of distance measurement points in the cloud data for evaluation after the position adjustment and a reflection brightness value for comparison at each of the plurality of distance measurement points in the cloud data for comparison after the position adjustment;
  • an identification means for identifying an abnormal portion of the object under measurement based on the reflection brightness difference value.
  • a surface abnormality detection device includes:
  • a classification means for evaluation for classifying an object under measurement into one or more clusters having the same structure, based on position information at a plurality of distance measurement points on a surface of the object under measurement included in cloud data for evaluation;
  • a classification means for comparison for classifying the object under measurement into one or more clusters having the same structure, based on position information at the plurality of distance measurement points included in cloud data for comparison;
  • a determination means for comparison for determining a reflection brightness normal value for each cluster of the cloud data for comparison based on a distribution of reflection brightness values at the plurality of distance measurement points of the cluster of the cloud data for comparison;
  • control means for associating the cluster of the cloud data for evaluation with the cluster of the cloud data for comparison recognized as having the same structure, based on the position information of the plurality of distance measurement points of the cluster of the cloud data for evaluation and the position information of the plurality of distance measurement points of the cluster of the cloud data for comparison;
  • a calculation means for calculating a reflection brightness normal difference value which is a difference between the reflection brightness value at each of the plurality of distance measurement points of the cluster of the cloud data for evaluation and the reflection brightness normal value of the cluster of the cloud data for comparison corresponding to the cluster of the cloud data for evaluation;
  • an identification means for identifying, for each cluster, an abnormal portion on the surface of the object under measurement based on the reflection brightness normal difference value.
  • a system according to the present disclosure includes:
  • the measurement device acquires
  • the surface abnormality detection device includes:
  • a surface abnormality detection device capable of detecting an abnormal portion having a displacement below the distance measurement accuracy when detecting the abnormal portion on the surface of a structure.
  • FIG. 1 is a block diagram illustrating a surface abnormality detection device according to a first example embodiment.
  • FIG. 2 is a block diagram illustrating a system according to the first example embodiment.
  • FIG. 3 is a diagram illustrating the classification of an object under measurement into the same structure, according to the first example embodiment.
  • FIG. 4 is a diagram illustrating an abnormal portion in the classified structure, according to the first example embodiment.
  • FIG. 5 is a flowchart illustrating an operation of the surface abnormality detection device according to the first example embodiment.
  • FIG. 6 is a flowchart illustrating an operation of a surface abnormality detection device according to a second example embodiment.
  • FIG. 7 is a diagram illustrating a laser incident angle according to a third example embodiment.
  • FIG. 8 is a flowchart illustrating an operation of a surface abnormality detection device according to the third example embodiment.
  • FIG. 9 is a diagram illustrating further classification (division) of a cluster based on a laser incident angle according to a fourth example embodiment.
  • FIG. 10 is a flowchart illustrating an operation of a surface abnormality detection device according to the fourth example embodiment.
  • FIG. 11 is a flowchart illustrating an operation of a surface abnormality detection device according to a fifth example embodiment.
  • FIG. 12 is a flowchart illustrating an operation of a surface abnormality detection device according to a sixth example embodiment.
  • FIG. 13 is a diagram illustrating a distance measurement device and an object under measurement according to a seventh example embodiment.
  • FIG. 14 is a block diagram illustrating a surface abnormality detection device according to the seventh example embodiment.
  • FIG. 15 is a flowchart illustrating an operation of the surface abnormality detection device according to the seventh example embodiment.
  • FIG. 16 is a block diagram illustrating a surface abnormality detection device according to an eighth example embodiment.
  • FIG. 17 is a flowchart illustrating an operation of the surface abnormality detection device according to the eighth example embodiment.
  • FIG. 1 is a block diagram illustrating the surface abnormality detection device according to the first example embodiment.
  • FIG. 2 is a block diagram illustrating the system according to the first example embodiment.
  • a surface abnormality detection device 11 includes a classification means 111 , a determination means 112 , and an identification means 113 .
  • the classification means 111 classifies an object under measurement into a cluster having the same structure, based on position information at a plurality of points on a surface of the object under measurement.
  • the position information can be represented as position information on three-dimensional coordinates, for example.
  • the determination means 112 determines a reflection brightness normal value of the cluster based on the distribution of reflection brightness values at the plurality of points on the surface of the cluster. In the case where there are a plurality of classified clusters, the determination means 112 determines a reflection brightness normal value for each of the plurality of clusters.
  • the identification means 113 identifies an abnormal portion on the surface of the cluster based on a difference between the reflection brightness normal value and the reflection brightness value at each of the plurality of points on the surface of the cluster. In the case where there are a plurality of classified clusters, the identification means 113 identifies an abnormal portion on the surface for each of the plurality of clusters.
  • the data including the position information at the plurality of points on the surface of the object under measurement and the reflection brightness value at each position is referred to as point cloud data of the object under measurement.
  • the data including the position information at the plurality of points on the surface of the cluster and the reflection brightness value at each position is referred to as point cloud data of the cluster.
  • the cluster will be described later.
  • a system 10 includes a distance measurement device 12 and the surface abnormality detection device 11 .
  • the distance measurement device may be also referred to as a measurement device.
  • the distance measurement device 12 includes a laser distance measurement device or the like, and acquires three-dimensional shape data of a surrounding object including the object under measurement.
  • the surface abnormality detection device 11 acquires the three-dimensional shape data from the distance measurement device 12 and identifies a portion where the surface state is abnormal, from the acquired three-dimensional shape data.
  • the three-dimensional shape data acquired by the distance measurement device 12 is acquired as “three-dimensional point cloud data” including the position information on three-dimensional coordinates of the plurality of points on the surface of the object (object under measurement) and the information of the reflection brightness (reflection brightness value) at each position.
  • the processing on the “three-dimensional point cloud data” will be described, but the present invention is not limited thereto.
  • the example embodiment is applicable to the point cloud data which includes the space information capable of identifying three-dimensional coordinates and the reflection brightness at the coordinates (position).
  • the three-dimensional point cloud data may be also referred to as three-dimensional data or point cloud data.
  • the plurality of points on the surface of the object under measurement may be also referred to as a point cloud.
  • FIG. 3 is a diagram illustrating the classification of the object under measurement into the same structure, according to the first example embodiment.
  • FIG. 4 is a diagram illustrating an abnormal portion in the classified structure, according to the first example embodiment.
  • a point cloud PC 10 illustrated in FIG. 3 indicates a point cloud on the surface of the object under measurement.
  • the distance measurement device 12 acquires the point cloud data including the position information on three-dimensional coordinates and the reflection brightness in the point cloud PC 10 . That is, the point cloud data acquired by the distance measurement device 12 is expressed as in the point cloud PC 10 as a set of points including the position information on three-dimensional coordinates and the reflection brightness. Since a plurality of structures whose surface states are different in paint or the like exist in the point cloud PC 10 , it is difficult to determine the abnormal portion on the surface by a uniform reflection brightness value.
  • the surface abnormality detection device 11 divides and classifies the point cloud constituting the same structure into clusters by a clustering process based on the position information on three-dimensional coordinates.
  • a point cloud PC 11 illustrated in FIG. 3 indicates a part of clusters after the clustering process.
  • a cluster C 101 and a cluster C 102 of the point cloud PC 11 are point clouds which are determined and classified as different structures.
  • examples of algorithms of the clustering include a method of determining the same cluster by using Euclidean distance as a threshold, and a region growth method of determining the same cluster based on the continuity of angles of perpendicular lines among neighboring points.
  • a point cloud PC 12 illustrated in FIG. 4 indicates a partial cluster C 102 p , which is a cluster which includes many point clouds in which the reflection brightness value is below a predetermined threshold, in the cluster C 102 .
  • the partial cluster C 102 p includes many point clouds in which the reflection brightness is weaker than the others. Note that the point clouds other than the cluster C 102 are not illustrated for simplification purposes.
  • a reflection brightness distribution G 11 illustrated in FIG. 4 represents the reflection brightness values of the point cloud corresponding to the cluster C 102 by a histogram.
  • the cluster C 102 for example, a portion (abnormal portion) whose surface has become rough due to rust has the reflection brightness that is lower than that at the other portions.
  • an abnormal portion there is a method of calculating an approximate curve L 101 with respect to the histogram of the portion in which the paint remains, and determining the point cloud having the reflection brightness deviating from the approximate curve L 101 as the point cloud having the abnormal surface.
  • This determination method enables separation of a histogram H 101 having normal reflection brightness values from a histogram H 102 having abnormal reflection brightness.
  • the abnormal portion in the structure can be determined by recognizing the point cloud corresponding to the histogram H 102 as the abnormal portion.
  • FIG. 5 is a flowchart illustrating the operation of the surface abnormality detection device according to the first example embodiment.
  • the surface abnormality detection device 11 acquires the three-dimensional point cloud data (step S 101 ).
  • the surface abnormality detection device 11 performs the clustering process based on the position information of the three-dimensional point cloud data, and classifies the three-dimensional point cloud data into the point cloud having the same structure, i.e., the cluster (step S 102 ).
  • the surface abnormality detection device 11 determines a normal value of the reflection brightness of the point cloud (cluster) classified into the same structure, based on the reflection brightness distribution of the point cloud (step S 103 ). In the case where there are a plurality of point clouds classified into the same structure, a normal value of the reflection brightness is determined for each of the plurality of point clouds.
  • the normal value of the reflection brightness is referred to as a reflection brightness normal value.
  • step S 104 determines the point cloud as the abnormal portion on the surface (step S 105 ). That is, when, a difference between the reflection brightness value of a point cloud among a plurality of points on the surface of a point cloud (cluster) and the reflection brightness normal value of the point cloud exceeds the threshold, the surface abnormality detection device 11 determines, as the abnormal portion, the point cloud (or the point) in this case.
  • the surface abnormality detection device 11 determines the point cloud as the normal portion on the surface (step S 106 ). That is, when, a difference between the reflection brightness value of a point cloud among a plurality of points on the surface of a point cloud (cluster) and the reflection brightness normal value of the point cloud is below the threshold, the surface abnormality detection device 11 determines, as the normal portion, the point cloud (or the point) in this case.
  • the surface abnormality detection device 11 of the first example embodiment can identify the abnormal portion on the surface from the three-dimensional point cloud data including the reflection brightness. This makes it possible to identify the abnormal portion for the surface roughness finer than the distance measurement accuracy of the distance measurement device 12 , and reduce false detection. Furthermore, a portion where the reflection brightness is abnormal is identified on a per structure basis, whereby the abnormal portion can be identified for the structures whose surface states are different.
  • a surface abnormality detection device capable of detecting an abnormal portion having a displacement below the distance measurement accuracy when detecting the abnormal portion on the surface of a structure.
  • a surface abnormality detection device 21 according to a second example embodiment is different from the surface abnormality detection device 11 according to the first example embodiment in that the reflection brightness attenuation caused according to the distance between a point cloud and an observation point is corrected for the reflection brightness of the point cloud.
  • the surface abnormality detection device 21 When a part of the structure (object under measurement) extends in a depth direction as viewed from the observation point, a light propagation distance is different between near point cloud and far point cloud on the surface of the structure. As a result, since the attenuation is caused by light absorption and light scattering, the reflection brightness changes. Therefore, as compared with the surface abnormality detection device 11 , the surface abnormality detection device 21 performs an additional process of correcting the reflection brightness according to the distance between the point cloud and the observation point. In this way, the surface abnormality detection device 21 can identify the abnormal portion with higher accuracy than in the surface abnormality detection device 11 .
  • FIG. 6 is a flowchart illustrating the operation of the surface abnormality detection device according to the second example embodiment.
  • the surface abnormality detection device 21 corrects the reflection brightness value of each point cloud based on the distance from the observation point to the point cloud (step S 201 ). That is, the reflection brightness value is corrected based on an attenuation amount due to the distance between the surface abnormality detection device 21 which is the observation point and the point (point cloud) on the surface of the cluster.
  • the reflection brightness value is a value obtained by correcting the attenuation amount based on the distance between the surface abnormality detection device 21 which is the observation point and the point (point cloud) on the surface of the cluster.
  • the reflection brightness may be corrected by performing the attenuation correction by the distance to the fourth power according to the radar equation.
  • the reflection brightness may be corrected by using an estimation value based on absorption by a propagation medium.
  • step S 201 After step S 201 , the processes from step S 103 to step S 106 are performed in a similar manner to those in the first example embodiment.
  • step S 201 is performed between step S 102 and step S 103 , but is not limited thereto.
  • the sequence of processes may be arbitrary when the requirement that step S 201 is performed before step S 103 is satisfied.
  • the surface abnormality detection device 21 according to the second example embodiment can identify the abnormal portion on the surface of the structure, in particular, the structure extending in the depth direction, with higher accuracy than in the surface abnormality detection device 11 according to the first example embodiment.
  • FIG. 7 is a diagram illustrating a laser incident angle according to a third example embodiment.
  • a surface abnormality detection device 31 according to a third example embodiment is different from the surface abnormality detection device 11 according to the first example embodiment in that a process of correcting reflection brightness relative to the laser incident angle at each point is added.
  • the angular dependence of the reflection brightness changes according to the nature of the surface.
  • the angular dependence of the reflection brightness of the laser reflected light changes according to the nature of the surface of the structure. Therefore, when the surface of the structure is curved, the abnormal portion on the surface of the structure can be identified with higher accuracy by correcting the reflection brightness.
  • a point cloud PC 32 illustrated in FIG. 7 is a schematic view in which a point cloud in a three-dimensional region R 31 in a point cloud PC 31 is enlarged.
  • the description will be made by way of example where a point cloud on a cylindrical pipe is used as the point cloud PC 31 .
  • a laser incident angle A 301 at a distance measurement point P 301 is calculated as an angle formed by a laser incident direction B 301 that connects the distance measurement point P 301 and an observation point (surface abnormality detection device 31 ) and a perpendicular line N 301 at the distance measurement point P 301 .
  • the perpendicular line N 301 is calculated by using a distance measurement point cloud in the surroundings of the distance measurement point P 301 .
  • FIG. 8 is a flowchart illustrating the operation of the surface abnormality detection device according to the third example embodiment.
  • step S 102 the processes from step S 101 to step S 102 are performed in a similar manner to those in the first example embodiment.
  • the surface abnormality detection device 31 calculates (estimates) the laser incident angle A 301 based on the laser incident direction at each point and the perpendicular line N 301 at each point (step S 301 ).
  • the position of the point cloud may be smoothed to reduce the dispersion of the perpendicular line N 301 due to an error of the distance measurement point P 301 .
  • the surface abnormality detection device 11 corrects (attenuates) the reflection brightness value at each point (distance measurement point) based on the laser incident angle A 301 (step S 302 ).
  • the reflection brightness value may be corrected by applying the known reflectance property, other than modeling of the bidirectional reflectance distribution function of the structure, or simple modeling assuming Lambertian reflection.
  • step S 302 the processes from step S 103 to step S 106 are performed in a similar manner to those in the first example embodiment.
  • step S 301 and step S 302 are performed between step S 102 and step S 103 , but is not limited thereto.
  • the sequence of processes may be arbitrary when the requirement that step S 301 and step S 302 are performed before step S 103 is satisfied.
  • the surface abnormality detection device 31 according to the third example embodiment can identify the abnormal portion on the surface of the structure, in particular, the curved structure, with higher accuracy than in the surface abnormality detection device 11 according to the first example embodiment.
  • FIG. 9 is a diagram illustrating further classification (division) of a cluster based on a laser incident angle according to a fourth example embodiment.
  • a point cloud PC 41 illustrated in FIG. 9 is a point cloud on a cylindrical pipe.
  • a surface abnormality detection device 41 identifies an abnormal portion by further dividing a point cloud in the cluster into point clouds having the same laser incident angle.
  • the description will be made by way of example where a point cloud on a cylindrical pipe is used as the point cloud PC 41 .
  • the surface abnormality detection device 41 further classifies (divides) the cluster into subclusters according to the laser incident angle.
  • a subcluster SC 401 into which the cluster is further classified includes a point cloud with a wide laser incident angle
  • a subcluster SC 402 includes a point cloud with a wide laser incident angle next to that of the subcluster SC 401 .
  • a reflection brightness distribution G 41 illustrated in FIG. 9 shows a histogram of reflection brightness values in the subcluster SC 401 .
  • a reflection brightness distribution G 42 illustrated in FIG. 9 shows a histogram of reflection brightness values in the subcluster SC 402 .
  • the surface abnormality detection device 41 extracts the abnormal value of the reflection brightness from each of the reflection brightness distribution G 41 and the reflection brightness distribution G 42 in a similar manner to the surface abnormality detection device 11 according to the first example embodiment. That is, the surface abnormality detection device 41 extracts, from each of the reflection brightness distribution G 41 and the reflection brightness distribution G 42 , the point cloud determined as the abnormal portion in which a difference between the reflection brightness value of the point cloud and the reflection brightness normal value exceeds the threshold. This makes it possible to identify a histogram H 422 in which the reflection brightness becomes the abnormal value.
  • the reflection brightness distributions having the normal value are calculated as an approximate curve L 411 and an approximate curve L 421 , respectively.
  • a histogram H 411 and a histogram H 421 in which the reflection brightness value becomes the normal value are identified by calculating the approximate curve L 411 and the approximate curve L 421 .
  • FIG. 10 is a flowchart illustrating the operation of the surface abnormality detection device according to the fourth example embodiment.
  • step S 301 is performed in a similar manner to that in the third example embodiment.
  • the surface abnormality detection device 41 further classifies the point cloud (cluster) classified as the same structure into the subclusters according to a value of the laser incident angle (step S 404 ). For example, the surface abnormality detection device 41 further classifies the cluster into the subcluster for each angle range of the laser incident angles in the point cloud of the cluster. When there are a plurality of point clouds, each point cloud is further classified according to the value of the laser incident angle.
  • the point cloud may be further classified by a fixed width with respect to the value of the laser incident angle.
  • the point cloud may be further classified by a width varying according to the reflection model or the number of points of the point cloud, with respect to the value of the laser incident angle.
  • the surface abnormality detection device 11 determines the normal value of the reflection brightness in each of the point clouds (subclusters) classified as being included in the same cluster (the same structure) and as having the same laser incident angle, based on the reflection brightness distribution of the point cloud (step S 402 ).
  • step S 402 the processes from step S 103 to step S 106 are performed in a similar manner to those in the first example embodiment.
  • the surface abnormality detection device 11 finally identifies the abnormal portion on the surface of the further classified point cloud (subcluster) based on the difference between the reflection brightness normal value of the classified point cloud (subcluster) and the reflection brightness value at each of the plurality of points on the surface of the classified point cloud (subcluster).
  • step S 301 is performed between step S 102 and step S 401 , but is not limited thereto.
  • the sequence of processes may be arbitrary when the requirement that step S 301 is performed before step S 404 is satisfied.
  • the surface abnormality detection device 41 can identify the abnormal portion on the surface with higher accuracy from the three-dimensional point cloud data having the reflection brightness in particular in a case where it is difficult to correct the reflection brightness using the laser incident angle with respect to the curved structure.
  • the surface abnormality detection device 41 according to the fourth example embodiment can identify the abnormal portion on the surface with higher accuracy than in the surface abnormality detection device 11 according to the third example embodiment.
  • a surface abnormality detection device 51 can determine an abnormal portion on a surface with higher accuracy by using identification of the abnormal portion on the surface that is determined based on a red-green-blue (RGB) value in addition to the identification of the abnormal portion on the surface based on the reflection brightness.
  • RGB red-green-blue
  • FIG. 11 is a flowchart illustrating the operation of the surface abnormality detection device according to the fifth example embodiment.
  • the surface abnormality detection device 51 acquires three-dimensional point cloud data including RGB information (step S 501 ).
  • the surface abnormality detection device 51 performs the clustering process based on the position information of the three-dimensional point cloud data, and classifies the three-dimensional point cloud data into the point cloud having the same structure, i.e., the cluster (step S 502 ).
  • the surface abnormality detection device 51 determines the abnormal portion where the reflection brightness value becomes the abnormal value in the point cloud (cluster) classified as having the same structure, based on the reflection brightness distribution of the point cloud (step S 503 ). When there are a plurality of point clouds, a portion where the reflection brightness value is abnormal is determined for each of the plurality of point clouds.
  • the surface abnormality detection device 51 determines a portion where the RGB value is abnormal in the point cloud classified as having the same structure, based on the RGB value of the point cloud (step S 504 ). When there are a plurality of point clouds, a portion where the RGB value is abnormal is determined for each of the plurality of point clouds. The portion where the RGB value is abnormal may be determined in a similar procedure to step S 503 after conversion to grayscale.
  • the surface abnormality detection device 51 determines an RGB normal value of the cluster based on the distribution of the RGB values at the plurality of points on the surface of the point cloud (cluster). Then, the surface abnormality detection device 51 identifies the abnormal portion on the surface of the cluster based on the difference between the RGB normal value and the RGB value at each of the plurality of points on the surface of the cluster.
  • the surface abnormality detection device 51 identifies a desired abnormal portion based on the abnormal portion determined based on the reflection brightness and the abnormal portion determined based on the RGB value (step S 505 ).
  • step S 505 the abnormal portion determined based on the reflection brightness and the abnormal portion determined based on the RGB value may be complementarily used.
  • Examples of a difference between the detection using the reflection brightness value and the detection using the RGB value include a rust fluid.
  • the rust fluid is determined as the abnormal portion based on the RGB value, but is not determined as the abnormal portion based on the reflection brightness value. Therefore, the outflow source can be identified.
  • the information can be used to identify the portion where the outflow source which is an original deterioration portion readily occurs, and an outflow path of the rust fluid, thereby enabling selection and determination of the appropriate repair method according to the degree of abnormality.
  • the surface abnormality detection device 51 can determine the abnormal portion on the surface with higher accuracy from the three-dimensional point cloud data having the reflection brightness value and the RGB value.
  • a surface abnormality detection device 61 can further improve the accuracy with which an abnormal portion on a surface is determined (identified), using the identification of an abnormal portion on the surface which is determined based on the roughness, in addition to the identification of an abnormal portion on the surface based on a reflection brightness value.
  • the spatial surface roughness can be calculated as a displacement of the point cloud from the smoothed surface.
  • the abnormality on the surface that is rougher than the accuracy of the distance measurement device 12 can be identified by identifying the abnormal portion on the surface based on the roughness.
  • the abnormal portion on the surface can be complementarily identified by using the portion where the reflection brightness is abnormal and the portion where the roughness is abnormal.
  • FIG. 12 is a flowchart illustrating the operation of the surface abnormality detection device according to the sixth example embodiment.
  • the surface abnormality detection device 61 acquires three-dimensional point cloud data (step S 601 ).
  • the surface abnormality detection device 61 performs the clustering process based on the position information of the three-dimensional point cloud data, and classifies the three-dimensional point cloud data into the point cloud having the same structure, i.e., the cluster (step S 602 ).
  • the surface abnormality detection device 61 calculates a roughness value at each point based on the surrounding point cloud (step S 603 ). That is, the surface abnormality detection device 61 calculates the roughness value at each of the plurality of points on the surface of the cluster based on the position information at the plurality of points on the surface of the cluster. For example, the smoothed surface is calculated based on the point cloud in the surroundings of an arbitrary point P, and the displacement of the point P from the smoothed surface is calculated as the roughness value of the point P.
  • the surface abnormality detection device 61 determines the abnormal portion where the reflection brightness value becomes the abnormal value in the point cloud (cluster) classified as having the same structure, based on the reflection brightness distribution of the point cloud (step S 604 ).
  • the surface abnormality detection device 61 determines the portion where the roughness value is abnormal in the point cloud classified as having the same structure, based on the roughness value of the point cloud (step S 605 ). When there are a plurality of point clouds, a portion where the roughness value is abnormal is determined for each of the plurality of point clouds.
  • the surface abnormality detection device 61 determines a roughness normal value of the cluster based on the distribution of the roughness values at the plurality of points on the surface of the cluster. Then, the surface abnormality detection device 61 identifies the abnormal portion on the surface of the cluster based on the difference between the roughness normal value and the roughness value at each of the plurality of points on the surface of the cluster.
  • the surface abnormality detection device 61 identifies a desired abnormal portion based on the abnormal portion determined based on the reflection brightness value and the abnormal portion determined based on the roughness value (step S 606 ).
  • step S 606 the abnormal portion determined based on the reflection brightness and the abnormal portion determined based on the roughness value may be complementarily used.
  • Examples of a difference between the detection using the reflection brightness value and the detection using the roughness value include lifting of coating due to internal corrosion.
  • the lifting of coating causes no change to the reflection brightness value since paint remains, but is detected as the roughness value.
  • the information can be used to identify the penetration range of corrosion from the internal corrosion portion connected to the rust exposed to the outside, thereby enabling selection and determination of the appropriate repair method.
  • the surface abnormality detection device 61 can generally determine the abnormal portion on the surface, even with respect to the target (structure) that is rougher than the accuracy of the distance measurement device 12 .
  • FIG. 13 is a diagram illustrating a distance measurement device and an object under measurement according to a seventh example embodiment.
  • FIG. 13 illustrates an object under measurement T 71 and an object under measurement T 72 , the images of which are captured with a distance measurement device 12 installed at a first observation point S 71 .
  • FIG. 13 illustrates the object under measurement T 71 and the object under measurement T 72 , the images of which are captured with a distance measurement device 12 installed at a second observation point S 72 .
  • the data (information) at a distance measurement point P 71 n on the surface of the object under measurement T 71 is acquired by the distance measurement device 12 installed at each of the first observation point S 71 and the second observation point S 72 .
  • the data (information) at a distance measurement point P 72 n on the surface of the object under measurement T 72 is acquired by the distance measurement device 12 installed at each of the first observation point S 71 and the second observation point S 72 .
  • the measurement is made at a laser incident angle A 711 from the first observation point S 71 , and the measurement is made at a laser incident angle A 712 from the second observation point S 72 .
  • the measurement is made at a laser incident angle A 721 from the first observation point S 71 , and the measurement is made at a laser incident angle A 722 from the second observation point S 72 .
  • the laser incident angle on the distance measurement point P 71 n from the first observation point S 71 is different from that from the second observation point S 72 , and therefore the reflection brightness value varies depending on the observation point.
  • the same is true for the case where an image of the measure target T 72 is captured from the first observation point S 71 and the second observation point S 72 .
  • the abnormal portion on the surface is identified by focusing on the fact that the isotropic nature of the reflected light varies depending on the surface roughness.
  • the laser reflected light tends to spread isotropically, and therefore a laser incident angle-dependent change in the reflection brightness value is small.
  • the laser reflected light has a large reflection brightness value in a direction of specular reflection, and therefore a laser incident angle-dependent change in the reflection brightness value is large.
  • the abnormal portion on the surface of each of the measure target T 71 and the object under measurement T 72 can be identified using a difference in reflection brightness acquired by the first observation point S 71 and the second observation point S 72 , and a difference in laser incident angle.
  • the difference in reflection brightness is a difference between the reflection brightness at a predetermined distance measurement point (e.g., the distance measurement point P 71 n ) acquired by the first observation point S 71 and the reflection brightness at the predetermined distance measurement point acquired by the second observation point S 72 .
  • the difference in laser incident angle is a difference between the laser incident angle on the predetermined distance measurement point from the first observation point S 71 and the laser incident angle on the predetermined distance measurement point from the second observation point S 72 .
  • the positions of the object under measurement T 71 and the object under measurement T 72 are associated with the shapes thereof by position matching and recognition, respectively, and then the difference in reflection brightness and the difference in laser incident angle are calculated, whereby the abnormal portion on the surfaces is identified.
  • the example has been described in which the reflection brightness at the same distance measurement point P 71 n (or P 72 n ) is acquired from the first observation point S 71 and the second observation point S 72 , but is not limited thereto.
  • the distance measurement point P 71 n can be acquired by the first observation point S 71 , but the distance measurement point P 71 n cannot be acquired by the second observation point S 72 , a neighboring point of the distance measurement point P 71 n or interpolation points of the reflection brightness and the laser incident angle at the distance measurement point P 71 n may be used.
  • FIG. 14 is a block diagram illustrating the surface abnormality detection device according to the seventh example embodiment.
  • the surface abnormality detection device 71 includes a first calculation means 714 a , a second calculation means 714 b , a position control means 715 , a brightness difference calculation means 716 , a correction means 717 , and an identification means 713 .
  • the first calculation means 714 a calculates a first incident angle of the laser for each of the plurality of distance measurement points based on the position information of the first observation point S 71 and the position information of the plurality of distance measurement points on the surface of the object under measurement.
  • the position information of the first observation point S 71 is included in first point cloud data.
  • the first calculation means 714 a calculates the first incident angle at the distance measurement point based on a direction connecting a distance measurement point on the surface of the object under measurement and the first observation point S 71 , and a perpendicular line at the distance measurement point.
  • the second calculation means 714 b calculates a second incident angle of the laser for each of the plurality of distance measurement points based on the position information of the second observation point S 72 and the position information of the plurality of distance measurement points included in the second point cloud data.
  • the second calculation means 714 b calculates the second incident angle at a distance measurement point based on a direction connecting the distance measurement point on the surface of the object under measurement and the second observation point S 72 , and a perpendicular line at the distance measurement point.
  • the position control means 715 adjusts to match the positions for each of the plurality of distance measurement points based on the position information of the plurality of distance measurement points in the first point cloud data and the position information of the plurality of distance measurement points in the second point cloud data.
  • the brightness difference calculation means 716 calculates, for each of the plurality of distance measurement points, a reflection brightness difference value which is a difference between a first reflection brightness value at each of the plurality of distance measurement points in the first point cloud data after the position adjustment and a second reflection brightness value at each of the plurality of distance measurement points in the second point cloud data after the position adjustment.
  • the correction means 717 calculates, for each of the plurality of distance measurement points, an incident angle difference which is a difference between the first incident angle at each of the plurality of distance measurement points in the first point cloud data after the position adjustment and the second incident angle at each of the plurality of distance measurement points in the second point cloud data after the position adjustment.
  • the correction means 717 corrects, for each of the plurality of distance measurement points, the reflection brightness difference value based on the calculated incident angle difference.
  • the identification means 713 identifies an abnormal portion of the object under measurement based on the reflection brightness difference value after correction.
  • FIG. 15 is a flowchart illustrating the operation of the surface abnormality detection device according to the seventh example embodiment.
  • the three-dimensional point cloud data of the image captured by the first observation point S 71 is referred to as the first point cloud data
  • the three-dimensional point cloud data of the image captured by the second observation point S 72 is referred to as the second point cloud data.
  • the surface abnormality detection device 71 acquires the first point cloud data (step S 701 ).
  • the surface abnormality detection device 71 acquires the second point cloud data (step S 702 ).
  • the surface abnormality detection device 71 calculates, with respect to the first point cloud data, the first incident angle of the laser at each distance measurement point based on the point cloud and the position information of the first observation point S 71 (step S 703 ).
  • the surface abnormality detection device 11 calculates, with respect to the second point cloud data, the second incident angle of the laser at each distance measurement point based on the point cloud and the position information of the second observation point S 72 (step S 704 ).
  • the laser incident angle is calculated as described in the third example embodiment, for example.
  • the surface abnormality detection device 71 performs the position matching between the first point cloud data and the second point cloud data, or the position matching between the first point cloud data and the second point cloud data by shape identification (step S 705 ).
  • the surface abnormality detection device 71 calculates a difference in reflection brightness value between the distance measurement points corresponding to each other (step S 706 ).
  • the difference in reflection brightness may be calculated using the points closest to each other in the corresponding point cloud, or an interpolated value at the corresponding position.
  • the surface abnormality detection device 71 also calculates a difference in laser incident angle between the distance measurement points.
  • the difference in reflection brightness is referred to as a reflection brightness difference or a reflection brightness difference value.
  • the surface abnormality detection device 71 corrects the reflection brightness difference value based on the laser incident angle difference calculated in step S 706 (step S 707 ).
  • the laser incident angle difference may be calculated using the points closest to each other in the corresponding point cloud, or an interpolated value at the corresponding position.
  • the reflection brightness difference value may be corrected by applying the known reflectance property, other than modeling of the bidirectional reflectance distribution function of the object under measurement, or simple modeling assuming Lambertian reflection.
  • step S 708 determines the point cloud as an abnormal portion on the surface (step S 709 ).
  • the surface abnormality detection device 71 determines the point cloud as a normal portion on the surface (step S 710 ).
  • the surface abnormality detection device 71 can identify the abnormal portion on the surface with higher accuracy from the three-dimensional point cloud data of the images captured from a plurality of points.
  • a surface abnormality detection device 81 according to an eighth example embodiment is different from the surface abnormality detection device 71 according to the seventh example embodiment in that an abnormal portion on the surface is identified by comparison to the three-dimensional point cloud data measured in the past, whereby the accuracy is improved.
  • the three-dimensional point cloud data for comparison measured in the past is referred to as “three-dimensional point cloud data (comparison)” or cloud data for comparison
  • the three-dimensional point cloud data for evaluation for determining the abnormality is referred to as “three-dimensional point cloud data (evaluation)” or cloud data for evaluation.
  • the simplest method of comparing the three-dimensional point cloud data (comparison) with the three-dimensional point cloud data (evaluation) is a method of acquiring a reflection brightness difference value of the point cloud using the point of closest proximity between the point clouds or the interpolation.
  • FIG. 16 is a block diagram illustrating the surface abnormality detection device according to the eighth example embodiment.
  • the surface abnormality detection device 81 includes a position control means 815 , a brightness difference calculation means 816 , and an identification means 813 .
  • the position information of a plurality of distance measurement points on the surface of the object under measurement is included. Also in the cloud data for comparison, the position information of a plurality of distance measurement points on the surface of the object under measurement is included.
  • the position control means 815 adjusts to match the positions for each of the plurality of distance measurement points based on the position information of the plurality of distance measurement points included in the cloud data for evaluation and the position information of the plurality of distance measurement points included in the cloud data for comparison.
  • the brightness difference calculation means 816 calculates, for each of the plurality of distance measurement points, a reflection brightness difference value which is a difference between a reflection brightness value for evaluation at each of the plurality of distance measurement points in the cloud data for evaluation after the position adjustment and a reflection brightness value for comparison at each of the plurality of distance measurement points in the cloud data for comparison after the position adjustment.
  • the identification means 813 identifies an abnormal portion of the object under measurement based on the reflection brightness difference value.
  • a normal value of the reflection brightness value is determined on a per cluster basis from the three-dimensional point cloud data (comparison), the clusters are associated with each other between the point clouds, and a deviation value of the reflection brightness value is determined as a difference between a reflection brightness value of the three-dimensional point cloud data (evaluation) and the reflection brightness normal value of the corresponding cluster, to thereby identify the abnormal portion.
  • the reflection brightness value may change. In such a case, the error can be reduced by processing the reflection brightness value on a per cluster basis.
  • FIG. 17 is a flowchart illustrating the operation of the surface abnormality detection device according to the eighth example embodiment.
  • the surface abnormality detection device 81 acquires the three-dimensional point cloud data (evaluation) (step S 801 ).
  • the surface abnormality detection device 81 acquires the three-dimensional point cloud data (comparison) (step S 802 ).
  • the surface abnormality detection device 81 performs the clustering process based on the position information of the point cloud with respect to the three-dimensional point cloud data (evaluation), and classifies the three-dimensional point cloud data (evaluation) into the point cloud having the same structure, i.e., the cluster (step S 803 ). That is, the surface abnormality detection device 81 classifies the object under measurement into one or more clusters having the same structure, based on the position information at the plurality of distance measurement points on the surface of the object under measurement included in the three-dimensional point cloud data (evaluation).
  • the surface abnormality detection device 81 performs the clustering process based on the position information of the point cloud with respect to the three-dimensional point cloud data (comparison), and classifies the three-dimensional point cloud data (comparison) into the point cloud having the same structure, i.e., the cluster (step S 804 ). That is, the surface abnormality detection device 81 classifies the object under measurement into one or more clusters having the same structure, based on the position information at the plurality of distance measurement points on the surface of the object under measurement included in the three-dimensional point cloud data (comparison).
  • the surface abnormality detection device 81 calculates the normal value of the reflection brightness (reflection brightness normal value) for each classified cluster in step S 804 (step S 805 ). That is, the surface abnormality detection device 81 determines the reflection brightness normal value for each cluster of the three-dimensional point cloud data (comparison) based on the distribution of the reflection brightness values in the plurality of distance measurement points of the cluster of the three-dimensional point cloud data (comparison).
  • the surface abnormality detection device 81 associates the clusters corresponding to each other between the three-dimensional point cloud data (evaluation) and the three-dimensional potin cloud data (comparison) with the positions, respectively, by position matching between the two pieces of three-dimensional point cloud data or shape identification of the clusters (step S 806 ). That is, the surface abnormality detection device 81 associates the cluster of the three-dimensional point cloud data (evaluation) with the cluster of the three-dimensional point cloud data (comparison) recognized as having the same structure, based on the position information of the plurality of distance measurement points of the cluster of the three-dimensional point cloud data (evaluation) and the position information of the plurality of distance measurement points of the cluster of the three-dimensional point cloud data (comparison).
  • the surface abnormality detection device 81 calculates a difference from the reflection brightness normal value based on the reflection brightness normal value calculated in step S 805 (step S 807 ).
  • the difference from the reflection brightness normal value is referred to as a reflection brightness normal difference value. That is, the surface abnormality detection device 81 calculates the reflection brightness normal difference value which is a difference between the reflection brightness value at each of the plurality of distance measurement points of the cluster of the three-dimensional point cloud data (evaluation) and the reflection brightness normal value of the cluster of the three-dimensional point cloud data (comparison) corresponding to the cluster of the three-dimensional point cloud data (evaluation).
  • the surface abnormality detection device 81 determines the point cloud as an abnormal portion on the surface (step S 809 ). That is, the surface abnormality detection device 81 identifies, for each cluster, the abnormal portion on the surface of the object under measurement based on the reflection brightness normal difference value.
  • the surface abnormality detection device 81 determines the point cloud as an abnormal portion on the surface (step S 810 ).
  • the surface abnormality detection device 81 can identify the abnormal portion on the surface with higher accuracy from the comparison with the three-dimensional point cloud data of the image captured in the past.
  • a processing device capable of reducing false detection when an abnormal matter is detected, a system, a method, and a non-transitory computer-readable medium.
  • the present invention has been described as a hardware configuration in the above-described example embodiments, the present invention is not limited the hardware configuration.
  • the processes in each of the components can be also implemented by causing a CPU (Central Processing Unit) to execute a computer program.
  • a CPU Central Processing Unit
  • the program can be stored in various types of non-transitory computer-readable media and thereby supplied to computers.
  • the non-transitory computer-readable media include various types of tangible storage media. Examples of the non-transitory computer-readable media include a magnetic recording medium (such as a flexible disk, a magnetic tape, and a hard disk drive), a magneto-optic recording medium (such as a magneto-optic disk), a CD-ROM (Read Only Memory), a CD-R, and a CD-R/W, and a semiconductor memory (such as a mask ROM, a PROM (Programmable ROM), an EPROM (Erasable PROM), a flash ROM, and a RAM (Random Access Memory)).
  • a magnetic recording medium such as a flexible disk, a magnetic tape, and a hard disk drive
  • a magneto-optic recording medium such as a magneto-optic disk
  • CD-ROM Read Only Memory
  • CD-R Compact Only Memory
  • CD-R/W and
  • the program can be supplied to computers by using various types of transitory computer-readable media.
  • Examples of the transitory computer-readable media include an electrical signal, an optical signal, and an electromagnetic wave.
  • the transitory computer-readable media can be used to supply programs to the computer through a wire communication path such as an electrical wire and an optical fiber, or wireless communication path.
  • a surface abnormality detection device comprising:
  • a classification means for classifying an object under measurement into one or more clusters having the same structure, based on position information at a plurality of points on a surface of the object under measurement;
  • a determination means for determining a reflection brightness normal value of the cluster based on a distribution of reflection brightness values at a plurality of points on a surface of the cluster
  • an identification means for identifying an abnormal portion on the surface of the cluster based on a difference between the reflection brightness normal value and the reflection brightness value at each of the plurality of points on the surface of the cluster.
  • the identification means determines, among the plurality of points on the surface of the cluster, a predetermined point where the difference between the reflection brightness value and reflection brightness normal value exceeds a threshold value as the abnormal portion.
  • the reflection brightness value is corrected based on an attenuation amount due to a distance between an own device which is an observation point and the point on the surface of the cluster.
  • a laser incident angle at a distance measurement point of the cluster is calculated based on a direction connecting the distance measurement point of the cluster and the own device, and a perpendicular line at the distance measurement point of the cluster, and
  • the reflection brightness value at the distance measurement point of the cluster is further corrected based on the laser incident angle.
  • the classification means further classifies the cluster into subclusters based on the laser incident angle
  • the determination means determines a reflection brightness normal value of the subcluster based on a distribution of reflection brightness values at a plurality of points on a surface of the subcluster, and the identification means identifies an abnormal portion on the surface of the subcluster based on a difference between the reflection brightness normal value of the subcluster and the reflection brightness value at each of the plurality of points on the surface of the subcluster.
  • the determination means determines an RGB normal value of the cluster based on a distribution of RGB values at the plurality of points on the surface of the cluster,
  • the identification means identifies an abnormal portion on the surface of the cluster based on a difference between the RGB normal value and the RGB value at each of the plurality of points on the surface of the cluster, and
  • the identification means identifies a desired abnormal portion based on the abnormal portion identified using the reflection brightness value and the abnormal portion identified using the RGB value.
  • a roughness value at each of the plurality of points on the surface of the cluster is calculated based on the position information at the plurality of points on the surface of the cluster
  • the determination means determines a roughness normal value of the cluster based on a distribution of the roughness values at the plurality of points on the surface of the cluster,
  • the identification means identifies an abnormal portion on the surface of the cluster based on a difference between the roughness normal value and the roughness value at each of the plurality of points on the surface of the cluster, and
  • the identification means identifies a desired abnormal portion based on the abnormal portion identified using the reflection brightness value and the abnormal portion identified using the roughness value.
  • a surface abnormality detection device comprising:
  • a first calculation means for calculating a first incident angle of a laser for each of a plurality of distance measurement points based on position information of a first observation point, and position information, included in first point cloud data, of the plurality of distance measurement points of a surface of an object under measurement;
  • a second calculation means for calculating a second incident angle of a laser for each of the plurality of distance measurement points based on position information of a second observation point, and position information of the plurality of distance measurement points included in second point cloud data;
  • a position control means for making an adjustment to match positions for each of the plurality of distance measurement points based on the position information of the plurality of distance measurement points in the first point cloud data and the position information of the plurality of distance measurement points in the second point cloud data;
  • a brightness difference calculation means for calculating, for each of the plurality of distance measurement points, a reflection brightness difference value which is a difference between a first reflection brightness value at each of the plurality of distance measurement points in the first point cloud data after the position adjustment and a second reflection brightness value at each of the plurality of distance measurement points in the second point cloud data after the position adjustment;
  • a correction means for calculating, for each of the plurality of distance measurement points, an incident angle difference which is a difference between the first incident angle at each of the plurality of distance measurement points in the first point cloud data after the position adjustment and the second incident angle at each of the plurality of distance measurement points in the second point cloud data after the position adjustment, and correcting, for each of the plurality of distance measurement points, the reflection brightness difference value based on the incident angle difference;
  • an identification means for identifying an abnormal portion of the object under measurement based on the reflection brightness difference value after the correction.
  • a surface abnormality detection device comprising:
  • a position control means for making an adjustment to match positions for each of a plurality of distance measurement points based on position information of the plurality of distance measurement points on a surface of an object under measurement, the position information being included in cloud data for evaluation and position information of the plurality of distance measurement points included in cloud data for comparison;
  • a brightness difference calculation means for calculating, for each of the plurality of distance measurement points, a reflection brightness difference value which is a difference between a reflection brightness value for evaluation at each of the plurality of distance measurement points in the cloud data for evaluation after the position adjustment and a reflection brightness value for comparison at each of the plurality of distance measurement points in the cloud data for comparison after the position adjustment;
  • an identification means for identifying an abnormal portion of the object under measurement based on the reflection brightness difference value.
  • a surface abnormality detection device comprising:
  • a classification means for evaluation for classifying an object under measurement into one or more clusters having the same structure, based on position information at a plurality of distance measurement points on a surface of the object under measurement included in cloud data for evaluation;
  • a classification means for comparison for classifying the object under measurement into one or more clusters having the same structure, based on position information at the plurality of distance measurement points included in cloud data for comparison;
  • a determination means for comparison for determining a reflection brightness normal value for each cluster of the cloud data for comparison based on a distribution of reflection brightness values at the plurality of distance measurement points of the cluster of the cloud data for comparison;
  • control means for associating the cluster of the cloud data for evaluation with the cluster of the cloud data for comparison recognized as having the same structure, based on the position information of the plurality of distance measurement points of the cluster of the cloud data for evaluation and the position information of the plurality of distance measurement points of the cluster of the cloud data for comparison;
  • a calculation means for calculating a reflection brightness normal difference value which is a difference between the reflection brightness value at each of the plurality of distance measurement points of the cluster of the cloud data for evaluation and the reflection brightness normal value of the cluster of the cloud data for comparison corresponding to the cluster of the cloud data for evaluation;
  • an identification means for identifying, for each cluster, an abnormal portion on the surface of the object under measurement based on the reflection brightness normal difference value.
  • a system comprising:
  • a measurement device configured to acquire a reflection brightness value at each of a plurality of points on a surface of an object under measurement
  • a method of a surface abnormality detection device comprising:
  • a non-transitory computer-readable medium storing a program configured to cause a computer to execute:

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Electromagnetism (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Health & Medical Sciences (AREA)
  • Chemical & Material Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Quality & Reliability (AREA)
  • Databases & Information Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Length Measuring Devices By Optical Means (AREA)
US17/599,747 2019-04-03 2019-04-03 Surface abnormality detection device and system Pending US20220170739A1 (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2019/014790 WO2020202496A1 (fr) 2019-04-03 2019-04-03 Dispositif et système de détection d'anomalie de surface

Publications (1)

Publication Number Publication Date
US20220170739A1 true US20220170739A1 (en) 2022-06-02

Family

ID=72666260

Family Applications (2)

Application Number Title Priority Date Filing Date
US17/599,747 Pending US20220170739A1 (en) 2019-04-03 2019-04-03 Surface abnormality detection device and system
US17/599,757 Pending US20220156914A1 (en) 2019-04-03 2020-03-17 Surface abnormality detection device and system

Family Applications After (1)

Application Number Title Priority Date Filing Date
US17/599,757 Pending US20220156914A1 (en) 2019-04-03 2020-03-17 Surface abnormality detection device and system

Country Status (3)

Country Link
US (2) US20220170739A1 (fr)
JP (2) JP7215568B2 (fr)
WO (2) WO2020202496A1 (fr)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019198562A1 (fr) * 2018-04-11 2019-10-17 富士フイルム株式会社 Dispositif de gestion de structure, procédé de gestion de structure et programme de gestion de structure
WO2021038685A1 (fr) * 2019-08-26 2021-03-04 日本電気株式会社 Dispositif de détection d'anomalie de surface, système, procédé, et support non transitoire lisible par ordinateur
US11703457B2 (en) * 2020-12-29 2023-07-18 Industrial Technology Research Institute Structure diagnosis system and structure diagnosis method
WO2022244267A1 (fr) * 2021-05-21 2022-11-24 日本電気株式会社 Dispositif de détection d'anomalie, procédé de commande et support lisible par ordinateur
US12002192B2 (en) * 2021-11-16 2024-06-04 Solera Holdings, Llc Transfer of damage markers from images to 3D vehicle models for damage assessment
WO2023218899A1 (fr) * 2022-05-13 2023-11-16 パナソニックＩｐマネジメント株式会社 Système de mesure et procédé de mesure
CN116977331B (zh) * 2023-09-22 2023-12-12 武汉展巨华科技发展有限公司 基于机器视觉的3d模型表面检测方法

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5774212A (en) * 1997-03-19 1998-06-30 General Electric Co. Method and apparatus for detecting and analyzing directionally reflective surface flaws
JP3080058B2 (ja) * 1997-12-25 2000-08-21 日本電気株式会社 レーダ装置及びレーダ測定方法
JP4872948B2 (ja) * 2008-02-27 2012-02-08 パルステック工業株式会社 ３次元形状測定装置および３次元形状測定方法
JP5654402B2 (ja) 2011-03-29 2015-01-14 株式会社日立製作所 設備異常経時変化判定装置、設備異常変化判定方法、およびプログラム
JP6590653B2 (ja) * 2014-11-19 2019-10-16 首都高技術株式会社 点群データ利用システム
CN109242828B (zh) * 2018-08-13 2022-04-19 浙江大学 基于光栅投影多步相移法的3d打印制品三维缺陷检测方法

Also Published As

Publication number Publication date
JP7215568B2 (ja) 2023-01-31
US20220156914A1 (en) 2022-05-19
JP7131692B2 (ja) 2022-09-06
WO2020202496A1 (fr) 2020-10-08
JPWO2020202496A1 (fr) 2020-10-08
JPWO2020203263A1 (fr) 2020-10-08
WO2020203263A1 (fr) 2020-10-08

Similar Documents

Publication Publication Date Title
US20220170739A1 (en) Surface abnormality detection device and system
US8121400B2 (en) Method of comparing similarity of 3D visual objects
US10332275B2 (en) Information processing apparatus, information processing method, and storage medium
CN107305632B (zh) 基于单目计算机视觉技术的目标对象距离测量方法与系统
US11681036B2 (en) Image processing apparatus, image processing method, and non-transitory computer readable medium storing image processing program
US20130293408A1 (en) Radar image processing
US20150198735A1 (en) Method of Processing 3D Sensor Data to Provide Terrain Segmentation
JP7386136B2 (ja) 雲高計測装置、計測点決定方法、および雲種類決定方法
WO2021258282A1 (fr) Dispositif et procédé de détection de cible, appareil d'imagerie, et plateforme mobile
CN112639516B (zh) 激光雷达测距方法、装置、计算机设备和存储介质
CN107274446B (zh) 一种利用法向一致性的尖锐几何边缘点识别方法
US11933884B2 (en) Radar image processing device, radar image processing method, and storage medium
CN113099120B (zh) 深度信息获取方法、装置、可读存储介质及深度相机
CN114690157A (zh) 激光雷达反射率的自动标定方法、目标检测方法以及装置
CN116819561A (zh) 一种点云数据匹配方法、系统、电子设备及存储介质
CN113805157B (zh) 基于标靶的高度测量方法、装置及设备
Ono et al. Robust detection of surface anomaly using LiDAR point cloud with intensity
CN115128637A (zh) 用于激光雷达传感器的作用距离确定的方法
US20220276181A1 (en) Surface anomaly detecting device, system, method, and non-transitory computer-readable medium
US20200410694A1 (en) Object detection device, object detection system, object detection method, and non-transitory computer-readable medium storing program
CN116228603B (zh) 一种挂车周围障碍物的报警系统及装置
KR101173565B1 (ko) 영상처리를 이용한 컨테이너 측정방법
JP7375944B2 (ja) 処理装置、異常検知システム、異常検知方法、及びプログラム
JP2023038693A5 (fr)
JP6687296B2 (ja) 監視装置

Legal Events

Date Code Title Description
AS Assignment

Owner name: NEC CORPORATION, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ONO, YOSHIMASA;TSUJI, AKIRA;NOGUCHI, HIDEMI;AND OTHERS;SIGNING DATES FROM 20211012 TO 20211108;REEL/FRAME:058857/0086

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION