WO2023238465A1 - 部品検査方法および装置 - Google Patents

部品検査方法および装置 Download PDF

Info

Publication number
WO2023238465A1
WO2023238465A1 PCT/JP2023/009217 JP2023009217W WO2023238465A1 WO 2023238465 A1 WO2023238465 A1 WO 2023238465A1 JP 2023009217 W JP2023009217 W JP 2023009217W WO 2023238465 A1 WO2023238465 A1 WO 2023238465A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
target
point cloud
parts
measurement
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2023/009217
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
典克 鷲見
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nissan Motor Co Ltd
Original Assignee
Nissan Motor Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nissan Motor Co Ltd filed Critical Nissan Motor Co Ltd
Priority to CN202380045118.3A priority Critical patent/CN119317810A/zh
Priority to EP23819452.6A priority patent/EP4538635A4/en
Priority to JP2024526242A priority patent/JP7750410B2/ja
Publication of WO2023238465A1 publication Critical patent/WO2023238465A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B5/00—Measuring arrangements characterised by the use of mechanical techniques
    • G01B5/003—Measuring of motor parts
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
    • G01B11/2518—Projection by scanning of the object
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/521—Depth or shape recovery from laser ranging, e.g. using interferometry; from the projection of structured light
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30164—Workpiece; Machine component

Definitions

  • the present invention relates to a component inspection method and apparatus for performing an inspection related to the spatial arrangement of one or more target components in an assembly formed by assembling multiple components.
  • the target A so-called missing area may occur in a part of the part where data could not be obtained. This defective area becomes a factor that reduces the accuracy of specifying the shape and position of the component.
  • Patent Document 1 discloses estimating the three-dimensional position and orientation of a target object and identifying the object by using missing regions as features, but does not disclose anything about reducing the number of missing regions during measurement. do not have.
  • the present invention provides a component inspection method for performing an inspection related to the spatial arrangement of one or more target components in an assembly formed by assembling a plurality of components.
  • a three-dimensional sensor scans the area containing the target part from the outside to obtain measurement data including a part of the outer surface of the target part, Obtain design data including the individual external shapes of the target parts and the positional relationships of the parts within the assembly, Determine the missing area in the measurement data of the target part by comparing it with the design data of the target part, Determine the measurement parameters of the three-dimensional sensor suitable for acquiring measurement data of this defective area, Additional scanning is performed in accordance with this measurement parameter.
  • the defective area can be made smaller. Measurement accuracy, which is a prerequisite for inspecting target parts, can be improved.
  • FIG. 2 is an explanatory diagram showing various parts in an engine room that is an assembly in one embodiment, viewed from above.
  • FIG. 1 is a functional block diagram of a gap inspection device according to an embodiment. Point cloud data of the engine room and its internal parts scanned by a 3D laser scanner. Point cloud data of the engine room and its internal parts at 40% progress. An explanatory diagram regarding reliability. 1 is a flowchart showing a process flow of a gap inspection method according to an embodiment. An explanatory diagram of a movement trajectory of a three-dimensional sensor during scanning.
  • FIG. 7 is an explanatory diagram showing a discrepancy between an instruction trajectory during scanning and an actual movement trajectory. An explanatory diagram of additional scanning for a missing area.
  • the present invention is applied to a gap inspection in which the spatial positions of two parts in an engine room are specified and the minimum gap between the two parts is calculated in a completed automobile inspection process.
  • FIG. 1 is a top view of some of the various parts in the engine room 1 of an automobile, which is an assembly of one embodiment, and shows the area near the intake manifold 2 of the engine.
  • An ignition coil 3 that is part of the engine's ignition system is arranged below the intake manifold 2.
  • a fuel tube 4 which is a pipe for supplying fuel to the engine, is located on the side of the intake manifold 2.
  • the fuel tube 4 is formed from a metal tube in consideration of oil resistance and fire resistance, and has a circular cross section along the radial direction. As shown in FIG.
  • the fuel tube 4 includes a straight portion 4a that extends generally in a straight line along the side of the intake manifold 2, and an inclined portion 4b that slopes toward the ignition coil 3 from one end of the straight portion 4a.
  • the fuel tube 4 made of a metal tube undergoes some deformation during installation and is therefore considered a non-rigid component.
  • the ignition coil 3 is a component that is considered to be a rigid body whose outer shape does not change.
  • the minimum spatial distance (minimum gap) between the ignition coil 3 and the fuel tube 4 is generally prescribed by law, and therefore, it is necessary to inspect the minimum gap in the completed vehicle inspection process.
  • the design minimum distance between the ignition coil 3 and the fuel tube 4 is defined at a position on the outer surface of at least one of the ignition coil 3 or the fuel tube 4 that is not visible from the outside, It is generally difficult for inspectors to measure using a ruler.
  • such a gap between two parts is determined by external measurement using a three-dimensional sensor and arithmetic processing.
  • the gap inspection device of one embodiment used in the inspection of finished automobiles includes a three-dimensional sensor, for example, a three-dimensional laser scanner 5, a database 6, a control device 7, and one or more displays. 8. In the finished vehicle inspection process, a large number of inspections are sequentially performed in a predetermined order.
  • the gap inspection device of one embodiment is configured as a part of an inspection device in a completed vehicle inspection process.
  • the 3D laser scanner 5 can obtain the 3D coordinates of the surface shape of the measurement target by irradiating the measurement target with a laser beam and measuring the reflection time, and can scan at a high speed of approximately tens of thousands of points per second. By performing measurements with , high-density point cloud data can be obtained.
  • Various sizes and types of three-dimensional laser scanners are known, but in one embodiment, a type that can be held in the hand by an operator to scan a measurement target is used. The target area is measured by manually scanning a specific area of the engine room 1 including the ignition coil 3 and fuel tube 4 along with other parts from the outside along a designated movement trajectory. By scanning with the three-dimensional laser scanner 5, as shown in FIG. 3, point cloud data of the entire area including a part of the outer surface of the ignition coil 3 and the fuel tube 4 is obtained.
  • point cloud data is actually acquired as time series data for each frame by scanning, but point cloud data for the entire area is acquired by overlapping them. Furthermore, it is desirable to perform a noise removal process on the acquired point cloud data to remove noise caused by, for example, dust particles that have entered during scanning. Furthermore, the three-dimensional laser scanner may require a marker that serves as a guide for recognizing the shape and position of a target part during scanning.
  • scanning in the present invention refers to the area scanning function of the three-dimensional sensor (for example, the three-dimensional laser scanner 5) itself, the movement of the three-dimensional sensor along a certain trajectory by a worker, for example,
  • a three-dimensional laser scanner 5 having a surface scanning function is used, and scanning is performed by moving the three-dimensional laser scanner 5 along a predetermined trajectory.
  • multiple 3D sensors may be fixedly placed at appropriate positions in space and the scanning function of each 3D sensor may be used to measure the target area. It is also possible to obtain point cloud data of the area.
  • the database 6 stores all the design data of the vehicle to be inspected. Therefore, the shape data of the ignition coil 3 and the shape data of the fuel tube 4, which are the targets of the gap inspection, are It is stored as design data along with data indicating the positional relationship within the design. Furthermore, the database 6 also stores shape data and positional relationship data of other parts existing within the area.
  • the design data is stored in the database 6 in the form of CAD data constituting a mesh, and necessary design data is read from the database 6 to the control device 7 during a gap inspection. In addition, after reading the design data to the control device 7, well-known hidden surface processing is applied to parts other than those related to the gap between the ignition coil 3 and the fuel tube 4 to reduce the data size. You can also do this.
  • the control device 7 includes a first alignment section 7a, a progress calculation section 7b, a second alignment section 7c, a third alignment section 7d, a reliability calculation section 7e, and a first interpolation section 7f. It is configured to include a second interpolation section 7g, a distance calculation section 7h, an information output section 7i, a missing area determination section 7j, a measurement parameter calculation section 7k, and a movement trajectory determination section 7m.
  • a measurement data acquisition unit that acquires measurement data including a part of the outer surface of the target part by scanning using the three-dimensional laser scanner 5, and a measurement data acquisition unit that acquires measurement data including a part of the outer surface of the target part.
  • the control device 7 includes a design data acquisition unit that acquires design data including the external shape and the positional relationship of the parts in the engine room 1 from the database 6.
  • the first alignment unit 7a is configured to measure the entire target area (the so-called target area in alignment) including the ignition coil 3 and the fuel tube 4 in the measurement data acquired by the three-dimensional laser scanner 5, and the data stored in the database 6. Rough alignment is performed with the entire area including the ignition coil 3 and fuel tube 4 in the design data (a so-called reference area for alignment). For example, the first alignment unit 7a uses the FPFH algorithm to search for a key point from the point cloud data of the entire target area, and determines the characteristics of this key point, such as the normal vector of the key point and the relative angle of the surroundings. Describe.
  • the FPFH algorithm is similarly used to search for key points, and the characteristics of these key points, such as key Describe the normal vector of a point and the relative angles around it. Then, the first alignment unit 7a compares the normal vectors between the target-related key points and the reference-related key points and the relative angles with the surrounding point group, so that the normal vectors and relative angles almost match. Search for combinations of key points that make up pairs. Then, the reference point cloud data is roughly aligned with the target point cloud data using a combination of paired key points.
  • the non-rigid fuel tube 4 is regarded as a rigid body, and as a result, average positioning is performed within the length range of the fuel tube 4.
  • Rough positioning of the target area by the first positioning unit 7a is performed in parallel with the progress of scanning of the target area.
  • the first positioning unit 7a may perform rough positioning of the target area using a known algorithm other than the FPFH algorithm.
  • the progress calculation unit 7b calculates necessary scanning within the target area before performing detailed positioning of the ignition coil 3 and fuel tube 4, which will be described later. Calculate the degree of progress that indicates how much progress has been made. For example, the progress calculation unit 7b calculates the number of points in the point cloud data of the target area based on the design data serving as a reference, and the point cloud that matches each other between the reference and the target by rough alignment by the first alignment unit 7a. The scanning progress is calculated from the ratio of the data points. The progress calculation unit 7b further compares the progress calculated in this way with a predetermined progress threshold (for example, 40% in this embodiment), and calculates the progress that gradually increases as the scanning progresses.
  • a predetermined progress threshold for example, 40% in this embodiment
  • this progress level threshold It is determined in real time whether this progress level threshold has been exceeded. For example, the progress level at that time and information as to whether or not this progress level exceeds the progress level threshold are displayed on the display 8 via the information output unit 7i. 5 will continue scanning. In other words, scanning (generation of point cloud data) by the three-dimensional laser scanner 5, rough alignment, and progress calculation are repeated in real time until a predetermined progress threshold is exceeded. For example, the area surrounded by a broken line in FIG. 4 corresponds to a progress level of 40%.
  • the degree of progress is evaluated not only by the ratio of the number of data points as described above, but also by the number of viewpoints passed by the three-dimensional laser scanner 5, the number of key points used in rough alignment, etc. Good too.
  • the second positioning unit 7c performs detailed positioning of each component in space on the condition that the degree of progress exceeds a threshold value. That is, the second positioning unit 7c searches for a pair of points in the reference point cloud data for all points in the target point cloud data obtained by measuring the ignition coil 3 and the fuel tube 4, and compares the points with the reference point cloud data. precisely align the point cloud data to the target point cloud data.
  • the fuel tube 4, which is a non-rigid body, is considered to be a rigid body here, and as a result, average positioning is achieved within the length range of the fuel tube 4.
  • Detailed alignment in the second alignment section 7c can be performed using a known appropriate algorithm.
  • the third positioning section 7d performs so-called non-rigid positioning of a non-rigid component (in this embodiment, the fuel tube 4) in consideration of deformation after the detailed positioning performed by the second positioning section 7c.
  • a non-rigid component in this embodiment, the fuel tube 4
  • an appropriate well-known algorithm can be used, but for example, the nearest pair is searched from the reference point cloud data and the target point cloud data, which are assumed to be rigid bodies and aligned in advance, and the two points of the pair are searched for.
  • the rotation, expansion, and parallel translation parameters are determined as parameters that bring the values closer to each other.
  • the outer shape of the fuel tube 4 is down-sampled so that the outer surface is made up of a plurality of triangles having vertices and edges, and each down-sampled area, that is, a plurality of adjacent triangles, is Using the vertices that represent the point cloud (cluster) of each region that contains the cluster, we rotate, expand, and transform the cluster with the constraint that the edge length does not change (strictly speaking, the length change is minimum). Decompose into parallel translation parameters. The entire deformation of the fuel tube 4 is obtained as a set of deformations in cluster units.
  • the third alignment unit 7d deforms the reference point group data about the fuel tube 4 and aligns it with the target point group data in space using a non-rigid alignment method that takes such deformation into consideration.
  • the reliability calculation unit 7e calculates the reliability of the alignment of each target component (that is, the ignition coil 3 and the fuel tube 4). In other words, the reliability calculation unit 7e calculates the reliability indicating how close the target point cloud data is to the aligned reference point cloud data for each of the ignition coil 3 and the fuel tube 4. do.
  • FIG. 5 is an explanatory diagram schematically showing point cloud data of the fuel tube 4 after alignment, in order to explain reliability.
  • the circular outer surface in a certain cross section of the fuel tube 4 is formed by 13 point group data.
  • the 13 point cloud data Dr arranged in a circle is reference point cloud data based on design data
  • the 7 point cloud data Dt arranged in a semicircle is target point cloud data based on measurement data. be.
  • the lower half becomes a so-called missing area
  • the point group data Dt of the target are arranged in a semicircular shape.
  • the reliability is expressed, for example, as a ratio between the points of the reference point group data Dr and the points of the target point group data Dt included within a radius L from each point of this reference point group data Dr. .
  • the range of radius L from a large number of points in the reference point group data Dr is represented by an outer circle C1 and an inner circle C2, respectively indicated by broken lines.
  • the reliability is 4/13.
  • the reliability is 2/13.
  • all points of the target that is, seven point group data Dt, are included within the range R, and the reliability is 7/13. In this way, the reliability is influenced by both the alignment accuracy and the size or proportion of the missing area in the measurement data.
  • the first interpolation unit 7f interpolates the missing portions of the measurement data of the ignition coil 3, that is, the point group data of the target, which are not scanned, using the reference point group data.
  • the back (lower) outer surface portion of the ignition coil 3 that is hidden from view from the upper side of the engine room 1 is interpolated using the reference point cloud data, and the point cloud of the ignition coil 3 including the hidden portion is generated.
  • the first interpolation unit 7f converts the generated point group data into mesh data constituting a surface using a well-known conversion method.
  • the second interpolation unit 7g interpolates the missing portions of the measurement data of the fuel tube 4, that is, the point group data of the target, which are not scanned, using the reference point group data.
  • the outer surface portion of the back side (lower side) of the fuel tube 4 that is hidden from view from the upper side of the engine room 1 is interpolated using the reference point cloud data, and the point cloud of the fuel tube 4 including the hidden portion is interpolated using the reference point cloud data.
  • the second interpolation unit 7g converts the generated point group data into mesh data constituting a surface using a well-known conversion method.
  • the distance calculation unit 7h calculates each distance on the surface of the ignition coil 3 based on the mesh data of the ignition coil 3 acquired by the first interpolation unit 7f and the mesh data of the fuel tube 4 acquired by the second interpolation unit 7g. The distance from each point to each point on the surface of the fuel tube 4 is calculated. Further, the distance calculation unit 7h calculates the minimum distance by comparing the calculated distances with each other. Note that the distance calculation unit 7h calculates the distance from each point of the ignition coil 3 to the fuel without converting the point cloud data of the ignition coil 3 and the point cloud data of the fuel tube 4 into mesh data. The distance to each point on the tube 4 may be calculated separately.
  • the information output unit 7i generates various images to be displayed on the display 8 and data to be output as audio from a speaker (not shown). For example, the finally determined minimum distance, the above-mentioned degree of progress, etc. are displayed on one or more displays 8 that can be viewed by the operator.
  • the missing region determination unit 7j determines the missing portions, that is, the missing regions in the measurement data for each target component (ignition coil 3 and fuel tube 4). For example, in the design data of each part, the surface to be scanned is divided into a number of grids, and the ratio of points scanned for each grid (the ratio of the number of points actually scanned to the number of points to be scanned) is calculated. , if this ratio is less than or equal to a predetermined threshold, it is determined that the grid is a missing area. This determines which part of the design data of the target part is the missing area.
  • the measurement parameter calculation unit 7k calculates measurement parameters for measurement by the three-dimensional laser scanner 5, which are necessary for acquiring measurement data of these defective areas, for the defective areas determined by the defective area determination unit 7j.
  • the measurement parameters refer to the position of the three-dimensional laser scanner 5 and its pointing direction (in other words, the angle), and if the measurement mode such as laser intensity can be changed, the measurement mode, etc.
  • the position and direction of the 3D laser scanner 5 required to obtain measurement data of the defective area are continuously moved. Generated as a trajectory.
  • the measurement parameters generated in this way are displayed on the display 8 via the information output section 7i.
  • the position and range of the missing area may also be displayed. The operator will perform additional scanning according to the information on the display 8.
  • FIG. 9 is an explanatory diagram illustrating the measurement parameter calculation principle in the measurement parameter calculation section 7k.
  • the measurement object OJ has an area OJa where measurement data has already been acquired and a missing area OJb where measurement data is missing.
  • the measurement parameter calculation unit 7k first sets a large number of virtual viewpoints around the measurement object OJ, as shown in FIG. 9(a).
  • the square pyramid in the figure schematically represents the three-dimensional laser scanner 5, and the orientation of the square base is the pointing direction of the three-dimensional laser scanner 5.
  • a large number of virtual viewpoints having different positions and orientation directions are set, as shown by reference numerals 5a, 5b, 5c, . .
  • a virtual viewpoint is set for each mode.
  • the quality of data acquisition of the missing area OJb is evaluated. For example, it is evaluated whether the target defective region can be visually recognized linearly from a virtual viewpoint, whether the distance to the defective region is appropriate for measurement, whether a large number of defective regions can be measured at once, and so on.
  • a plurality of relatively advantageous virtual viewpoints having the same measurement mode are interpolated and connected into one continuous line, and the three-dimensional laser scanner 5 including the pointing direction is A movement trajectory TR11 is generated.
  • the condition is that the pointing direction does not change suddenly.
  • the symbol S indicates the start point of scanning, and the symbol E indicates the end point.
  • a preferred movement trajectory is generated for each mode.
  • the preferable movement trajectory TR11 generated in this way is displayed on the display 8 as an instruction trajectory for additional scanning, for example, as an image including the pointing direction as shown in the figure.
  • the movement trajectory determining unit 7m estimates the movement trajectory of the three-dimensional laser scanner 5 on which the worker is scanning based on the data acquired by the three-dimensional laser scanner 5, and determines the instruction indicating this movement trajectory. This is to determine whether there is a deviation from the trajectory. For example, as shown in FIG. 7, for a specific measurement target OJ (simplified in the figure) such as an assembly in the engine room 1, an instruction trajectory (work Trajectories (trajectories) TR1 and TR2 in which the operator should operate the three-dimensional laser scanner 5 are set, and instructions thereof are given to the operator by, for example, a display on the display 8 or a printed matter. The operator moves the three-dimensional laser scanner 5 along the indicated trajectories TR1 and TR2 to perform scanning.
  • the instruction trajectory TR1 is an instruction trajectory for scanning performed with the three-dimensional laser scanner 5 in the first measurement mode
  • the instruction trajectory TR2 is an instruction trajectory for scanning performed in the second mode. If the actual scanning operation by the operator does not follow the instruction trajectory correctly, the number of missing areas will increase, which is undesirable. Therefore, the movement trajectory determination unit 7m determines whether the actual movement trajectory deviates from the instructed trajectory, and if it deviates, it notifies the operator to that effect and prompts the operator to correct the movement trajectory. be.
  • FIG. 8 is an explanatory diagram illustrating the principle of movement trajectory determination. Since the basic shape or configuration of the area to be scanned is known as the design data obtained from the database 6, the space of the 3D laser scanner 5 is It is possible to estimate the position and the pointing direction of the three-dimensional laser scanner 5, and furthermore, it is possible to estimate the movement trajectory including the pointing direction of the three-dimensional laser scanner 5.
  • the movement trajectory TR1' estimated in this way is successively compared with the instruction trajectory TR1. For example, as shown in FIG. 8, the nearest two points from the coordinate string of points P1, P2, P3, . By determining the angle difference and the distance difference between the vector pairs connecting the points and comparing them, it is possible to determine the degree of coincidence between the local instruction trajectory TR1 and the movement trajectory TR1'. Note that other known appropriate algorithms can be used as the algorithm for determining the deviation of the movement trajectory.
  • a three-dimensional laser scanner 5 has a built-in vibrator, and warns and informs the worker by vibrating the three-dimensional laser scanner 5 held in the worker's hand.
  • the vibrator emits vibrations with a strength corresponding to the degree of deviation as information that prompts correction.
  • the three-dimensional laser scanner 5 begins to vibrate, and the further away from the instructed trajectory the stronger the vibration becomes, and then the closer it gets to the instructed trajectory, the weaker the vibration becomes. Even if the operator does not strictly grasp the instruction trajectory in space, scanning work along the instruction trajectory can be easily performed.
  • step S1 an operator operates the three-dimensional laser scanner 5 to scan a predetermined area in the engine room 1 including the ignition coil 3 and fuel tube 4, which are the target parts, from the outside to detect the outer surface of each part. Obtain point cloud data containing part of the . This scanning and generation of point cloud data progresses gradually along with the scanning operation.
  • step S2 CAD data of the target parts, such as the ignition coil 3, fuel tube 4, and other surrounding parts, is acquired from the database 6, along with CAD data indicating their spatial positional relationships.
  • step S3 the CAD data is converted to point cloud data using a known appropriate conversion method.
  • step S4 the first alignment unit 7a performs rough alignment between the entire target area and the entire reference area.
  • point cloud data of the entire area including the ignition coil 3, fuel tube 4, and other parts and point cloud data of the entire area including the ignition coil, fuel tube, and other parts in the design data stored in the database 6.
  • Rough alignment is performed by searching for key points and aligning by using combinations of paired key points.
  • step S5 the progress calculation unit 7b calculates the points of the point cloud data of the reference area and the matched points in the point cloud data of the scanned area.
  • the scanning progress is calculated from the ratio of .
  • step S6 it is determined whether this progress exceeds a predetermined progress threshold (for example, 40%). If the degree of progress is less than or equal to the threshold value, the process moves to step S7, the degree of progress is displayed on the display 8, and the worker continues scanning. In other words, the processing from step S1 onwards is repeated.
  • a predetermined progress threshold for example, 40%
  • step S8 the aforementioned movement trajectory determination unit 7m estimates the actual movement trajectory of the three-dimensional laser scanner 5 by the worker, and further in step S9, the estimated movement trajectory is a correct instruction trajectory. Determine whether it is in accordance with the If it follows the instructed trajectory, the process returns to step S1 and scanning is repeated. If the estimated movement trajectory deviates from the correct instruction trajectory, the operator is notified of this in step S10 and prompted to correct the trajectory, and then returns to step S1 to continue scanning. As described above, the notification to the operator is preferably made by using a vibrator built into the three-dimensional laser scanner 5 with a vibration having a strength corresponding to the degree of deviation.
  • steps S1 to S10 are repeated from the start of scanning until the degree of progress reaches a predetermined degree of progress threshold. If the progress level threshold is not reached even if the entire instruction trajectory is passed, it is necessary to perform scanning multiple times along the same instruction trajectory.
  • step S6 If the degree of progress exceeds the threshold in step S6, the process moves from step S6 to step S11, and the point cloud data of the ignition coil 3 and fuel tube 4, which are the target parts, are converted into point cloud data of measurement data and design data for the entire area. are extracted from the point cloud data.
  • Point cloud data extracted from measurement data becomes a so-called target
  • point cloud data extracted from design data becomes a so-called reference.
  • the point cloud data of the target part serving as a reference may be generated from CAD data of a single part.
  • step S12 the second positioning section 7c performs detailed positioning of the ignition coil 3 and fuel tube 4 using a known appropriate algorithm.
  • the fuel tube 4 is considered to be a rigid body. For example, as described above, search for paired points between the target point cloud data that has undergone rough alignment and the reference point cloud data, and perform detailed alignment so that the reference approaches the target. .
  • step S13 the data of the fuel tube 4 is down-sampled in step S13.
  • step S14 non-rigid positioning is performed in consideration of the deformation of the fuel tube 4.
  • the reference point group data is aligned to the target position while being transformed.
  • the nearest pair is searched from the reference point cloud data and the target point cloud data, which are aligned as rigid bodies as described above, and rotation, enlargement, and translation are used as parameters to bring them closer to each other. seek.
  • step S15 the reliability of alignment is calculated for each of the ignition coil 3 and the fuel tube 4.
  • the reliability is expressed, for example, as a ratio between the number of points in the reference point group data and the number of points in the target point group data included within a predetermined radius from each point in the reference point group data.
  • step S16 it is determined whether the reliability of the alignment of the ignition coil 3 and the reliability of the alignment of the fuel tube 4 each satisfy a predetermined reliability. If both reliability levels satisfy the predetermined reliability level, it is assumed that the alignment has been completed, and the process moves to step S17.
  • step S17 the first interpolation section 7f and the second interpolation section 7g interpolate the unscanned portions of the target point cloud data of the ignition coil 3 and fuel tube 4 with the reference point cloud data aligned with the targets. do.
  • point cloud data of the ignition coil 3 and the fuel tube 4 including unscanned portions that is, hidden portions
  • step S18 using a known conversion method, the point cloud data including the unscanned portions of both the ignition coil 3 and the fuel tube 4 are converted into mesh data forming a surface.
  • step S19 the distance calculation unit 7h calculates the minimum distance between the ignition coil 3 and the fuel tube 4 based on the mesh data of the ignition coil 3 and the mesh data of the fuel tube 4. That is, the distance between any two points on each surface is determined, and the minimum value therebetween is determined as the minimum distance.
  • step S20 the test results including this minimum distance and other necessary information are displayed on the display 8.
  • the calculated minimum distance may be compared with a threshold value, and some kind of warning may be displayed if it is less than the threshold value.
  • step S16 determines whether the reliability of alignment is insufficient for any component. If it is determined in step S16 that the reliability of alignment is insufficient for any component, the process proceeds from step S16 to step S21, and a missing area is determined for the component with insufficient reliability.
  • the surface to be scanned is divided into many grids, and the ratio of point clouds scanned for each grid (the ratio of the number of points actually scanned to the number of points to be scanned) is calculated. ), and if this ratio is less than or equal to a predetermined threshold, it is determined that the grid is a missing area.
  • step S21 it means that the lack of reliability is not caused by the missing area, for example, the target part is not recognized correctly, so some kind of warning etc. is displayed on the display 8. Finish the process. In this case, for example, scanning must be restarted from the beginning.
  • step S21 If it is determined in step S21 that there is a missing area (in other words, the insufficient reliability is due to the missing area), the process proceeds from step S21 to step S22, in which the target area (engine A plurality of virtual viewpoints are set around the room 1) and the advantages of each are evaluated. Then, in step S23, measurement parameters necessary for additional scanning (trajectory of movement of the three-dimensional laser scanner 5, pointing direction, measurement mode, etc.) are calculated, and in step S24, additional scanning is performed using these measurement parameters. Instruct workers what to do. This may be done by, for example, a display on the display 8, an audio instruction, or the like. It is desirable that the necessary movement trajectory calculated as a parameter is displayed on the screen as an instruction trajectory for additional scanning.
  • step S16 the alignment reliability satisfies a predetermined reliability
  • step S17 it is determined that there is no missing area (for example, below a certain percentage)
  • the missing area is determined for the part where data could be obtained if scanned properly, the measurement parameters necessary to obtain the data of this missing area are calculated, and the Since the operator is prompted to perform additional scanning in accordance with the measurement parameters, three-dimensional measurement data can be efficiently acquired. Furthermore, it is possible to suppress various kinds of deterioration in inspection accuracy due to the presence of defective areas during scanning. In particular, since the missing area is determined only for parts with low alignment reliability and additional scanning is performed to fill the missing area of the part, the additional scanning is efficient.
  • the determination of the missing area is performed after aligning the reference point cloud data with the target point cloud data, it is possible to more accurately grasp which part of the target part is the missing area.
  • the present invention is not limited to such applications, and is applicable to assemblies including a plurality of parts. It can be widely applied to inspections related to the spatial arrangement of target parts, such as their position and orientation in space. In the above embodiment, two parts are the target parts in order to inspect the distance between the two parts, but the present invention can be applied even if there is only one target part.
  • the three-dimensional sensor is not limited to the three-dimensional laser scanner 5 of the above embodiment, but may be of any type as long as it can acquire and generate three-dimensional point cloud data. It can be widely applied, such as a ToF format or a triangulation method such as a stereo camera.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Optics & Photonics (AREA)
  • Theoretical Computer Science (AREA)
  • Length Measuring Devices By Optical Means (AREA)
PCT/JP2023/009217 2022-06-09 2023-03-10 部品検査方法および装置 Ceased WO2023238465A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN202380045118.3A CN119317810A (zh) 2022-06-09 2023-03-10 部件检查方法和装置
EP23819452.6A EP4538635A4 (en) 2022-06-09 2023-03-10 METHOD AND DEVICE FOR INSPECTING A COMPONENT
JP2024526242A JP7750410B2 (ja) 2022-06-09 2023-03-10 部品検査方法および装置

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022093398 2022-06-09
JP2022-093398 2022-06-09

Publications (1)

Publication Number Publication Date
WO2023238465A1 true WO2023238465A1 (ja) 2023-12-14

Family

ID=89117973

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/009217 Ceased WO2023238465A1 (ja) 2022-06-09 2023-03-10 部品検査方法および装置

Country Status (4)

Country Link
EP (1) EP4538635A4 (https=)
JP (1) JP7750410B2 (https=)
CN (1) CN119317810A (https=)
WO (1) WO2023238465A1 (https=)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011174891A (ja) 2010-02-25 2011-09-08 Canon Inc 位置姿勢計測装置、位置姿勢計測方法、及びプログラム
JP2014153149A (ja) * 2013-02-07 2014-08-25 Nikon Corp 形状測定装置、構造物製造システム、形状測定方法、及びプログラム
JP2015068654A (ja) * 2013-09-26 2015-04-13 株式会社ニコン 形状測定装置、構造物製造システム及び形状測定用コンピュータプログラム
WO2017126060A1 (ja) * 2016-01-20 2017-07-27 三菱電機株式会社 ３次元計測装置及びその計測支援処理方法
JP2018054430A (ja) * 2016-09-28 2018-04-05 株式会社東芝 検査装置及び計測軌道経路生成方法
JP2020125916A (ja) * 2019-02-01 2020-08-20 Ｋｙｏｔｏ Ｒｏｂｏｔｉｃｓ株式会社 移載対象ワーク記憶システム

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9482529B2 (en) * 2011-04-15 2016-11-01 Faro Technologies, Inc. Three-dimensional coordinate scanner and method of operation
US10824773B2 (en) * 2017-03-28 2020-11-03 Faro Technologies, Inc. System and method of scanning an environment and generating two dimensional images of the environment

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011174891A (ja) 2010-02-25 2011-09-08 Canon Inc 位置姿勢計測装置、位置姿勢計測方法、及びプログラム
JP2014153149A (ja) * 2013-02-07 2014-08-25 Nikon Corp 形状測定装置、構造物製造システム、形状測定方法、及びプログラム
JP2015068654A (ja) * 2013-09-26 2015-04-13 株式会社ニコン 形状測定装置、構造物製造システム及び形状測定用コンピュータプログラム
WO2017126060A1 (ja) * 2016-01-20 2017-07-27 三菱電機株式会社 ３次元計測装置及びその計測支援処理方法
JP2018054430A (ja) * 2016-09-28 2018-04-05 株式会社東芝 検査装置及び計測軌道経路生成方法
JP2020125916A (ja) * 2019-02-01 2020-08-20 Ｋｙｏｔｏ Ｒｏｂｏｔｉｃｓ株式会社 移載対象ワーク記憶システム

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4538635A4

Also Published As

Publication number Publication date
JP7750410B2 (ja) 2025-10-07
JPWO2023238465A1 (https=) 2023-12-14
EP4538635A1 (en) 2025-04-16
CN119317810A (zh) 2025-01-14
EP4538635A4 (en) 2025-09-17

Similar Documents

Publication Publication Date Title
EP2257797B1 (en) System, program product, and related methods for registering three-dimensional models to point data representing the pose of a part
KR101155816B1 (ko) 3차원 계측을 행하는 화상 처리 장치 및 화상 처리 방법
US6690841B2 (en) Method and apparatus for image registration
JP5388921B2 (ja) ３次元距離計測装置及びその方法
JP4442661B2 (ja) ３次元計測方法および３次元計測装置
JP7228951B2 (ja) 損失価格評価システム
JP4774824B2 (ja) ３次元計測処理の計測対象範囲の確認方法および計測対象範囲の設定方法ならびに各方法を実施する装置
US12553833B2 (en) Visual inspection apparatus and visual inspection method
CN117255931A (zh) 图像处理装置、图像处理方法及图像处理程序
JP7750410B2 (ja) 部品検査方法および装置
Chatterjee et al. Viewpoint planning and 3D image stitching algorithms for inspection of panels
JP7726400B2 (ja) 隙間検査方法および隙間検査装置
JP7747201B2 (ja) 部品検査方法および部品検査装置
JP2021128099A (ja) 付属品の取り付け位置検査方法及び取り付け位置検査装置
CN120953977B (zh) 一种焊接模型化管理方法及其系统
CN118408607B (zh) 汽车零部件检测方法及装置
JP2020201107A (ja) 撮影処理装置、及び撮影処理方法
JP2021081384A (ja) 面歪評価装置、面歪評価方法、及び、プログラム
Shi et al. Develop a Non-contact Automated Dimensional Inspection System for Automotive Manufacturing Industry

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23819452

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024526242

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 202380045118.3

Country of ref document: CN

WWE Wipo information: entry into national phase

Ref document number: 2023819452

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

WWP Wipo information: published in national office

Ref document number: 202380045118.3

Country of ref document: CN

ENP Entry into the national phase

Ref document number: 2023819452

Country of ref document: EP

Effective date: 20250109

WWP Wipo information: published in national office

Ref document number: 2023819452

Country of ref document: EP

WWW Wipo information: withdrawn in national office

Ref document number: 2023819452

Country of ref document: EP