WO2021181441A1 - 異常判定システム、撮影装置および異常判定方法 - Google Patents

異常判定システム、撮影装置および異常判定方法 Download PDF

Info

Publication number
WO2021181441A1
WO2021181441A1 PCT/JP2020/009914 JP2020009914W WO2021181441A1 WO 2021181441 A1 WO2021181441 A1 WO 2021181441A1 JP 2020009914 W JP2020009914 W JP 2020009914W WO 2021181441 A1 WO2021181441 A1 WO 2021181441A1
Authority
WO
WIPO (PCT)
Prior art keywords
measured
displacement
optical path
lens
plane displacement
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/009914
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
浩 今井
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Priority to JP2022506988A priority Critical patent/JPWO2021181441A1/ja
Priority to US17/908,019 priority patent/US20230111602A1/en
Priority to PCT/JP2020/009914 priority patent/WO2021181441A1/ja
Publication of WO2021181441A1 publication Critical patent/WO2021181441A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B17/00—Systems with reflecting surfaces, with or without refracting elements
    • G02B17/02—Catoptric systems, e.g. image erecting and reversing system
    • G02B17/06—Catoptric systems, e.g. image erecting and reversing system using mirrors only, i.e. having only one curved mirror
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G06T7/001—Industrial image inspection using an image reference approach
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B5/00—Optical elements other than lenses
    • G02B5/04—Prisms
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30181—Earth observation
    • G06T2207/30184—Infrastructure

Definitions

  • the present invention relates to a technique for non-contact detection of abnormalities such as cracks on the surface of an object.
  • defects such as cracks, peeling, and internal cavities that occur on the surface can adversely affect the soundness of the structure. Therefore, it is necessary to detect such defects as anomalies as quickly and accurately as possible.
  • Patent Document 1 describes image features of defects such as cracks that are previously obtained from a binarized image generated by binarizing an image obtained by photographing a structure with a camera.
  • a method for detecting a defect in a structure is disclosed.
  • Patent Documents 2 and 3 disclose a technique for detecting defects in a structure based on the stress generated in the structure.
  • Patent Documents 4 and 5 disclose a technique for detecting a defect of an object from a moving image obtained by photographing the object with one camera.
  • displacement in the direction along the surface of the object surface also referred to as in-plane displacement
  • displacement in the direction along the optical axis of the camera also referred to as out-of-plane displacement
  • NS defects (abnormalities) of objects such as cracks, peelings, and internal cavities are detected based on the detected out-of-plane displacement and in-plane displacement.
  • Non-Patent Document 1 discloses a method of measuring in-plane displacement from a moving image of the surface of a structure.
  • in-plane displacement and out-of-plane displacement of an object are calculated from an image taken by one camera, and defects in the object are detected using the calculated in-plane displacement and out-of-plane displacement. ..
  • the main object of the present invention is to facilitate the measurement resolution of both in-plane displacement and out-of-plane displacement on the surface of the object, and to improve the detection accuracy of detecting the defect of the object from the captured image. It is to provide the technology that can be done.
  • the abnormality determination system is, as one form thereof.
  • An imaging device that outputs a time-series image including a plurality of captured images in which the surface of the object to be measured is photographed over time, and It is interposed in the optical path portion between the object to be measured and the lens in the optical path from the object to be measured to the image pickup surface through the lens provided in the imaging device, and reaches the image pickup surface from the lens.
  • An optical path bending member that bends the traveling direction of light so as to incline the traveling direction of light up to the lens in a direction closer to the optical axis of the lens.
  • the out-of-plane displacement which is the displacement in the normal direction on the surface of the object to be measured, calculated by utilizing the displacement of the surface of the object to be measured measured from the time-series image, and the measured object to be measured. It is provided with an abnormality determination device for determining an abnormality of the object to be measured by using an in-plane displacement which is a displacement on the surface of the object to be measured, which is calculated by subtracting the out-of-plane displacement from the displacement of the surface. ..
  • An imaging surface that captures an image of the surface of the object to be measured, and A lens that guides external light to the imaging surface, The traveling direction of light from the lens to the imaging surface, which is interposed in the optical path portion between the object to be measured and the lens in the optical path from the object to be measured to the imaging surface through the lens. It is provided with an optical path bending member that bends the traveling direction of light so as to incline the lens in a direction closer to the optical axis of the lens.
  • the abnormality determination method is, as one form thereof.
  • An optical path bending member that bends the traveling direction of the light so as to incline the traveling direction of the light from the lens to the imaging surface provided in the photographing apparatus in a direction closer to the optical axis of the lens is provided.
  • the out-of-plane displacement which is a linear displacement
  • the in-plane displacement which is the displacement on the surface of the object to be measured, calculated by subtracting the out-of-plane displacement from the measured displacement of the surface of the object to be measured. It is used to determine the abnormality of the object to be measured.
  • the present invention it is possible to easily increase the measurement resolution of both in-plane displacement and out-of-plane displacement on the surface of an object, and to improve the detection accuracy of detecting defects in an object from a captured image.
  • FIG. 12A It is a figure explaining the in-plane displacement when there is a crack on the surface of the object to be measured together with FIG. 13B. It is a figure explaining the in-plane displacement when there is a crack on the surface of the object to be measured together with FIG. 13A.
  • FIG. 1 is a block diagram showing the configuration of the abnormality determination system of the first embodiment according to the present invention together with the object to be measured.
  • the abnormality determination system 1 of the first embodiment has a function of determining cracks and peeling of the surface of the object to be measured 3 and abnormalities of the internal cavity.
  • the object 3 to be measured is a structure such as a building, a tunnel, a bridge, or a machine such as a car or a manufacturing device.
  • the object 3 to be measured does not itself displace due to movement, rotational movement, or the like when it is measured by the abnormality determination system 1. However, it bends or vibrates when some force is applied to the object.
  • the abnormality determination system 1 includes a photographing device 10, an abnormality determination device 11, a notification device 12, and an optical path bending member 13.
  • the photographing device 10 is a device that photographs the surface of the object 3 to be measured, and has a function of generating and outputting a time-series frame image (hereinafter, also referred to as a time-series image).
  • the frame rate of the time-series image is appropriately set in the range of, for example, 60 fps (frames per second) to 1000 fps.
  • FIG. 2 is a block diagram showing the configuration of the photographing apparatus 10 together with the optical path bending member 13.
  • the photographing device 10 includes an imaging surface 101 and a lens 102.
  • An optical path bending member 13 is integrally mounted on the photographing device 10. That is, the imaging surface 101, the lens 102, and the optical path bending member 13 are arranged in this order, and light enters the inside of the photographing apparatus 10 through the optical path bending member 13, and the light passes through the lens 102. It reaches the imaging surface 101 by the optical path.
  • the image pickup surface 101 has a configuration in which a plurality of image pickup elements that convert light into electric signals are arranged in a matrix, and image data is generated from the electric signals output from each image pickup element to generate a frame image. Is generated.
  • the optical path bending member 13 has a configuration in which the traveling direction of light that is about to enter the photographing device 10 from the outside is bent in a direction that approaches the optical axis of the lens 102.
  • 3 and 4 respectively, show specific examples of an optical path that passes through the optical path bending member 13 and the lens 102 in order to reach the imaging surface 101.
  • the optical path is bent by the optical path bending member 13. That is, when the optical path bending member 13 is not provided, light is incident on the lens 102 with an inclination of an angle ⁇ a with respect to the optical axis of the lens 102, as shown by the dotted line in FIG. It is assumed that the light is emitted from the lens 102.
  • the optical path bending member 13 is not limited in its configuration as long as the optical path can be bent in a direction closer to the optical axis of the lens 102, and may be, for example, a triangular prism as shown in FIG. 5A. , May be configured with a mirror as shown in FIG. 5B.
  • such a triangular prism is arranged so that the incident angle ⁇ of light is 35 °. In this case, the refraction angle ⁇ c of the light in the triangular prism is 20 °.
  • the angle of view of the photographing apparatus 10 when the optical path bending member 13 is provided will be described.
  • the angle of half the angle of view of the lens 102 be ⁇ a shown in FIG.
  • the refraction angle (deflection angle) of the light by the optical path bending member 13 is set to ⁇ c.
  • the angle formed by the optical path formed by the light entering the photographing apparatus 10 after being bent by the optical path bending member 13 and the optical axis of the lens 102 is defined as ⁇ b.
  • the distance (focal length) between the main surface of the lens 102 and the imaging surface 101 is f
  • the distance between the optical path bending member 13 and the surface Qa of the object 3 to be measured is f
  • Ja is the limit position of the visual field range of the photographing apparatus 10 when the optical path bending member 13 is not provided on the surface Qa of the object 3 to be measured.
  • the position where the portion of the position Ja on the surface Qa of the object 3 to be measured passes through the optical path bending member 13 and the lens 102 in order and is imaged on the imaging surface 101 is defined as Ra. ..
  • the position where the virtual line passing through the main surface of the lens 102 is extended from this position Ra and reaches the surface Qa of the object to be measured 3 is defined as Jc.
  • X1 be the length between Ja and Jc on the surface Qa of the object to be measured 3
  • X2 be the length between the intersection Jd and Jc with the optical axis of the lens 102 on the surface Qa of the object 3 to be measured.
  • mathematical formula 1 and mathematical formula 2 are derived from the geometrical relationship of light rays.
  • Formula 3 is derived by rewriting Formula 2 using Formula 1.
  • the angle ⁇ a corresponding to the angle of view of the photographing apparatus 10 when the optical path bending member 13 is not provided, the refraction angle ⁇ c of the light by the optical path bending member 13, and the lengths L1 and L2 are given.
  • the angle ⁇ b can be calculated from the inverse function of the obtained tan ⁇ b.
  • L2 is 20 mm
  • the angle of view of the photographing apparatus 10 when the optical path bending member 13 is not provided is set to 20 °
  • the refraction of the optical path bending member 13 is set to 20 °.
  • the angle ⁇ c is 20 °
  • the angle ⁇ b is found to be 0.42 °.
  • a frame image is generated based on the light that has reached the imaging surface 101 through the optical path bending member 13 and the lens 102 as described above, and a time-series image based on the generated frame image is generated.
  • the photographing device 10 is connected to the abnormality determination device 11, and outputs the generated time-series image to the abnormality determination device 11.
  • the abnormality determination device 11 has a function of determining a crack or peeling of the surface of the object 3 to be measured and an abnormality of the internal cavity by using the time series image received from the photographing device 10.
  • the abnormality determination device 11 is a computer, and includes a processor such as a CPU (Central Processing Unit) and a storage device such as a memory or an HDD (Hard Disk Drive) which is a storage medium.
  • FIG. 6 is a block diagram showing a functional configuration of the abnormality determination device 11.
  • the abnormality determination device 11 can have a function corresponding to the program by executing the computer program (hereinafter, also referred to as a program) stored in the storage device by the processor.
  • the abnormality determination device 11 has a functional unit of a displacement calculation unit 111, an out-of-plane displacement calculation unit 112, an in-plane displacement calculation unit 113, and a determination unit 114 as shown in FIG.
  • the displacement calculation unit 111 has a function of calculating (measuring), for example, the displacement (displacement direction and displacement amount) on the surface of the object 3 to be measured between the frame images in the time-series image received from the photographing device 10 for each pixel of the frame image.
  • the frame image to be processed by the displacement calculation unit 111 may be all the frame images included in the time-series image, or for example, a preset frame for each number of preset frames from the time-series frame images. It may be an image.
  • the displacement calculation unit 111 compares the adjacent frame images when a plurality of frame images to be processed are arranged in chronological order to determine the displacement of the surface of the object 3 to be measured in the captured image, for example, the frame image.
  • Calculate (measure) for each pixel For example, there is a method using an image correlation calculation based on a correlation or change between frame images, and a gradient method. Further, when calculating the displacement of the surface of the object 3 to be measured, the quadratic curve interpolation method may be used in the image correlation calculation. In this case, the displacement calculation unit 111 of the image pickup element on the image pickup surface 101. The displacement can be calculated at the level of 1/100 of the array pitch.
  • the displacement calculation unit 111 may have a function of generating a displacement distribution map in a two-dimensional space based on the calculated displacement.
  • the displacement calculation unit 111 may further have the following functions. That is, the normal direction of the surface of the object 3 to be measured may not be the direction along the optical axis of the lens 102. In this case, the displacement calculation unit 111 corrects the displacement according to the deviation of the surface of the object to be measured 3 with respect to the optical axis of the lens 102 in the normal direction by executing the perspective projection conversion process, and the corrected displacement. Is used to generate a displacement distribution map in two-dimensional space.
  • the displacement calculation unit 111 is used to calculate the image correlation by using the SAD (Sum of Absolute Difference) method, the SSD (Sum of Squared Difference) method, the NCC (Normalized Cross Correlation) method, or the ZNCC (Zero-mean Normalized Cross). Correlation) method or the like may be used. Further, the displacement calculation unit 111 may use these methods in combination.
  • FIGS. 8A and 8B the imaging surface 101 is orthogonal to the optical axis of the lens 102 (omitted in FIGS. 8A and 8B) of the photographing apparatus 10, and the direction along the optical axis is the Z direction.
  • the two directions orthogonal to each other and parallel to the imaging surface 101 are the X direction and the Y direction.
  • FIG. 8A shows an optical system on an XZ plane including an optical axis and along the X and Z directions
  • FIG. 8B shows a YZ plane including an optical axis and along the Y and Z directions. The optical system above is shown.
  • the coordinates representing the position on the imaging surface 101 are represented by using a two-dimensional Cartesian coordinate system with the intersection with the optical axis as the origin. Further, the coordinate system representing the position on the surface of the object 3 to be measured (here, also referred to as the object space coordinate) conforms to the coordinate system representing the position on the imaging surface 101. However, since the image of the object is inverted on the imaging surface 101, the coordinates in the X and Y directions representing the positions on the imaging surface 101 and the coordinates in the X and Y directions in the object space coordinates are positive and negative. The orientations are set opposite to each other.
  • FIGS. 8A and 8B it is assumed that the point M on the surface Qa of the object to be measured 3 is imaged at the point N on the imaging surface 101.
  • the surface Qa of the object to be measured 3 is displaced in the Z direction due to vibration, for example, and the displacement of the point M due to vibration is ⁇ Z.
  • This displacement is the out-of-plane displacement.
  • the displacement due to this displacement is the displacement according to the out-of-plane displacement.
  • the displacement in the X direction from the point N to the point Nb is expressed as ⁇ Xi
  • the displacement in the Y direction from the point N to the point Nb is expressed as ⁇ Yi.
  • the point M of the surface Qa of the object 3 to be measured is displaced in the X direction and the Y direction by ⁇ X and ⁇ Y, respectively. Due to this displacement, the image of the point M is imaged at the position of the point Nc on the imaging surface 101.
  • the displacement from the point Nb to the point Nc is the in-plane displacement.
  • the displacement in the X direction from the point Nb to the point Nc is represented by ⁇ Xi
  • the displacement in the Y direction from the point Nb to the point Nc is represented by ⁇ Yi.
  • the distance between the principal point of the lens 102 and the surface Qa of the object to be measured 3 is defined as the imaging distance L, and the distance between the principal point of the lens 102 and the imaging surface 101 is focused.
  • the distance be f.
  • the distance between the point M and the origin O in the X direction is X
  • the distance between the point M and the origin O in the Y direction is Y.
  • the displacement ⁇ Xi in the X direction and the displacement ⁇ Yi in the Y direction of the out-of-plane displacement can be expressed by Equation 4.
  • the displacement ⁇ Xi in the X direction and the displacement ⁇ Yi in the Y direction of the in-plane displacement can be expressed by Equation 5. (Formula 4)
  • the out-of-plane displacement calculation unit 112 of the abnormality determination device 11 has a function of calculating the out-of-plane displacement of the object to be measured 3 as follows by using the time-series image obtained by the photographing device 10. The method of calculating the out-of-plane displacement will be described with reference to FIG.
  • the out-of-plane displacements with respect to the points M1 and M2 are ⁇ X1i. It becomes ⁇ X2i.
  • the out-of-plane displacement ⁇ Z can be obtained from the mathematical formula 7 by using the difference ⁇ d between the out-of-plane displacements ⁇ X1i and ⁇ X2i represented by the following mathematical formula 6.
  • Equation 7 corresponds to the distance ⁇ between the images N1a and N2a. Further, there is a relationship as expressed in Equation 8 between such a distance ⁇ and a distance ⁇ . Further, the difference ⁇ d is calculated by the displacement calculation unit 111.
  • the out-of-plane displacement calculation unit 112 calculates the out-of-plane displacement ⁇ Z based on the mathematical formulas 6 and 7. (Formula 6)
  • the out-of-plane displacement vector Vo on the imaging surface 101 is a group of radial vectors centered on the intersection O (in other words, the imaging center) with the optical axis of the imaging device 10. .. That is, even if the out-of-plane displacement of the surface Qa of the object 3 to be measured is uniform, the length of the out-of-plane displacement vector Vo on the imaging surface 101 increases in proportion to the distance from the imaging center O.
  • the relationship between the distance from the imaging center O in the X direction and the length of the out-of-plane displacement vector is as shown by the straight line D1 shown in the graph of FIG. 10B.
  • the inclination corresponds to the out-of-plane displacement ⁇ Z.
  • the out-of-plane displacement calculation unit 112 can also calculate the out-of-plane displacement ⁇ Z by using the length of the out-of-plane displacement vector as described above. That is, the out-of-plane displacement calculation unit 112 may calculate the out-of-plane displacement by performing a linear regression calculation on the relationship shown in FIG. 10B, in addition to the calculation method based on the mathematical formulas 7 and 8.
  • the photographing device 10 includes an optical path bending member 13. Therefore, when the optical path bending member 13 is not provided, the optical path is as shown by the dotted lines in FIGS. 3 and 4, whereas when the optical path bending member 13 is provided, the optical path is as shown in FIGS. 3 and 4.
  • the optical path is as shown by the solid line of 4. That is, when the optical path bending member 13 is not provided, the point Ja on the surface Qa of the object to be measured 3 is imaged at the point Rc on the imaging surface 101 in FIGS. 3 and 4. Further, in FIG. 3, when the point Ja is displaced to the position of the point Jb due to, for example, vibration of the surface Qa of the object 3 to be measured, the point Jb is imaged at the point Rd on the imaging surface 101.
  • the point Ja on the surface Qa of the object to be measured 3 is imaged at the point Ra on the imaging surface 101 in FIGS. 3 and 4. Further, in FIG. 3, when the point Ja is displaced to the position of the point Jb due to, for example, vibration of the surface Qa of the object 3 to be measured, the point Jb is imaged at the point Rb on the imaging surface 101.
  • the out-of-plane displacement of the surface Qa of the object 3 to be measured shown in FIG. 3 from the point Ja to the point Jb is from the point Rc on the imaging surface 101. It is represented by the displacement of the point Rd.
  • the out-of-plane displacement of the surface Qa of the object 3 to be measured from the point Ja to the point Jb is represented by the displacement from the point Ra to the point Rb on the imaging surface 101. ..
  • the out-of-plane displacement vector on the surface Qa of the object 3 to be measured on the imaging surface 101 is different from that in the case where the optical path bending member 13 is not provided. It will be closer to the optical axis.
  • the length of the out-of-plane displacement vector on the imaging surface 101 as shown by the line D1 in FIGS. 10B and 10C. Becomes longer in proportion to the distance from the imaging center O.
  • the relationship between the length of the out-of-plane displacement vector and the distance from the imaging center O is the d1 portion of the line D1 shown in FIG. 10C (for example, the distance of the imaging surface 101 from the imaging center O is the imaging surface 101.
  • the portion of the distance beyond the edge of the image) shifts in the direction closer to the image pickup center O, and is represented by the straight line D2.
  • the length of the out-of-plane displacement vector on the imaging surface 101 is such that the optical path bending member 13 is provided, so that the optical path bending member 13 is not provided. It will be longer than in the case. That is, by providing the optical path bending member 13, the effect that the measurement resolution of the out-of-plane displacement can be improved can be obtained.
  • the out-of-plane displacement calculation unit 112 may calculate the out-of-plane displacement as follows when the in-plane displacement is sufficiently smaller than the out-of-plane displacement. That is, the out-of-plane displacement calculation unit 112 obtains the coefficient k that minimizes S (k) represented by the formula 9, substitutes the obtained coefficient k into the formula 11, and calculates the out-of-plane displacement ⁇ Z by calculating the formula 11. be able to.
  • the coefficient k in Equations 9 and 11 corresponds to the out-of-plane displacement and is represented by Equation 10. Further, the mathematical formula 11 is derived from the mathematical formula 10. (Formula 9)
  • i in the mathematical formula 9 represents a number assigned in advance for identifying the image pickup element constituting the image pickup surface 101.
  • the displacement of the image pickup on the image pickup surface 101 according to the displacement of the surface Qa due to the vibration of the object 3 to be measured is expressed as the displacement vector Vi.
  • Vxi in Equation 9 represents the x component of the displacement vector Vi
  • Vyi represents the y component of the displacement vector Vi.
  • xi and yi in Equation 9 are x and y components of the displacement vector on the imaging surface 101 according to the out-of-plane displacement that should be measured when the optical path bending member 13 is not provided. (Formula 10)
  • the out-of-plane displacement calculation unit 112 may calculate the out-of-plane displacement ⁇ Z by the method as described above.
  • the in-plane displacement calculation unit 113 shown in FIG. 6 has a function of calculating the in-plane displacement.
  • FIG. 11 is a diagram for explaining the relationship between the out-of-plane displacement vector and the in-plane displacement vector.
  • the dotted line Vk represents the displacement calculated by the displacement calculation unit 111 (hereinafter, referred to as the measurement vector Vk (Vx i , Vy i )).
  • the measurement vector Vk (Vx i , Vy i ) is a composite vector of the out-of-plane displacement vector ⁇ ( ⁇ x i , ⁇ y i ) and the in-plane displacement vector ⁇ ( ⁇ x i , ⁇ y i).
  • the in-plane displacement calculation unit 113 subtracts the X component of the out-of-plane displacement vector ⁇ from the X component of the measurement vector Vk at each point of each section calculated by the displacement calculation unit 111, so that the X of the in-plane displacement vector ⁇ is X.
  • the component is separated from the measurement vector Vk.
  • An interpolation method using a quadric surface or an equiangular straight line may be used when the out-of-plane displacement calculation unit 112 calculates the out-of-plane displacement and the in-plane displacement calculation unit 113 calculates the in-plane displacement.
  • the determination unit 114 has a function of detecting an abnormality of the object to be measured 3 based on a time change of the displacement of the surface of the object 3 to be measured.
  • the determination unit 114 includes a three-dimensional spatial distribution information analysis unit 115 and a time change information analysis unit 116, as shown in FIG.
  • the three-dimensional spatial distribution information analysis unit 115 has a function of analyzing the three-dimensional displacement distribution of the object 3 to be measured at the time of interest.
  • the time change information analysis unit 116 has a function of analyzing the time change of the three-dimensional displacement in the portion of interest on the surface of the object 3 to be measured.
  • FIG. 12A shows an example of the object to be measured 3.
  • the object 3 to be measured shown in FIG. 12A is a cantilever, and the fixed end portion 4 is fixed. Further, in the example here, the object 3 to be measured has a size of 700 mm (millimeters) in length, 150 mm in width, and 3 mm in thickness. Further, it is assumed that such an object 3 to be measured vibrates naturally.
  • FIG. 12B shows the frequency characteristics of the out-of-plane displacement of the object 3 to be measured due to the natural vibration.
  • the object 3 to be measured normally vibrates naturally, it has the frequency characteristics of vibration as shown by the solid line in FIG. 12B, and the natural frequencies of the primary, secondary, and tertiary are set to It is assumed that the frequency is 10 Hz, 50 Hz, and 150 Hz.
  • the natural frequency of the object to be measured 3 at the time of such normal vibration is stored in advance in a storage device provided in the abnormality determination device 11.
  • the object 3 to be measured is marked with a dotted line in FIG. 12B. It has the frequency characteristics of vibration as shown, and the frequency characteristics are different from those in the normal state. As shown in FIG. 12B, the natural frequencies of the primary, secondary, and tertiary objects 3 under abnormal vibration tend to be lower than those in the normal state.
  • the frame rate of the moving image of the photographing device 10 is set to 3 in view of the sampling theorem. It is set to 400 fps, which is more than twice the next natural frequency of 150 Hz. As described above, the frame rate may be appropriately set in consideration of the frequency characteristics of the vibration of the object to be measured, and is not limited to 400 fps.
  • the imaging distance is 1 m
  • the focal length of the lens of the photographing device is 50 mm
  • the pixel pitch is 5 ⁇ m
  • a resolution of 0.1 mm per pixel is realized.
  • the displacement measurement unit 111 realizes a displacement measurement resolution of 1 ⁇ m by interpolating the displacement to 1/100 pixel by using the quadratic curve interpolation method in the above-mentioned image correlation calculation.
  • the in-plane displacement when a crack is generated on the surface of the object to be measured (cantilever) 3 as shown in FIG. 13A will be described.
  • the opening on the surface of the object 3 to be measured due to the crack opens and closes due to the vibration of the object 3 to be measured.
  • the opening and closing of the opening due to this crack causes in-plane displacement on the surface of the object to be measured 3.
  • FIG. 14A is a diagram showing the time change of the in-plane displacement at the point Ca and the point Cb
  • FIG. 14B is a diagram showing the in-plane displacement distribution on a straight line passing through the point Ca and the point Cb.
  • the spatial displacement distribution in the plane becomes continuous as shown by the solid line in FIG. 14B.
  • the spatial in-plane displacement distribution exhibits a sharp, intermittent change between points Ca and Cb, as shown by the dotted line in FIG. 14B.
  • the three-dimensional spatial distribution information analysis unit 115 of the determination unit 114 analyzes the three-dimensional displacement distribution of the object to be measured at a plurality of time points of interest. Further, the time change information analysis unit 116 analyzes the time change of the three-dimensional displacement in a plurality of portions on the surface of the object to be measured. Based on the information obtained by the three-dimensional spatial distribution information analysis unit 115 and the time change information analysis unit 116, the determination unit 114 determines the abnormality of the object 3 to be measured. This determination result is output to, for example, the notification device 12.
  • the notification device 12 notifies the determination result visually by, for example, a screen display, or audibly by a speaker or the like.
  • the information output by the notification device 12 may be information in a form for being read by a machine, in addition to information in a form that can be visually and audibly recognized by a person.
  • the determination unit 114 determines the abnormality of the object to be measured 3 by utilizing both the out-of-plane displacement and the in-plane displacement.
  • the determination unit 114 may determine the abnormality of the object to be measured 3 by using either the out-of-plane displacement or the in-plane displacement.
  • the determination unit 114 may be used for other purposes such as material estimation by utilizing the property that the vibrating body has different natural frequencies depending on the material even if the dimensions are the same. Information may be output according to the purpose.
  • FIG. 15 is a block diagram showing an example of the hardware configuration of the abnormality determination device 11.
  • the abnormality determination device 11 is composed of a signal processing device (computer device) 900.
  • the signal processing device 900 includes the following configuration as an example.
  • -CPU Central Processing Unit
  • ROM Read Only Memory
  • RAM Random Access Memory
  • -Program 904 loaded into RAM 903
  • a storage device 905 that stores the program 904.
  • Drive device 907 that reads and writes the storage medium 906.
  • -Communication interface 908 that connects to the communication network 909 -I / O interface 910 for inputting / outputting data -Bus 911 connecting each component
  • Each of the above-mentioned functional units of the abnormality determination device 11 is realized by the CPU 901 acquiring and executing the program 904 that realizes those functions.
  • the program 904 is stored in the storage device 905 or the ROM 902 in advance, for example, and the CPU 901 loads the program 904 into the RAM 903 and executes the program 904 as needed.
  • the program 904 may be supplied to the CPU 901 via the communication network 909, or may be stored in the storage medium 906 in advance, and the drive device 907 may read the program and supply the program to the CPU 901.
  • FIG. 16 is a flowchart showing an example of the abnormality determination process executed by the abnormality determination device 11.
  • the abnormality determination device 11 acquires a time-series image in which the surface Qa of the object to be measured 3 is photographed from the photographing device 10 (S1). After that, the displacement calculation unit 111 calculates the displacement of the object 3 to be measured on the surface Qa using the set of the m (m> 1) th image and the m + 1st frame image included in the time series image (S2). ).
  • the out-of-plane displacement calculation unit 112 calculates the out-of-plane displacement of the object 3 to be measured on the surface Qa by using the displacement obtained by the displacement calculation unit 111 (S3). Further, the in-plane displacement calculation unit 113 calculates the in-plane displacement by subtracting the out-of-plane displacement by the out-of-plane displacement calculation unit 112 from the displacement by the displacement calculation unit 111 (S4).
  • the displacement calculation unit 111 determines whether or not the out-of-plane displacement and the in-plane displacement have been calculated for the predetermined n (> 1) frame images included in the time series image (S5).
  • the process returns to step S2, and the displacement calculation unit 111 returns to the next frame image included in the time-series image.
  • the displacement of the object 3 to be measured on the surface Qa is calculated by using the set of m + 1, that is, the frame images of the first m + 1 and the second m + 2.
  • step S5 when the displacement calculation unit 111 determines that the calculation processing of the out-of-plane displacement and the in-plane displacement has been completed for the n frame images (Yes in S5), the determination unit 114 performs the determination processing. Run. That is, the determination unit 114 analyzes the calculated out-of-plane displacement and in-plane displacement (S6), and uses the analysis result to determine the abnormality of the object to be measured 3 (S7). After the abnormality determination, the abnormality determination device 11 outputs the determination result to the notification device 12.
  • the notification of the notification device 12 allows the user to determine, for example, the necessity of repairing or detailed investigation of the object 3 to be measured.
  • the abnormality determination device 11 executes the abnormality determination process.
  • the abnormality determination system 1 of the first embodiment has a photographing device 10 including an optical path bending member 13.
  • the optical path bending member 13 has a function of bending the traveling direction of light in a direction closer to the optical axis of the lens 102.
  • the length of the portion where the out-of-plane displacement of the surface Qa of the object to be measured 3 photographed by the photographing device 10 is projected is the length when the optical path bending member 13 is not provided. Compared, it can be made longer. That is, the abnormality determination system 1 in the first embodiment can improve the measurement resolution of the out-of-plane displacement of the surface Qa of the object to be measured 3.
  • the angle of view is adjusted by the lens 102 in order to improve the measurement resolution of the in-plane displacement of the surface Qa of the object 3 to be measured.
  • the optical path bending member 13 is not provided, the measurement resolution of the out-of-plane displacement of the surface Qa of the object to be measured 3 is reduced, but the photographing device 10 is provided with the optical path bending member 13. This prevents a decrease in the measurement resolution of the out-of-plane displacement. Further, in that state, the measurement resolution of the out-of-plane displacement can be improved. That is, the abnormality determination system 1 can easily improve the measurement resolution of the in-plane displacement and the out-of-plane displacement of the surface Qa of the object to be measured 3.
  • the abnormality determination system 1 of the first embodiment utilizes the in-plane displacement and the out-of-plane displacement of the surface Qa of the object to be measured 3 calculated from the time-series images taken by the photographing device 10, and the object to be measured 3 is measured. It is possible to improve the accuracy of detecting the abnormality of.
  • the optical path bending member 13 can be retrofitted to the photographing device 10 by providing a structure that can be attached to the outside of the lens of the photographing device 10.
  • the optical path bending member 13 can be attached to the photographing device 10 constituting the existing abnormality determination system, and the abnormality determination system can be inexpensive and improve the detection accuracy without performing a large-scale work.
  • the version can be upgraded.
  • the optical path bending member 13 is a member that is attached to the photographing device 10 and integrated with the photographing device 10.
  • the abnormality determination system 1 may use the optical path bending member 14 separated from the photographing device 10.
  • the optical path bending member 14 is interposed in the optical path from the object to be measured 3 to the photographing device 10.
  • the optical path bending member 13 may be built in the photographing device 10. In this case, the optical path bending member 13 is provided in the optical path through which the light entering the photographing device 10 reaches the lens 102.
  • FIG. 18 is a block diagram showing a configuration of an embodiment of the abnormality determination system according to the present invention.
  • the abnormality determination system 20 of the second embodiment includes an abnormality determination device 21, an imaging device 22, and an optical path bending member 23.
  • the photographing device 22 outputs a time-series image including a plurality of captured images in which the surface of the object to be measured is photographed with the passage of time.
  • the optical path bending member 23 is interposed in the optical path portion between the object to be measured and the lens 24 in the optical path from the object to be measured to the imaging surface 25 through the lens 24 provided in the photographing device 22.
  • the optical path bending member 23 bends the traveling direction of the light so as to tilt the traveling direction of the light from the lens 24 to the imaging surface 25 in a direction closer to the optical axis of the lens 24.
  • the optical path bending member 23 is configured by using, for example, a prism or a mirror as described above.
  • the abnormality determination device 21 determines an abnormality of the object to be measured using the time-series image output from the photographing device 22. That is, the displacement of the surface of the object to be measured is measured from the time series image. Using the displacement of the surface of the object to be measured, the out-of-plane displacement, which is the displacement in the normal direction on the surface of the object to be measured, is calculated. Further, by subtracting the out-of-plane displacement from the displacement of the surface of the object to be measured, the in-plane displacement, which is the displacement on the surface of the object to be measured, is calculated. The abnormality determination device 21 determines an abnormality of the object to be measured by utilizing the out-of-plane displacement and the in-plane displacement as described above. Such an abnormality determination device 21 is composed of, for example, a computer device like the above-mentioned abnormality determination device 11.
  • the abnormality determination system 20 of the second embodiment also includes an optical path bending member 23 similar to the optical path bending member 13 of the abnormality determination system 1 of the first embodiment. As a result, the abnormality determination system 20 can obtain the effect that the measurement resolutions of both the out-of-plane displacement and the in-plane displacement can be easily improved.
  • the optical path bending member 23 may form a photographing device 22 together with the lens 24 and the imaging surface 25.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Optics & Photonics (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Length Measuring Devices By Optical Means (AREA)
PCT/JP2020/009914 2020-03-09 2020-03-09 異常判定システム、撮影装置および異常判定方法 Ceased WO2021181441A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2022506988A JPWO2021181441A1 (https=) 2020-03-09 2020-03-09
US17/908,019 US20230111602A1 (en) 2020-03-09 2020-03-09 Abnormality determination system, imaging device and abnormality determination method
PCT/JP2020/009914 WO2021181441A1 (ja) 2020-03-09 2020-03-09 異常判定システム、撮影装置および異常判定方法

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/009914 WO2021181441A1 (ja) 2020-03-09 2020-03-09 異常判定システム、撮影装置および異常判定方法

Publications (1)

Publication Number Publication Date
WO2021181441A1 true WO2021181441A1 (ja) 2021-09-16

Family

ID=77670482

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/009914 Ceased WO2021181441A1 (ja) 2020-03-09 2020-03-09 異常判定システム、撮影装置および異常判定方法

Country Status (3)

Country Link
US (1) US20230111602A1 (https=)
JP (1) JPWO2021181441A1 (https=)
WO (1) WO2021181441A1 (https=)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002218295A (ja) * 2001-01-18 2002-08-02 Olympus Optical Co Ltd 撮像装置
JP2004205652A (ja) * 2002-12-24 2004-07-22 Nikon Corp ワイドコンバーターレンズ
JP2006064975A (ja) * 2004-08-26 2006-03-09 Olympus Corp 顕微鏡および薄板エッジ検査装置
JP2007171149A (ja) * 2005-12-21 2007-07-05 Nippon Electro Sensari Device Kk 表面欠陥検査装置
JP2012078342A (ja) * 2010-10-05 2012-04-19 Mitsutoyo Corp 画像相関変位センサ
WO2016152076A1 (ja) * 2015-03-20 2016-09-29 日本電気株式会社 構造物の状態判定装置と状態判定システムおよび状態判定方法
WO2019235409A1 (ja) * 2018-06-05 2019-12-12 日本電気株式会社 変位量測定装置、変位量測定方法、および記録媒体

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102007037726B4 (de) * 2007-08-09 2010-07-08 Lavision Gmbh Verfahren zur berührungslosen Messung von Verformungen einer Oberfläche eines Messobjektes
AU2016308995B2 (en) * 2015-08-19 2019-02-14 National Institute Of Advanced Industrial Science And Technology Method, device, and program for measuring displacement and vibration of object by single camera

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002218295A (ja) * 2001-01-18 2002-08-02 Olympus Optical Co Ltd 撮像装置
JP2004205652A (ja) * 2002-12-24 2004-07-22 Nikon Corp ワイドコンバーターレンズ
JP2006064975A (ja) * 2004-08-26 2006-03-09 Olympus Corp 顕微鏡および薄板エッジ検査装置
JP2007171149A (ja) * 2005-12-21 2007-07-05 Nippon Electro Sensari Device Kk 表面欠陥検査装置
JP2012078342A (ja) * 2010-10-05 2012-04-19 Mitsutoyo Corp 画像相関変位センサ
WO2016152076A1 (ja) * 2015-03-20 2016-09-29 日本電気株式会社 構造物の状態判定装置と状態判定システムおよび状態判定方法
WO2019235409A1 (ja) * 2018-06-05 2019-12-12 日本電気株式会社 変位量測定装置、変位量測定方法、および記録媒体

Also Published As

Publication number Publication date
US20230111602A1 (en) 2023-04-13
JPWO2021181441A1 (https=) 2021-09-16

Similar Documents

Publication Publication Date Title
JP6767642B2 (ja) 測定装置および測定方法
US11928805B2 (en) Information processing apparatus, information processing method, and storage medium for defect inspection and detection
JP6954368B2 (ja) 変位成分検出装置、変位成分検出方法、及びプログラム
WO2016152076A1 (ja) 構造物の状態判定装置と状態判定システムおよび状態判定方法
JPWO2016152075A1 (ja) 構造物の状態判定装置と状態判定システムおよび状態判定方法
JP2022040159A (ja) ３次元歪測定のためのフィールドスプリッタ
WO2017179535A1 (ja) 構造物の状態判定装置と状態判定システムおよび状態判定方法
JP2008039767A (ja) 反射性物体の表面の形状を検知する方法及びシステム
JPWO2018043251A1 (ja) 欠陥検出装置、欠陥検出方法、及びコンピュータ読み取り可能記録媒体
JP6813025B2 (ja) 状態判定装置、状態判定方法、及びプログラム
US11846498B2 (en) Displacement amount measuring device, displacement amount measuring method, and recording medium
Srimontriphakdi et al. A non-contact approach for cable tension evaluation based on 3D laser scanning data and nonlinear finite element analysis
CN110111390A (zh) 基于双目视觉光流跟踪的薄壁件全向振动测量方法及系统
JP6954367B2 (ja) 計測システム、補正処理装置、補正処理方法、及びプログラム
JP6996569B2 (ja) 計測システム、補正処理装置、補正処理方法、及びプログラム
JP2016176806A (ja) 構造物の状態判定装置と状態判定システムおよび状態判定方法
Kim et al. Dynamic characteristics of multi-degree-of-freedom frame systems derived from vision-based displacement measurement using consumer-grade camera video
WO2021181441A1 (ja) 異常判定システム、撮影装置および異常判定方法
Besharatian et al. Benchmarking dynamic properties of structures using non-contact sensing
US10091493B2 (en) Device and method for scanning object outline image
JP7248149B2 (ja) 異常判定装置、異常判定方法およびコンピュータプログラム
JP7156529B2 (ja) 変位測定装置、変位測定方法、およびプログラム
JP7485155B2 (ja) データ符号化プログラム、及びデータ符号化装置
WO2023013058A1 (ja) 状態判定装置、状態判定方法、及び、コンピュータ読み取り可能な記録媒体
De Nunzio et al. Crack localization on a statically deflected beam by high-resolution photos

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20924789

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022506988

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20924789

Country of ref document: EP

Kind code of ref document: A1