JP2021025845A - Displacement distribution measuring device, displacement distribution measuring method, and control program of displacement distribution measuring device - Google Patents

Abstract

To calculate displacement distribution on a surface of an object even when a part of a pattern such as a grid in a captured image is missing.SOLUTION: A displacement distribution measuring device 200 that measures displacement distribution on a surface of a test object TP based on an image of a pattern PTN formed on the surface of the test object TP includes: an imaging unit 811 that generates a pattern image PN indicating the image of the pattern PTN; a complementing unit 812 that complements the pattern image PN corresponding to a missing area HL to generate a complementary image PC when the pattern image PN includes the missing area HL where a part of the pattern image PN is missing; and a calculation unit 813 that calculates the displacement distribution on the surface of the test object TP based on the complementary image PC.SELECTED DRAWING: Figure 3

Description

The present invention relates to a displacement distribution measuring device, a displacement distribution measuring method, and a control program of the displacement distribution measuring device.

Various techniques for measuring the displacement of the surface of an object are known.
For example, in Patent Document 1, a displacement measuring device detects a region of a grid from a photographing unit that captures a displacement measuring image including a displacement measuring grid provided at a predetermined position and a displacement measuring image. Obtained from the phase distribution, the lattice region detection unit that extracts the image of the lattice region, the phase distribution derivation unit that derives the phase distribution of the displacement with respect to the image of the lattice region by the sampling displacement method for the extracted image of the lattice region. A technique is disclosed that includes a phase difference before and after a displacement at a predetermined position, and a displacement determining unit that determines a displacement at a predetermined position from a predetermined lattice pitch.

Japanese Unexamined Patent Publication No. 2011-174874

However, in the displacement measuring device described in Patent Document 1, the phase distribution of moire with respect to the image of the lattice region is derived, and the phase difference before and after the displacement at a predetermined position obtained from the phase distribution and the predetermined lattice pitch are used. To determine the displacement at a given position. Therefore, if a part of the grid region corresponding to the grid pattern in the captured image is missing, it may not be possible to derive the phase distribution of the moire, and the displacement at a predetermined position can be determined. It may not be possible. Examples of causes for a part of the grid region to be missing include failure to transfer the grid pattern to the measurement target, measurement of a target having a structural defect such as a hole, and a camera setting error.

The present invention has been made in view of such circumstances, and is a displacement distribution measuring device capable of calculating the displacement distribution on the surface of an object even when a part of a pattern such as a grid in a captured image is missing. , A displacement distribution measuring method, and a control program of a displacement distribution measuring device.

A first aspect of the present invention is a displacement distribution measuring device that measures a displacement distribution on the surface of an object based on an image of a pattern formed on the surface of the object, wherein a pattern image showing the image of the pattern is obtained. An imaging unit to be generated, a complementary unit that complements the pattern image corresponding to the missing region to generate a complementary image when the pattern image includes a missing region in which a part of the pattern image is missing, and the complement. The present invention relates to a displacement distribution measuring device including a calculation unit for calculating a displacement distribution on the surface of the object based on an image.

A second aspect of the present invention is a displacement distribution measuring method for measuring the displacement distribution on the surface of the object based on the image of the pattern formed on the surface of the object, wherein the pattern image showing the image of the pattern is obtained. An imaging step to be generated, a complement step to complement the pattern image corresponding to the missing region to generate a complementary image when the pattern image includes a missing region in which a part of the pattern image is missing, and the complement The present invention relates to a displacement distribution measuring method including a calculation step of calculating a displacement distribution on the surface of the object based on an image.

A third aspect of the present invention is a control program of a displacement distribution measuring device including a computer and measuring a displacement distribution on the surface of the object based on an image of a pattern formed on the surface of the object. An imaging unit that generates a pattern image showing an image of the pattern, and when the pattern image includes a missing region in which a part of the pattern image is missing, the pattern image corresponding to the missing region is complemented and complemented. The present invention relates to a control program of a displacement distribution measuring device that functions as a complementary unit that generates an image and a calculation unit that calculates the displacement distribution of the surface of the object based on the complementary image.

According to the first aspect of the present invention, when the pattern image includes a missing region in which a part of the pattern image is missing, the complementary unit complements the pattern image corresponding to the missing region to obtain the complementary image. The generation unit calculates the displacement distribution on the surface of the object based on the complementary image.
Therefore, the displacement distribution on the surface of the object can be calculated even when a part of the pattern image is missing.

According to the second aspect of the present invention, when the pattern image includes a missing region in which a part of the pattern image is missing, the complementary image is complemented with the pattern image corresponding to the missing region in the complement step. In the generation and calculation step, the displacement distribution on the surface of the object is calculated based on the complementary image.
Therefore, the displacement distribution on the surface of the object can be calculated even when a part of the pattern image is missing.

According to the third aspect of the present invention, when the pattern image includes a missing region in which a part of the pattern image is missing, the complementary unit complements the pattern image corresponding to the missing region to obtain the complementary image. The generation unit calculates the displacement distribution on the surface of the object based on the complementary image.
Therefore, the displacement distribution on the surface of the object can be calculated even when a part of the pattern image is missing.

It is a figure which shows an example of the structure of the tensile tester which concerns on embodiment of this invention. It is a top view which shows an example of the structure of the displacement distribution measuring apparatus which concerns on embodiment of this invention. It is a figure which shows an example of each structure of the 1st control part and 2nd control part which concerns on 1st Embodiment of this invention. It is a figure which shows an example of the pattern image which concerns on 1st Embodiment of this invention. It is a figure which shows an example of the displacement distribution calculation processing of the calculation part which concerns on embodiment of this invention. It is a figure which shows an example of the pattern image including the defect region which concerns on embodiment of this invention. It is a figure which shows an example of the problem at the time of executing the displacement distribution calculation process which concerns on embodiment of this invention with respect to the pattern image including the defect region. It is a figure which shows an example of the training data which generates the trained model which concerns on embodiment of this invention. It is a flowchart which shows an example of the processing of the 2nd control part which concerns on embodiment of this invention. It is a figure which shows the 1st example of the process of the complementary part which concerns on 1st Embodiment of this invention. It is a figure which shows the 2nd example of the process of the complementary part which concerns on 1st Embodiment of this invention. It is a flowchart which shows an example of the processing of the 1st control part in 2nd example. It is a figure which shows the 3rd example of the process of the complementary part which concerns on 1st Embodiment of this invention. It is a flowchart which shows an example of the processing of the 1st control part in 3rd example. It is a figure which shows an example of the 2nd Embodiment of a complementary part.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[1. Common configuration]
[1-1. Tensile tester configuration]
FIG. 1 is a diagram showing an example of the configuration of the tensile tester 1 according to the embodiment of the present invention.
The tensile tester 1 according to the embodiment of the present invention applies a test force F to the TP to be tested and conducts a material test for measuring mechanical properties such as tensile strength, yield point, elongation, and drawing of a sample. The test force F is a tensile force.
The tensile tester 1 includes a tensile tester main body 2 that applies a test force F to a test target TP, which is a material to be tested, to perform a tensile test, a control unit 4 that controls a tensile test operation by the tensile tester main body 2. To be equipped.
The tensile tester 1 corresponds to an example of a “material tester”. The test target TP corresponds to an example of an “object”.

The testing machine main body 2 can move along the table 26, a pair of screw rods 28 and 29 rotatably erected on the table 26 in a vertically oriented state, and these screw rods 28 and 29. A crosshead 10, a load mechanism 12 for moving the crosshead 10 to apply a load to the TP to be tested, and a load cell 14 are provided. The load cell 14 is a sensor that measures a test force F, which is a tensile load applied to the test target TP, and outputs a test force measurement signal SG1.

The load mechanism 12 includes worm reducers 16 and 17 connected to the lower ends of the screw paddles 28 and 29, a servomotor 18 connected to the worm reducers 16 and 17, and a rotary encoder 20. The rotary encoder 20 is a sensor that measures the amount of rotation of the servomotor 18 and outputs a rotation measurement signal SG2 having a number of pulses corresponding to the amount of rotation to the control unit 4.
The load mechanism 12 transmits the rotation of the servomotor 18 to the pair of screw paddles 28 and 29 via the worm reducers 16 and 17, and the screw paddles 28 and 29 rotate in synchronization with each other to rotate the cross head. 10 moves up and down along the screw paddles 28 and 29.

The crosshead 10 is provided with an upper grip 21 for gripping the upper end of the test target TP, and the table 26 is provided with a lower grip 22 for gripping the lower end of the test target TP. .. At the time of the tensile test, the testing machine main body 2 raises the crosshead 10 under the control of the control unit 4 in a state where both ends of the TP to be tested are gripped by the upper gripping tool 21 and the lower gripping tool 22. , The test force F is given to the test target TP.

A displacement sensor 15 is arranged on the test target TP. As the test target TP, for example, a dumbbell type test target formed with a constricted center is used. The displacement sensor 15 is a sensor that measures the elongation measurement value ED by measuring the distance between a pair of reference points of the test target TP and outputs the elongation measurement signal SG3. A pair of gauge points are placed at the top and bottom of the constricted area of the TP under test.

The control unit 4 includes a general control device 30, a display device 32, and a test program execution device 34.
The integrated control device 30 is a device that centrally controls the testing machine main body 2, and is connected to the testing machine main body 2 so as to be able to transmit and receive signals. The signals received from the testing machine main body 2 are the test force measurement signal SG1 output from the load cell 14, the rotation measurement signal SG2 output from the rotary encoder 20, the elongation measurement signal SG3 output from the displacement sensor 15, and appropriate measures required for control and testing. Signal etc.
The display device 32 is a device that displays various information based on the signal input from the control control device 30, and for example, the control device 30 is a device of the test target TP based on the elongation measurement signal SG3 during the tensile test. The elongation measurement value ED, which is the elongation measurement value, is displayed on the display device 32. Further, for example, the integrated control device 30 displays the displacement measurement value XD indicating the displacement of the crosshead 10 based on the rotation measurement signal SG2 on the display device 32 during the tensile test.

The tensile test program execution device 34 receives user operations such as setting operations of various setting parameters such as test conditions for tensile tests and execution instruction operations, and outputs the functions to the integrated control device 30 and analyzes the data of the test force measurement value FD. It is a device equipped with a function to perform.
The tensile test program execution device 34 according to the embodiment of the present invention includes a computer, which includes a processor such as a CPU (Central Processing Unit) and an MPU (Micro-Processing Unit), and a ROM (Read Only Memory) and a RAM (Random). It includes a memory device such as an Access Memory), a storage device such as an HDD (Hard Disk Drive) and an SSD (Solid State Drive), and an interface circuit for connecting a central processing unit 30 and various peripheral devices. Then, the processor executes a tensile test program, which is a computer program stored in a memory device or a storage device, to realize the above-mentioned various functions.

Next, the integrated control device 30 according to the embodiment of the present invention will be described in more detail. The integrated control device 30 includes a signal input / output unit 40 and a control circuit unit 50.
The signal input / output unit 40 constitutes an input / output interface circuit for transmitting and receiving signals to and from the testing machine main body 2, and in the embodiment of the present invention, the first sensor amplifier 42 and the second sensor amplifier 45 The counter circuit 43 and the servo amplifier 44 are provided.
The first sensor amplifier 42 is an amplifier that amplifies the test force measurement signal SG1 output from the load cell 14 and outputs it to the control circuit unit 50. The second sensor amplifier 45 is an amplifier that amplifies the elongation measurement signal SG3 output by the displacement sensor 15 and outputs it to the control circuit unit 50.
The counter circuit 43 counts the number of pulses of the rotation measurement signal SG2 output by the rotary encoder 20, and indicates the amount of rotation of the servomotor 18, that is, the displacement measurement value XD of the crosshead 10 that moves up and down due to the rotation of the servomotor 18. The signal A3 is output to the control circuit unit 50 as a digital signal. The servo amplifier 44 is a device that controls the servomotor 18 according to the control of the control circuit unit 50.

The control circuit unit 50 includes a communication unit 51 and a feedback control unit 52.
The control circuit unit 50 includes a processor such as a CPU and MPU, a memory device such as a ROM and a RAM, a storage device such as an HDD and an SSD, an interface circuit between the signal input / output unit 40, and a tensile test program execution device 34. A computer including a communication device for communication, a display control circuit for controlling the display device 32, and various electronic circuits is provided. Further, each functional unit shown in FIG. 1 is realized by the processor of the control circuit unit 50 executing the control program stored in the memory device or the storage device.
Further, an A / D converter is provided in the interface circuit of the signal input / output unit 40, and the test force measurement signal SG1 and the elongation measurement signal SG3 of the analog signal are converted into digital signals by the A / D converter.
The control circuit unit 50 is not limited to a computer, and may be composed of one or a plurality of appropriate circuits such as integrated circuits such as IC chips and LSIs.

The communication unit 51 communicates with the test program execution device 34, and receives test condition settings, various setting parameter setting values, tensile test execution instructions, interruption instructions, and the like from the test program execution device 34. Further, the communication unit 51 transmits the elongation measurement value ED based on the elongation measurement signal SG3 and the test force measurement value FD based on the test force measurement signal SG1 to the test program execution device 34 at appropriate timings. Further, the communication unit 51 transmits the displacement measurement value XD based on the rotation measurement signal SG2 to the test program execution device 34 at an appropriate timing.

The feedback control unit 52 feedback-controls the servomotor 18 of the testing machine main body 2 to execute the tensile test. The feedback control unit 52 is a circuit that executes feedback control of the servomotor 18.
When the feedback control unit 52 executes the position control, the feedback control unit 52 executes the position control for, for example, the test force measurement value FD output by the load cell 14. In this case, the feedback control unit 52 calculates the command value dX of the displacement measurement value XD so that the test force measurement value FD matches the test force target value FT, and servos the command signal A4 indicating the command value dX. Output to amplifier 44. The test force target value FT indicates a target value of the test force measurement value FD.
In addition, "position control" means that the detected value measured by a sensor or the like is controlled so as to match the target value.

Although the case of executing the position control will be described, the feedback control unit 52 may execute the speed control. "Speed control" means controlling the amount of change of the detected value measured by a sensor or the like per unit time so as to match the target value.

[1-2. Configuration of displacement distribution measuring device]
FIG. 2 is a plan view showing an example of the configuration of the displacement distribution measuring device 200 according to the embodiment of the present invention. As shown in FIG. 2, the displacement distribution measuring device 200 includes a camera 6, a first control unit 8, and a second control unit 7.

A pattern PTN is formed on the specific surface TP1 of the TP to be tested. The specific surface TP1 indicates a surface of the TP to be tested on the side close to the camera 6. The side of the test target TP close to the camera 6 is shown on the lower side in FIG. The pattern PTN will be described later with reference to FIG.

The camera 6 generates a pattern image PN showing an image of the pattern PTN according to the instruction of the first control unit 8. The camera 6 includes an image sensor such as a CCD (Charge Coupled Device) and a CMOS (Complementary MOS). The pattern image PN will be described later with reference to FIG.

The camera 6 is arranged so that the photographing direction C6 of the camera 6 passes through the center in the width direction of the test target TP and is orthogonal to the specific surface TP1 of the test target TP. The width direction of the TP to be tested shows the left-right direction in FIG. The shooting direction C6 indicates the center of the shooting range of the camera 6.

The first control unit 8 controls the operation of the displacement distribution measuring device 200. Further, the first control unit 8 is configured to be able to communicate with the control circuit unit 50, and controls the operation of the camera 6 according to an instruction from the control circuit unit 50. Specifically, the first control unit 8 determines the shooting timing of the camera 6 according to the instruction from the control circuit unit 50. The configuration of the first control unit 8 will be described in detail later with reference to FIG.

The second control unit 7 generates a trained model MD, and transmits the generated trained model MD to the first control unit 8. The trained model MD will be described in detail later with reference to FIG.

[1-3. Configuration of 1st control unit and 2nd control unit]
FIG. 3 is a diagram showing an example of the configuration of each of the first control unit 8 and the second control unit 7 according to the embodiment of the present invention.
As shown in FIG. 3, the first control unit 8 includes an imaging unit 811, a complementary unit 812, a calculation unit 813, and a trained model storage unit 814.
Further, the first control unit 8 includes a first processor 81A such as a CPU or MPU, a first memory device 81B such as a ROM or RAM, a first storage device 81C such as an HDD or SSD, a control circuit unit 50, and a first control circuit unit 8. 2 It is composed of a personal computer including a first communication device 81D that communicates with each of the control units 7 and various electronic circuits. Further, each functional unit shown in FIG. 3 is realized by the first processor 81A of the first control unit 8 executing the first control program stored in the first memory device 81B or the first storage device 81C. The first processor 81A corresponds to an example of a "computer". The first control program corresponds to an example of a “control program”.

The first control unit 8 is not limited to a personal computer, and may be configured by one or a plurality of appropriate circuits such as integrated circuits such as IC chips and LSIs. Further, the first control unit 8 may be configured as, for example, a tablet terminal or a smartphone.
Further, the first control unit 8 may include programmed hardware such as a DSP (Digital Signal Processor) or an FPGA (Field Programmable Gate Array). Further, the first control unit 8 may include a SoC (System-on-a-Chip) -FPGA.

The image pickup unit 811 causes the camera 6 to image the pattern PTN to generate a pattern image PN showing the image of the pattern PTN. Specifically, the imaging unit 811 generates the pattern image PN according to the instruction from the control circuit unit 50.
Further, the imaging unit 811 generates a pattern image PN at predetermined time intervals. The predetermined time is, for example, 0.01 seconds. Specifically, the imaging unit 811 generates a pattern image PN at predetermined time intervals from the start of the tensile test execution by the tensile tester 1 to the end of the tensile test execution.

[First Embodiment of Complementary Unit 812]
When the pattern image PN includes a missing area HL in which a part of the pattern image PN is missing, the complementing unit 812 according to the first embodiment complements the pattern image PN corresponding to the missing area HL to provide an overall complementary image PC. Generate. The whole complementary image PC shows the whole image of the pattern image PN that complements the pattern image PN corresponding to the missing area HL. The whole complementary image PC corresponds to an example of "complementary image".
Further, the complement unit 812 is generated based on the learning data DL in which the information of the pattern image PN including the missing region HL and the information of the entire complementary image PC complemented with the pattern image PN corresponding to the missing region HL are associated with each other. It is equipped with a trained model MD.
Further, the complement unit 812 inputs the information of the pattern image PN including the missing region HL into the learned model MD, and outputs the information of the missing complementary image PCH corresponding to the missing region HL from the trained model MD. The missing complementary image PCH indicates a pattern image PN corresponding to the missing region HL.
Further, the complement unit 812 generates an overall complementary image PC by synthesizing the pattern image PN including the missing region HL and the missing complementary image PCH corresponding to the missing region HL.
Further, the complementary unit 812 generates the entire complementary image PC every time the imaging unit 811 generates the pattern image PN.
The complementary unit 812 according to the first embodiment will be described in detail later with reference to FIGS. 10 to 14.

The calculation unit 813 calculates the displacement distribution on the surface of the test target TP based on the overall complementary image PC. The calculation unit 813 will be described in detail later with reference to FIGS. 4 to 5.

The trained model storage unit 814 stores the trained model MD generated by the second control unit 7.

As shown in FIG. 3, the second control unit 7 includes a determination unit 711, a generation unit 712, a learning unit 713, a learning data storage unit 714, and a learned model storage unit 715.
Further, the second control unit 7 includes a second processor 71A such as a CPU and an MPU, a second memory device 71B such as a ROM and a RAM, a second storage device 71C such as an HDD and an SSD, and a first control unit 8. It is composed of a personal computer including a second communication device 71D for communication and various electronic circuits. Further, each functional unit shown in FIG. 3 is realized by the second processor 71A of the second control unit 7 executing the second control program stored in the second memory device 71B or the second storage device 71C.

The second control unit 7 is not limited to a personal computer, and may be configured by one or a plurality of appropriate circuits such as integrated circuits such as IC chips and LSIs. Further, the second control unit 7 may be configured as, for example, a tablet terminal or a smartphone.
Further, the second control unit 7 may include programmed hardware such as a DSP (Digital Signal Processor) or an FPGA (Field Programmable Gate Array). Further, the second control unit 7 may include a SoC (System-on-a-Chip) -FPGA.

The determination unit 711 randomly determines the position PS and the size SZ of the missing region HL. The position PS of the missing area HL indicates the position of the missing area HL in the pattern image PN. The maximum value of the size SZ is, for example, 1/10 of the area of the pattern image PN. The minimum value of the size SZ is, for example, 1/1000 of the area of the pattern image PN. That is, the determination unit 711 randomly determines the size SZ from 1/1000 of the area of the pattern image PN to 1/10 of the area of the pattern image PN.

The generation unit 712 generates the pattern image PN including the missing region HL constituting the learning data DL by cutting out the image corresponding to the missing region HL from the pattern image PN not including the missing region HL.
Further, the generation unit 712 generates a pattern image PN including the missing region HL of the predetermined shape of the position PS and the size SZ determined by the determination unit 711. In the present embodiment, the predetermined shape includes a rectangular shape.

The learning unit 713 performs machine learning based on the learning data DL in which the information of the pattern image PN including the missing region HL generated by the generation unit 712 and the information of the overall complementary image PC complemented by the pattern image PN are associated with each other. Execute and generate a trained model MD.
In the embodiment of the present invention, the overall complementary image PC corresponds to the pattern image PN that does not include the missing region HL. That is, the learning unit 713 executes machine learning based on the learning data DL in which the information of the pattern image PN including the missing area HL and the information of the pattern image PN not including the missing area HL are associated with each other.
Specifically, machine learning was performed by deep learning using pix2pix. "Pix2pix" is an algorithm called "Image-to-Image Transition Wizard" published by Berkeley AI Research (BAIR) Laboratory, UC Berkeley's Philip Isola, etc. in 2014. The feature of pix2pix is that image processing is performed using a method called GAN (Generative Adversarial Network).
This paper introduces various image processing such as complementation of missing images using pix2pix.
The number of learning data DLs was, for example, 60.

The learning data storage unit 714 stores the learning data DL.

The trained model storage unit 715 stores the trained model MD.

[1-4. Displacement distribution calculation method by the calculation unit]
Next, a method of calculating the displacement distribution on the surface of the test target TP by the calculation unit 813 will be described with reference to FIGS. 4 and 5.
FIG. 4 is a diagram showing an example of a pattern image PN according to an embodiment of the present invention. The pattern image PN1 on the left side of FIG. 4 shows an example of the pattern PTN of the test target TP in the initial state. The pattern PTN indicates a grid-like pattern PTN. That is, in the pattern image PN1, square black-painted images Qij (i = 1 to n, j = 1 to 8) are arranged at equal intervals in the left-right direction and the up-down direction.
The size of the black-painted image Qij is, for example, a square with a side of 1 mm. The black-painted image Qij is arranged in the horizontal direction and the vertical direction, for example, at intervals of 1 mm.

In the embodiment of the present invention, the size of the black-painted image Qij is, for example, a square having a side of 1 mm, but a square having a side larger than 1 mm, for example, a square having a side of 1.2 mm and having a side of 1. It may be a 5 mm square, a square having a side of 2 mm, or the like. The size of the black-painted image Qij may be a square having a side smaller than 1 mm, for example, a square having a side of 0.8 mm, a square having a side of 0.5 mm, a square having a side of 0.3 mm, or the like.

Further, in the embodiment of the present invention, the black-painted image Qij is arranged in the horizontal direction and the vertical direction at an interval of, for example, 1 mm, but the black-painted image Qij is arranged in the horizontal direction and the vertical direction at an interval larger than 1 mm. May be done. The black-painted image Qij may be arranged in the horizontal direction and the vertical direction, for example, at 1.2 mm intervals, 1.5 mm intervals, 2 mm intervals, and the like. Further, the black-painted image Qij may be arranged in the left-right direction and the up-down direction at intervals smaller than 1 mm. The black-painted image Qij may be arranged in the horizontal direction and the vertical direction, for example, at intervals of 0.8 mm, 0.5 mm, 0.3 mm, and the like.

The pattern image PN2 on the right side of FIG. 4 shows an example of the pattern PTN in a state where the test target TP is extended in the vertical direction. That is, in the pattern image PN2, square black-painted images Rij (i = 1 to n, j = 1 to 8) are arranged at substantially equal intervals in the horizontal direction and the vertical direction.

FIG. 5 is a diagram showing an example of displacement distribution calculation processing of the calculation unit 813 according to the embodiment of the present invention. In FIG. 5, the calculation unit 813 calculates, for example, the amount of displacement in the X-axis direction. The reference point of the displacement amount is the center of the black-painted image Q14 or the black-painted image R14 located at the uppermost end. That is, by comparing the distance between the black-painted image Ri4 (i = 2 to n) and the black-painted image R14 and the distance between the black-painted image Qi4 (i = 2 to n) and the black-painted image Q14, X Calculate the amount of displacement in the axial direction.

The graph G11 at the top of FIG. 5 is a graph obtained by sine-wave approximation of the change in luminance B in the X-axis direction. The horizontal axis of the graph G11 is the X-axis, and the vertical axis is the luminance B. The brightness B of the black-painted image Ri4 is low, and the brightness B is high between the black-painted image Ri4 and the adjacent black-painted image R (i + 1) 4. Therefore, the change in the luminance B in the X-axis direction is represented by a rectangular graph shown in the figure.
The graph G11 is a graph obtained by approximating a rectangular graph with a sine wave between the black-painted image Ri4 and the adjacent black-painted image R (i + 1) 4. When the pattern image PN is the pattern image PN1, the black-painted images Qi4 in the pattern image PN1 are formed at equal intervals, so that the graph G11 becomes a sine wave.

The second graph G12 from the top of FIG. 5 is a graph showing the phase of the graph G11. The horizontal axis of the graph G12 is the X-axis, and the vertical axis is the phase Q. The phase Q varies from −π to + π. Therefore, the graph G12 becomes a saw blade-shaped graph. That is, in the graph G12, the phase increases linearly from −π to + π as the X coordinate increases, and the phase decreases stepwise to −π at the position where the phase reaches + π.

The lowermost graph G13 of FIG. 5 shows a graph after making a phase connection with respect to the second graph G12 from the top of FIG. The horizontal axis of the graph G13 is the X-axis, and the vertical axis is the phase φ. In the graph G12, the phase decreases stepwise to −π at the position where the phase reaches + π, but in the graph G13, every time the phase decreases from + π to −π so as not to change the phase stepwise. , Increase the phase by 2 × π. As a result, in the graph G13, the phase φ linearly increases as the X coordinate increases.

The graph G14 corresponds to the pattern image PN2. The slope of the graph G14 is smaller than that of the graph G13. That is, in the graph G13, the difference in the coordinates of the X-axis for changing the phase φ by 2 × π is the distance L1, whereas in the graph G14, the X-axis for changing the phase φ by 2 × π. The difference between the coordinates of is the distance L2. The distance L2 is larger than the distance L1. The difference between the distance L2 and the distance L1 indicates the displacement ΔL.
In this way, the calculation unit 813 can calculate the displacement distribution on the surface of the TP to be tested.

[1-5. Challenges when including defective areas]
FIG. 6 is a diagram showing an example of the pattern image PN3 including the missing region HL according to the embodiment of the present invention. The pattern image PN3 shown in FIG. 6 shows an example of the pattern PTN in a state where the test target TP is extended in the vertical direction. That is, in the pattern image PN3, square black-painted images Sij (i = 1 to n, j = 1 to 8) are arranged at substantially equal intervals in the horizontal direction and the vertical direction.

Further, a missing region HL is formed in the pattern image PN3. The missing area HL indicates a region where a part of the pattern image PN3 is missing. The missing region HL shown in FIG. 6 is formed in a circular shape. Due to the missing region HL, four black-painted images Sij (i = 5 to 6, j = 4 to 5) are missing. On the X-axis, the black-painted image S54 and the black-painted image S64 are missing.

FIG. 7 is a diagram showing an example of a problem when executing the displacement distribution calculation process according to the embodiment of the present invention on the pattern image PN3 including the missing region HL.
The graph G21 at the top of FIG. 7 is a graph obtained by sine-wave approximation of the change in luminance B in the X-axis direction. The horizontal axis of the graph G21 is the X-axis, and the vertical axis is the luminance B. Since the black-painted image S54 and the black-painted image S64 are missing in the missing region HL, the brightness B corresponds to, for example, the color of the material constituting the test target TP. Therefore, in the missing region HL, the brightness B changes irregularly. As a result, the sine wave approximation cannot be performed in the missing region HL.

The second graph G22 from the top of FIG. 7 is a graph showing the phase of the graph G21. The horizontal axis of the graph G22 is the X-axis, and the vertical axis is the phase Q. The phase Q varies from −π to + π. In the missing region HL, since the luminance B changes irregularly, the phase Q also changes from −π to + π, but the graph does not have a sawtooth shape.

The lowermost graph G23 of FIG. 7 shows a graph after making a phase connection with respect to the second graph G22 from the top of FIG. 7. The horizontal axis of the graph G23 is the X-axis, and the vertical axis is the phase φ. In the graph G22, the phase decreases stepwise to −π at the position where the phase reaches + π, but in the graph G23, every time the phase decreases from + π to −π so as not to change the phase stepwise. , Increase the phase by 2 × π. As a result, in the graph G23, the phase φ increases as the X coordinate increases. However, in the missing region HL in the graph G23, the phase φ is increasing while changing irregularly as the X coordinate increases. In the graph G13, the phase φ increases linearly as the X coordinate increases, whereas in the graph G23, the phase φ does not linearly increase as the X coordinate increases. Therefore, as it is, the calculation unit 813 cannot calculate the displacement distribution on the surface of the TP to be tested.
In the embodiment of the present invention, in order to solve this problem, the complementary unit 812 complements the pattern image PN corresponding to the missing region HL to generate the entire complementary image PC. The calculation unit 813 calculates the displacement distribution on the surface of the test target TP based on the overall complementary image PC.

[1-6. Learning data]
FIG. 8 is a diagram showing an example of a training data DL that generates a trained model MD according to the first embodiment of the present invention.
The learning data DL correlates the information of the pattern image PN including the missing area HL with the information of the pattern image PM not including the missing area HL. In the following description, the pattern image PN including the missing area HL may be referred to as a missing image PN.
The pattern image PM that does not include the missing region HL is generated by the imaging unit 811. The missing image PN is generated by the generation unit 712 from the pattern image PM that does not include the missing region HL.

The pattern image PM arranged on the left side of the uppermost row of FIG. 8 shows an example of the pattern image PM that does not include the missing region HL. The pattern image PM is a horizontally long rectangular image in which square black-painted images are arranged at equal intervals in the horizontal direction and the vertical direction.
The missing image PN arranged on the upper right side of FIG. 8 shows an example of a pattern image including the missing region HL. The missing image PN is generated by arranging the rectangular missing region HL in the vertical center on the left side of the pattern image PM.

The pattern image PM arranged on the left side of the second row from the top of FIG. 8 shows an example of the pattern image PM that does not include the missing region HL. The pattern image PM is a square image in which square black-painted images are arranged at equal intervals in the horizontal direction and the vertical direction. The size of the black-painted image is smaller than the size of the black-painted image of the pattern image PM arranged on the left side of the uppermost row in FIG.
The missing image PN arranged on the right side of the second row from the top of FIG. 8 shows an example of a pattern image including the missing region HL. The missing image PN is generated by arranging the rectangular missing region HL at the center of the right end of the pattern image PM in the vertical direction.

The pattern image PM arranged on the left side of the third row from the top of FIG. 8 shows an example of the pattern image PM that does not include the missing region HL. The pattern image PM is a vertically long rectangular image, and square black-painted images are arranged at equal intervals in the horizontal direction and the vertical direction. The size of the black-painted image is smaller than the size of the black-painted image of the pattern image PM arranged on the left side of the uppermost row in FIG. 8, and the black-painted image PM arranged on the left side of the second row from the top of FIG. Larger than the size of the image.
The missing image PN arranged on the right side of the third row from the top of FIG. 8 shows an example of a pattern image including the missing region HL. The missing image PN is generated by arranging the vertically long elliptical missing region HL on the upper left side of the missing image PN.

As described with reference to FIG. 8, the generation unit 712 generates a missing image PN having a predetermined shape of the position PS and the size SZ determined by the determination unit 711. The predetermined shape includes a rectangular shape. That is, if there is one pattern image PM that does not include the missing region HL, a plurality of missing image PNs can be generated. For example, when any of nine positions is selected as the position PS and any of the four sizes is selected as the size SZ, 36 (= 9 × 4) missing image PNs can be generated.
Therefore, the generation unit 712 can generate a large number of missing image PNs by generating the missing image PNs having a predetermined shape of the position PS and the size SZ determined by the determination unit 711.

Further, the trained model MD is machine-learned by the learning unit 713 so as to output the information of the entire complementary image PC when the information of the missing image PN is input. The whole complementary image PC corresponds to the pattern image PM that does not include the missing area HL.

[1-7. Processing of the second control unit]
FIG. 9 is a flowchart showing an example of processing of the second control unit 7 according to the embodiment of the present invention. In the learning data storage unit 714, a plurality of pattern image PMs that do not include the missing area HL are stored in advance.
First, in step S101, the second control unit 7 reads the pattern image PM from the learning data storage unit 714.
Next, in step S103, the determination unit 711 determines the position PS and the size SZ of the missing region HL.
Next, in step S105, the generation unit 712 generates a missing image PN including the missing region HL of the predetermined shape of the position PS and the size SZ determined by the determination unit 711 based on the pattern image PM. The predetermined shape is, for example, a rectangular shape.
Next, in step S107, the generation unit 712 associates the missing image PN with the pattern image PM and stores them in the learning data storage unit 714.

Next, in step S109, the determination unit 711 determines whether or not to end the determination of the size SZ.
If the determination unit 711 determines that the determination of the size SZ is not completed (step S109; NO), the process returns to step S103. When the determination unit 711 determines that the determination of the size SZ is completed (step S109; YES), the process proceeds to step S111.
Then, in step S111, the determination unit 711 determines whether or not to end the determination of the position PS.
If the determination unit 711 determines that the determination of the position PS is not completed (step S111; NO), the process returns to step S103. When the determination unit 711 determines that the determination of the position PS is completed (step S111; YES), the process proceeds to step S113.
Then, in step S113, the second control unit 7 determines whether or not to finish reading the pattern image PM from the learning data storage unit 714.
When the second control unit 7 determines that the reading of the pattern image PM from the learning data storage unit 714 is not completed (step S113; NO), the process returns to step S101. When the second control unit 7 determines that the reading of the pattern image PM from the learning data storage unit 714 is completed (step S113; YES), the process proceeds to step S115.
Then, in step S115, the learning unit 713 executes machine learning based on the learning data DL stored in the learning data storage unit 714, generates a learned model MD, and stores the generated learned model MD in the trained model storage. It is stored in the unit 715, and the process is completed.

[1-8. First example in the processing of the complementary part]
FIG. 10 is a diagram showing a first example of processing of the complementary unit 812 according to the first embodiment of the present invention.
The complement unit 812 inputs the information of the pattern image PN including the missing area HL into the trained model MD, and outputs the information of the pattern image PM not including the missing area HL from the trained model MD.
Then, the complementing unit 812 executes a cutting process for cutting out the missing complementary image PCH corresponding to the missing region HL from the pattern image PM.
Next, the complement unit 812 generates an overall complementary image PC by synthesizing the missing complementary image PCH and the pattern image PN including the missing region HL.
In this way, the complementary unit 812 generates the entire complementary image PC.

The reason why the pattern image PM is not used as the overall complementary image PC is that the pattern image PM in which the pattern image is distorted in a region other than the missing region HL may be output from the trained model MD.
Even in such a case, the complementary unit 812 generates the entire complementary image PC by synthesizing the missing complementary image PCH and the pattern image PN including the missing region HL, so that the distorted pattern image PM Can be suppressed from being included in the overall complementary image PC.

[1-9. Second example in the processing of the complementary part]
FIG. 11 is a diagram showing a second example of processing of the complementary unit 812 according to the first embodiment of the present invention. In the second example, as shown in FIG. 11, each time the imaging unit 811 generates a pattern image PN including the missing region HL, the complement unit 812 executes the process described with reference to FIG.

The imaging unit 811 generates a pattern image PN including the missing region HL at the time T1, the time T2, and the time T3, for example. The pattern image PN generated at time T1 may be described as the pattern image PK1. The pattern image PN generated at time T2 may be described as the pattern image PK2. The pattern image PN generated at time T3 may be described as the pattern image PK3. The time between time T1 and time T2 is a predetermined time, and the time between time T2 and time T3 is a predetermined time. The predetermined time is, for example, 0.01 seconds.

The complement unit 812 inputs the information of the pattern image PK1 including the missing area HL into the trained model MD, and outputs the information of the pattern image PM1 not including the missing area HL from the trained model MD.
Then, the complementing unit 812 executes a cutting process for cutting out the missing complementary image PCH1 corresponding to the missing region HL from the pattern image PM1.
Next, the complement unit 812 generates the entire complementary image PC1 by synthesizing the missing complementary image PCH1 and the pattern image PK1 including the missing region HL.

Further, the complement unit 812 inputs the information of the pattern image PK2 including the missing area HL into the trained model MD, and outputs the information of the pattern image PM2 not including the missing area HL from the learned model MD.
Then, the complementing unit 812 executes a cutting process for cutting out the missing complementary image PCH2 corresponding to the missing region HL from the pattern image PM2.
Next, the complementary unit 812 generates the entire complementary image PC2 by synthesizing the missing complementary image PCH2 and the pattern image PK2 including the missing region HL.

Further, the complement unit 812 inputs the information of the pattern image PK3 including the missing area HL into the trained model MD, and outputs the information of the pattern image PM3 not including the missing area HL from the trained model MD.
Then, the complementing unit 812 executes a cutting process for cutting out the missing complementary image PCH3 corresponding to the missing region HL from the pattern image PM3.
Next, the complementary unit 812 generates the entire complementary image PC3 by synthesizing the missing complementary image PCH3 and the pattern image PK3 including the missing region HL.

The processing of the first control unit 8 in the second example will be described.
FIG. 12 is a flowchart showing an example of the processing of the first control unit 8 in the second example.
First, in step S201, the first control unit 8 determines whether or not the tensile tester 1 has started the tensile test.
When the first control unit 8 determines that the tensile tester 1 has not started the tensile test (step S201; NO), the process is in the standby state. When the first control unit 8 determines that the tensile tester 1 has started the tensile test (step S201; YES), the process proceeds to step S203.
Then, in step S203, the imaging unit 811 determines whether or not it is the imaging timing. The imaging timing indicates the timing at which the camera 6 executes imaging.
When the imaging unit 811 determines that it is not the imaging timing (step S203; NO), the process is in the standby state. When the imaging unit 811 determines that it is the imaging timing (step S203; YES), the process proceeds to step S205.
Then, in step S205, the imaging unit 811 causes the camera 6 to image the pattern PTN to generate a pattern image PN showing the image of the pattern PTN.

Next, in step S207, the first control unit 8 determines whether or not the pattern image PN includes the missing region HL.
When the first control unit 8 determines that the pattern image PN does not include the missing region HL (step S207; NO), the process proceeds to step S215. When the first control unit 8 determines that the pattern image PN includes the missing region HL (step S207; YES), the process proceeds to step S209.
Then, in step S209, the complement unit 812 inputs the information of the pattern image PN including the missing area HL into the trained model MD, and outputs the information of the pattern image PM not including the missing area HL from the trained model MD. Let me.
Next, in step S211 the complementary unit 812 executes a cutting process for cutting out the missing complementary image PCH corresponding to the missing region HL from the pattern image PM.
Next, in step S213, the complement unit 812 generates an overall complementary image PC by synthesizing the missing complementary image PCH and the pattern image PN including the missing region HL.

Next, in step S215, the calculation unit 813 calculates the displacement distribution on the surface of the test target TP based on the overall complementary image PC.
Next, in step S217, the first control unit 8 determines whether or not the tensile tester 1 has completed the tensile test.
When the first control unit 8 determines that the tensile tester 1 has not completed the tensile test (step S217; NO), the process returns to step S203. When the first control unit 8 determines that the tensile tester 1 has completed the tensile test (step S217; YES), the process ends.

Step S205 corresponds to an example of the “imaging step”. Step S209, step S211 and step S213 correspond to an example of the "complementary step". Step S215 corresponds to an example of the “calculation step”.

[1-10. Third example in the processing of the complementary part]
FIG. 13 is a diagram showing a third example of processing of the complementary unit 812 according to the first embodiment of the present invention. In the third example, the complementary unit 812 executes the process described with reference to FIG. 10 only when the imaging unit 811 first generates the pattern image PN including the missing region HL.

The imaging unit 811 generates a pattern image PN including the missing region HL at the time T1, the time T2, and the time T3, for example. The pattern image PN generated at time T1 may be described as the pattern image PK1. The pattern image PN generated at time T2 may be described as the pattern image PK2. The pattern image PN generated at time T3 may be described as the pattern image PK3. The time between time T1 and time T2 is a predetermined time, and the time between time T2 and time T3 is a predetermined time. The predetermined time is, for example, 0.01 seconds.

The complement unit 812 inputs the information of the pattern image PK1 including the missing area HL1 to the trained model MD, and outputs the information of the pattern image PM1 not including the missing area HL from the trained model MD.
Then, the complementing unit 812 executes a cutting process for cutting out the missing complementary image PCH1 corresponding to the missing region HL1 from the pattern image PM1.
Next, the complement unit 812 generates the entire complementary image PC1 by synthesizing the missing complementary image PCH1 and the pattern image PK1 including the missing region HL1.

Further, the complementing unit 812 executes a projective transformation on the missing complementary image PCH1 based on the missing region HL1 included in the pattern image PK1 and the missing region HL2 included in the pattern image PK2 to obtain the missing complementary image PCH2. Generate. Specifically, the complement unit 812 executes a projective transformation on the missing complementary image PCH1 so that the size of the missing complementary image PCH2 matches the size of the missing region HL2 to generate the missing complementary image PCH2. ..
The pattern image PK1 corresponds to an example of the "first pattern image". The pattern image PK2 corresponds to an example of the "second pattern image". The missing complementary image PCH1 corresponds to an example of the "first complementary image". The missing complementary image PCH2 corresponds to an example of the "second complementary image".
Then, the complementary unit 812 generates the entire complementary image PC2 by synthesizing the missing complementary image PCH2 and the pattern image PK2.

Further, the complementing unit 812 executes a projective transformation on the missing complementary image PCH1 based on the missing region HL1 included in the pattern image PK1 and the missing region HL3 included in the pattern image PK3 to obtain the missing complementary image PCH3. Generate. Specifically, the complement unit 812 executes a projective transformation on the missing complementary image PCH1 so that the size of the missing complementary image PCH3 matches the size of the missing complementary image PCH3 to generate the missing complementary image PCH3. ..
The pattern image PK3 corresponds to an example of the "second pattern image". The missing complementary image PCH3 corresponds to an example of the "second complementary image".
Then, the complementing unit 812 generates the entire complementary image PC3 by synthesizing the missing complementary image PCH3 and the pattern image PK3.

The processing of the first control unit 8 in the third example will be described.
FIG. 14 is a flowchart showing an example of the processing of the first control unit 8 in the third example.
First, in step S301, the first control unit 8 determines whether or not the tensile tester 1 has started the tensile test.
When the first control unit 8 determines that the tensile tester 1 has not started the tensile test (step S301; NO), the process is in the standby state. When the first control unit 8 determines that the tensile tester 1 has started the tensile test (step S301; YES), the process proceeds to step S303.
Then, in step S303, the imaging unit 811 determines whether or not it is the imaging timing. The imaging timing indicates the timing at which the camera 6 executes imaging.
When the imaging unit 811 determines that it is not the imaging timing (step S303; NO), the process is in the standby state. When the imaging unit 811 determines that it is the imaging timing (step S303; YES), the process proceeds to step S305.
Then, in step S305, the imaging unit 811 causes the camera 6 to image the pattern PTN to generate a pattern image PN showing the image of the pattern PTN.

Next, in step S307, the first control unit 8 determines whether or not the pattern image PN includes the missing region HL.
When the first control unit 8 determines that the pattern image PN does not include the missing region HL (step S307; NO), the process proceeds to step S321. When the first control unit 8 determines that the pattern image PN includes the missing region HL (step S307; YES), the process proceeds to step S309.
Then, in step S309, the first control unit 8 determines whether or not the missing complementary image PCH has been generated.
When the first control unit 8 determines that the missing complementary image PCH has not been generated (step S309; NOS), the process proceeds to step S311.
Then, in step S311 the complementary unit 812 inputs the information of the pattern image PN including the missing area HL into the trained model MD, and outputs the information of the pattern image PM not including the missing area HL from the trained model MD. Let me.
Next, in step S313, the complement unit 812 executes a cutting process for cutting out the missing complementary image PCH corresponding to the missing region HL from the pattern image PM. Then, the process proceeds to step S319.

When the first control unit 8 determines that the missing complementary image PCH has been generated (step S309; YES), the process proceeds to step S315.
Then, in step S315, the complementary unit 812 acquires the size of the missing region HL.
Next, in step S317, the complement unit 812 executes a projective transformation on the missing complementary image PCH based on the size of the missing complementary image PCH to generate the missing complementary image PCH.
Next, in step S319, the complement unit 812 generates an overall complementary image PC by synthesizing the missing complementary image PCH and the pattern image PN including the missing region HL.

Next, in step S321, the calculation unit 813 calculates the displacement distribution on the surface of the test target TP based on the overall complementary image PC.
Next, in step S323, the first control unit 8 determines whether or not the tensile tester 1 has completed the tensile test.
When the first control unit 8 determines that the tensile tester 1 has not completed the tensile test (step S323; NO), the process returns to step S303. When the first control unit 8 determines that the tensile tester 1 has completed the tensile test (step S323; YES), the process ends.

Step S205 corresponds to an example of the “imaging step”. Step S311 and step S313, step S315, step S317 and step S319 correspond to an example of the "complementary step". Step S321 corresponds to an example of the "calculation step".

[Second Embodiment of Complementary Unit 812]
FIG. 15 is a diagram showing an example of the second embodiment of the complementary unit 812.
In the first embodiment of the complement unit 812, the complement unit 812 generated the complement image PC using the trained model MD, but in the second embodiment of the complement unit 812, the complement unit 812 uses the trained model MD. Generates a complementary image PC without.
More specifically, the complementary unit 812 may complement the pattern image PN corresponding to the missing region HL to generate the complementary image PC.
As shown in FIG. 15, the complementary unit 812 copies, for example, a pattern image PCA having the same size as the missing area HL from the pattern image PN around the missing area HL. Then, the complementary unit 812 may generate the complementary image PC by fitting the copied pattern image PCA into the missing region HL.
In this process, the process of determining the pattern image PCA to be copied may be executed according to the operation from the user.
The pattern image PCA is preferably copied from a position close to the missing area HL. In this case, the displacement in the pattern image PCA approximates the displacement of the pattern image PN in the missing region HL.

[6. Aspects and effects]
It will be understood by those skilled in the art that the above-described embodiments and modifications are specific examples of the following aspects.

(Section 1)
The displacement distribution measuring device according to one aspect is a displacement distribution measuring device that measures the displacement distribution on the surface of the object based on the image of the pattern formed on the surface of the object, and is a pattern image showing the image of the pattern. An imaging unit that generates a complementary image, and a complementary unit that complements the pattern image corresponding to the missing area to generate a complementary image when the pattern image includes a missing area in which a part of the pattern image is missing. A displacement distribution measuring device including a calculation unit for calculating a displacement distribution on the surface of the object based on a complementary image.

According to the displacement distribution measuring device according to the first item, when the pattern image includes a missing region in which a part of the pattern image is missing, the complementary unit complements the pattern image corresponding to the missing region. A complementary image is generated, and the calculation unit calculates the displacement distribution on the surface of the object based on the complementary image. Therefore, the displacement distribution on the surface of the object can be calculated even when a part of the pattern image is missing.

(Section 2)
In the displacement distribution measuring device according to the first item, the complementary unit corresponds the information of the pattern image including the missing region with the information of the complementary image to which the pattern image corresponding to the missing region is complemented. It has a trained model, which is generated based on the attached training data.

According to the displacement distribution measuring device according to the second item, the complementary unit includes the information of the pattern image including the missing region and the information of the complementary image to which the pattern image corresponding to the missing region is complemented. A trained model, which is generated based on the training data associated with the above, is provided. Therefore, the complementary image can be easily obtained by using the trained model.

(Section 3)
In the displacement distribution measuring device according to the second item, the complementary unit inputs the information of the pattern image including the missing region into the trained model, and inputs the information of the complementary image corresponding to the missing region to the trained model. Output from the trained model.

According to the displacement distribution measuring device according to the third item, the complementary unit inputs the information of the pattern image including the missing region into the trained model, and the information of the complementary image corresponding to the missing region. Is output from the trained model. Therefore, the complementary image corresponding to the missing region can be easily obtained by using the trained model.

(Section 4)
In the displacement distribution measuring device according to the third item, the complementary unit generates the complementary image by synthesizing the pattern image including the missing region and the complementary image corresponding to the missing region. ..

According to the displacement distribution measuring device according to the fourth item, the complementary unit obtains the complementary image by synthesizing the pattern image including the missing region and the complementary image corresponding to the missing region. Generate. Therefore, the complementary unit can generate an accurate complementary image.

(Section 5)
In the displacement distribution measuring device according to any one of items 2 to 4, the imaging unit generates the pattern image at predetermined time intervals, and the complementary unit generates the pattern image by the imaging unit. Each time it is generated, the complementary image is generated.

According to the displacement distribution measuring device according to the fifth item, the imaging unit generates the pattern image at predetermined time intervals, and the complementary unit generates the pattern image every time the imaging unit generates the pattern image. Generate an image. Therefore, the displacement distribution for each predetermined time can be calculated.

(Section 6)
In the displacement distribution measuring device according to the fifth item, when the pattern image includes the missing region, the complementary unit includes the missing region each time the imaging unit generates the pattern image. The complementary image is generated by inputting image information into the trained model and outputting information of the complementary image corresponding to the missing region from the trained model.

According to the displacement distribution measuring device according to the sixth item, each time the imaging unit generates the pattern image, the complementary unit inputs the information of the pattern image including the missing region into the trained model. Then, the complementary image is generated by outputting the information of the complementary image corresponding to the missing region from the trained model. Therefore, every time the imaging unit generates the pattern image, the information of the complementary image corresponding to the missing region is output from the trained model to generate the complementary image, so that an accurate complementary image can be obtained. Can be generated.

(Section 7)
In the displacement distribution measuring device according to the fifth item, when the pattern image includes the missing region, the complementary unit inputs the information of the first pattern image including the missing region into the trained model. Then, by outputting the information of the first complementary image corresponding to the missing region from the trained model, the complementary image is generated, and after the first pattern image, the imaging unit second. Each time the pattern image is generated, the second complementary image corresponding to the missing region in the second pattern image is projected and the first complementary image is projected according to the deformation of the missing region. Generate by doing.

According to the displacement distribution measuring device according to the seventh item, every time the imaging unit generates the second pattern image after the first pattern image, the missing part in the second pattern image. The second complementary image corresponding to the region is generated by projecting and transforming the first complementary image according to the deformation of the missing region. Therefore, since the second complementary image is generated by projecting the first complementary image according to the deformation of the missing region, the second complementary image can be generated by a simple process.

(Section 8)
In the displacement distribution measuring device according to any one of the items 1 to 7, the pattern shows a grid pattern.

According to the displacement distribution measuring device according to the eighth item, since the pattern is a grid pattern, the displacement distribution on the surface of the object can be easily calculated.

(Section 9)
In the displacement distribution measuring device according to the eighth item, the calculation unit calculates the displacement distribution on the surface of the object by calculating the amount of change in the spacing of the grids.

According to the displacement distribution measuring device according to the ninth item, the calculation unit calculates the displacement distribution on the surface of the object by calculating the amount of change in the spacing of the grids. Therefore, the displacement distribution on the surface of the object can be calculated accurately.

(Section 10)
In the displacement distribution measuring device according to any one of paragraphs 1 to 9, the object is a test object deformed by a material testing machine.

According to the displacement distribution measuring device according to the tenth item, since the object is a test material deformed by the material tester, the displacement distribution on the surface of the test material deformed by the material tester can be accurately calculated. ..

(Section 11)
In the displacement distribution measuring device according to item 10, the material testing machine is a tensile testing machine, and a circular hole is formed as the missing region in the test target.

According to the displacement distribution measuring device according to the eleventh item, a circular hole is formed in the test material as the missing region. In this way, even when the missing region exists, the complementary portion complements the pattern image corresponding to the missing region to generate a complementary image, so that the test material deformed by the material testing machine. The displacement distribution on the surface can be calculated accurately.

(Section 12)
In the displacement distribution measuring device according to any one of the items 1 to 9, the object is a structure including a bridge.

According to the displacement distribution measuring device according to the twelfth item, since the object is a structure including a bridge, the displacement distribution on the surface of the structure including the bridge can be accurately calculated.

(Section 13)
The displacement distribution measurement method according to one aspect is a displacement distribution measurement method for measuring the displacement distribution on the surface of the object based on the image of the pattern formed on the surface of the object, and is a pattern image showing the image of the pattern. The imaging step of generating the complementary image, and the complementary step of complementing the pattern image corresponding to the missing region to generate a complementary image when the pattern image includes a missing region in which a part of the pattern image is missing. This is a displacement distribution measurement method including a calculation step of calculating the displacement distribution of the surface of the object based on the complementary image.

According to the displacement shape detecting method described in the thirteenth item, the same effect as that of the displacement distribution measuring device described in the first item is obtained.

(Section 14)
The control program of the displacement distribution measuring device according to one aspect is a control program of the displacement distribution measuring device which is equipped with a computer and measures the displacement distribution of the surface of the object based on the image of the pattern formed on the surface of the object. The computer is used as an imaging unit that generates a pattern image showing an image of the pattern, and when the pattern image includes a missing area in which a part of the pattern image is missing, the pattern image corresponding to the missing area is displayed. This is a control program of a displacement distribution measuring device that functions as a complementary unit that complements and generates a complementary image and a calculation unit that calculates the displacement distribution of the surface of the object based on the complementary image.

According to the control program of the displacement distribution measuring device according to the item 14, the same effect as that of the displacement distribution measuring device described in the first item is obtained.

[7. Other embodiments]
In the embodiment of the present invention, the case where the object is the test target TP of the tensile tester 1 has been described, but the object may be a structure including a bridge. When the object is a bridge, for example, a pattern is formed on the side surface of the main girder. In this case, for example, the displacement distribution of the main girder in the vertical direction and the horizontal direction can be measured.

Further, in the embodiment of the present invention, the case where the material tester is the tensile tester 1 has been described, but the embodiment of the present invention is not limited to this. The material tester may apply the test force to the test target TP and deform the test target TP to perform the material test. For example, the material tester may be a compression tester, a bending tester, or a torsion tester.

Further, in the embodiment of the present invention, the case where the pattern image PN includes a missing region HL in which a part of the pattern image PN is missing has been described, but the embodiment of the present invention is not limited to this. For example, a hole may be formed in a part of the TP to be tested. In D6484 of the ASTM (American Society for Testing and Materials) standard, a hole is formed in a part of the TP to be tested.

Further, in the embodiment of the present invention, the case where the first control unit 8 functions as the imaging unit 811, the complementary unit 812, and the calculation unit 813 has been described, but the embodiment of the present invention is not limited thereto. The control circuit unit 50 may function as at least one of the image pickup unit 811 and the complement unit 812 and the calculation unit 813. For example, the control circuit unit 50 may function as an imaging unit 811, a complementary unit 812, and a calculation unit 813.

The displacement distribution measuring device 200 according to the embodiment of the present invention is merely an example of the aspect of the displacement distribution measuring device according to the present invention, and can be arbitrarily modified and applied without departing from the gist of the present invention. ..

In the embodiment of the present invention, the learning step has been described based on the learning algorithm of "pix2pix", but an algorithm that does not use deep learning, for example, SVM (Support Vector Machine) or machine learning in which the user directly specifies a learning parameter is also possible. Good. Further, instead of the estimation by learning, a deterministic estimation method determined by a mathematical model may be used.

1 Tensile tester (material tester)
2 Tension tester body 4 Control unit 6 Camera 7 2nd control unit 8 1st control unit 10 Cross head 12 Load mechanism 14 Load cell 15 Servo motor 20 Rotary encoder 21 Upper gripper 22 Lower gripper 26 Table 28, 29 Screw rod 30 General control device 32 Display device 34 Test program execution device 40 Signal input / output unit 42 Sensor amplifier 43 Counter circuit 44 Servo amplifier 50 Control circuit unit 200 Displacement distribution measurement device 71A 2nd processor 71B 2nd memory device 71C 2nd storage device 71D 2 Communication device 711 Decision unit 712 Generation unit 713 Learning unit 714 Learning data storage unit 715 Learned model storage unit 81A First processor (computer)
81B 1st memory device 81C 1st storage device 81D 1st communication device 811 Imaging unit 812 Complementary unit 813 Calculation unit 814 Learning model storage unit DL Learning data HL Missing area MD Learned model PC, PC1, PC2, PC3 Overall complementary image ( Complementary image)
PCH, PCH1, PCH2, PCH3 Missing complementary image PK1, PK2, PK3 pattern image PM pattern image PN, PN1, PN2, PN3 pattern image PS position PTN pattern Qij, Rij, Sij Black-painted image SZ size TP test target TP1 specific surface

Claims (14)

A displacement distribution measuring device that measures the displacement distribution on the surface of an object based on an image of a pattern formed on the surface of the object.
An imaging unit that generates a pattern image showing the image of the pattern, and
When the pattern image includes a missing region in which a part of the pattern image is missing, a complementary unit that complements the pattern image corresponding to the missing region to generate a complementary image.
A calculation unit that calculates the displacement distribution of the surface of the object based on the complementary image,
Displacement distribution measuring device. The complementary unit is a trained model generated based on learning data in which the information of the pattern image including the missing region and the information of the complementary image to which the pattern image corresponding to the missing region is complemented are associated with each other. The displacement distribution measuring device according to claim 1, further comprising. The complementary unit inputs the information of the pattern image including the missing area to the trained model, and outputs the information of the complementary image corresponding to the missing area from the trained model according to claim 2. The displacement distribution measuring device described. The displacement distribution measuring device according to claim 3, wherein the complementary unit generates the complementary image by synthesizing the pattern image including the missing region and the complementary image corresponding to the missing region. The imaging unit generates the pattern image at predetermined time intervals and generates the pattern image.
The displacement distribution measuring device according to any one of claims 2 to 4, wherein the complementary unit generates the complementary image each time the imaging unit generates the pattern image. When the pattern image includes the missing region, the complementary unit inputs the information of the pattern image including the missing region into the trained model each time the imaging unit generates the pattern image. The displacement distribution measuring device according to claim 5, wherein the complementary image is generated by outputting the information of the complementary image corresponding to the missing region from the trained model. The complementary part
When the pattern image includes the missing area, the information of the first pattern image including the missing area is input to the trained model, and the information of the first complementary image corresponding to the missing area is input. By outputting from the trained model, the complementary image is generated.
After the first pattern image, every time the imaging unit generates the second pattern image, the missing complementary image corresponding to the missing region in the second pattern image is removed. The displacement distribution measuring device according to claim 5, which is generated by projecting and transforming the first complementary image according to the deformation of the region. The displacement distribution measuring device according to any one of claims 1 to 7, wherein the pattern shows a grid pattern. The displacement distribution measuring device according to claim 8, wherein the calculation unit calculates the displacement distribution on the surface of the object by calculating the amount of change in the spacing of the grids. The displacement distribution measuring device according to any one of claims 1 to 9, wherein the object is a test object deformed by a material testing machine. The displacement distribution measuring device according to claim 10, wherein the material testing machine is a tensile testing machine, and a circular hole is formed as the missing region in the test object. The displacement distribution measuring device according to any one of claims 1 to 9, wherein the object is a structure including a bridge. A displacement distribution measuring method for measuring the displacement distribution on the surface of an object based on an image of a pattern formed on the surface of the object.
An imaging step that generates a pattern image showing the image of the pattern, and
When the pattern image includes a missing region in which a part of the pattern image is missing, a complementing step of complementing the pattern image corresponding to the missing region to generate a complementary image.
A calculation step for calculating the displacement distribution on the surface of the object based on the complementary image, and
Displacement distribution measurement method including. A control program of a displacement distribution measuring device equipped with a computer and measuring a displacement distribution on the surface of the object based on an image of a pattern formed on the surface of the object.
The computer
An imaging unit that generates a pattern image showing an image of the pattern,
When the pattern image includes a missing region in which a part of the pattern image is missing, a complementary unit that complements the pattern image corresponding to the missing region to generate a complementary image, and
A calculation unit that calculates the displacement distribution of the surface of the object based on the complementary image.
A control program for the displacement distribution measuring device that functions as.

Images