JP2010169590A - Thermal deformation measurement method and apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To correctly measure a three-dimensional thermal deformation characteristic of a to-be-measured object in which a temperature and a shape are changed in various patterns. <P>SOLUTION: A synchronization signal Strg is generated in a temporal sequence. A combination of thermal image data from a thermal camera 20 and visible image data from a plurality of visible cameras 21 at the same imaging time point is sampled in synchronization with the synchronization signal Strg. Three-dimensional shape data at each imaging time point is calculated through a stereo image processing based on the visible image data at each imaging time point. A displacement on the visible image of an image in an observation region at each imaging time point is calculated through a pattern tracking processing by a digital image correlation method based on a plurality of the visible image data at different imaging time points. Data corresponding to location and temperature data in a three-dimensional coordinate system of the observation region at each imaging time point is recorded based on calculation results. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a thermal deformation measuring method and a thermal deformation measuring apparatus for measuring a three-dimensional shape change according to a temperature change of an object to be measured.

In recent years, electronic devices such as processors, power modules, connectors, and the like have become more complicated in three-dimensional structures and used in higher temperatures. For example, in a processor such as a CPU or DSP (Digital Signal Processor), the amount of heat generation is remarkably increased as the calculation speed is increased, and the breakage of packages and electrodes due to thermal deformation is becoming a problem. In addition, electronic devices such as inverter power modules used in electric vehicles, hybrid vehicles, digital home appliances, and the like have increased in heat generation as output is improved. These require high reliability, and heat resistance evaluation is becoming extremely important. Furthermore, electronic devices such as connectors, which are electrical components used in automobiles and aircraft, are also required to have heat resistance by being placed close to a heat source such as an engine.
In general, in designing electronic devices, a three-dimensional shape change corresponding to a temperature change of the electronic device is calculated by a thermal deformation simulation using a computer, and the heat resistance of the electronic device is evaluated based on the calculation result. Often. However, because the heat generation position of a processor such as a CPU is local and high, the thermal deformation behavior may differ from the design value due to differences in local characteristics due to the material composition distribution, tissue structure, etc. . Therefore, it is necessary to improve the simulation accuracy by actually measuring and evaluating the actual deformation pattern of the electronic device and correcting the deviation from the design value.
On the other hand, an MCP (Multi Chip Package) in which a plurality of IC chips such as a processor IC, a memory IC, and a signal processing IC are housed in one package is used for a portable information terminal or the like. In particular, PoP (Package on Package) in which multiple BGA (Ball Grid Array) modules are stacked vertically and MEMS (Micro Electro Mechanical Systems) modules are also built in, and chips are stacked vertically into a single package. An MCP having a complicated structure such as a simplified SiP (System in Package) is also being put into practical use. New types of packages such as SiP and PoP are difficult to simulate with high accuracy due to their complicated structure. Therefore, it is very important to grasp the heat deformation pattern by actual measurement and evaluate the heat resistance of a complicated electronic device having such a structure.

On the other hand, pattern tracking processing using a digital image correlation method is known as a method for measuring deformation of an object to be measured using an image.
In general, the pattern tracking process in the digital image correlation method is a method in which processing according to the following procedure is performed by an arithmetic means.
First, among the images at different time points obtained by sequentially imaging the object to be measured, the positions of the subsets having a high correlation are associated with each subset that is an image pattern of a small area divided into a plurality of areas. . It can be said that the subset (image pattern) associated in this way is an image of the same imaging region displaced with the passage of time.
Then, by sequentially tracking the associated subsets according to the transition of the imaging time point, a displacement amount of a predetermined imaging region corresponding to the change of the imaging time point is calculated. The displacement amount of each of the imaging parts calculated in this way is the displacement amount on the image of each part on the surface of the object to be measured, that is, the shape change in the coordinate system of the imaged image of the object to be measured. To express. When the object to be measured is a moving object, the amount of movement of the object to be measured on the image is measured by pattern tracking processing using the digital image correlation method.

In addition, as a method for measuring the three-dimensional shape of an object to be measured using a plurality of two-dimensional images, a shape measurement method using stereo image processing is known.
In general, the shape measuring method by the stereo image processing is a method of performing processing according to the following procedure by a calculation means.
First, among the plurality of images obtained by imaging the object to be measured from different directions with a plurality of cameras, for each subset that is image data of a small area divided into a plurality, the correlation between the subsets having a high correlation is obtained. Associate positions.
Next, based on the triangulation method, the three-dimensional shape data in the real space in which the object to be measured is arranged is calculated from the positional relationship of the associated subset and the positional relationship of the camera. The three-dimensional shape data is data of each coordinate in a predetermined three-dimensional coordinate system in real space. The three-dimensional shape data has a known coordinate correspondence with each of a plurality of images obtained from a plurality of cameras. The shape measurement method based on the stereo image processing can be applied to the measurement of the three-dimensional shape of an object to be measured using three or more two-dimensional images.

Japanese Patent Application Laid-Open No. H10-228561 discloses measuring the stress fluctuation of a structure by the following method using the digital image correlation method.
In other words, in the measurement method disclosed in Patent Document 1, first, a structure to be measured is imaged with an infrared camera and a visible camera whose visual fields coincide with each other, and a time-series thermal image and a visible image are acquired. To do. The visible camera is a general camera for acquiring the visible image data which is normal image data in which luminance data is set for each pixel. The thermal image data is data in which temperature data is set for each pixel.
Next, the distortion of the structure is detected by applying the digital image correlation method to the visible image. The strain detected thereby is a strain in the two-dimensional direction of the structure.
Then, the stress distribution of the structure is calculated based on the distortion of the structure, and the position of the thermal image is corrected based on the stress distribution.
Finally, the stress variation of the structure is detected based on the thermal image after the position correction.

Patent Document 2 discloses a temperature measurement method that associates coordinates between a visible image and a thermal image using the shape measurement method based on the stereo image processing.
That is, in the temperature measurement method disclosed in Patent Document 2, an imaging range including a part of a temperature measurement target is imaged by one infrared camera and two visible cameras. The imaging ranges of each of these three cameras overlap but do not match.
Furthermore, in the temperature measurement method disclosed in Patent Document 2, the shape measurement method by the stereo image processing is applied to two visible images obtained by the two visible cameras, and a three-dimensional shape is calculated. Then, based on the correspondence relationship between the visible image and the three-dimensional shape, and the correspondence relationship between the thermal image obtained by the infrared camera and the three-dimensional shape, the visible image and the thermal image A coordinate correspondence between is derived.
Thus, the temperature of the object to be measured existing at the position designated on the visible image can be specified in the thermal image without completely matching the visual field range of the infrared camera and the visible camera. it can.

JP 2008-232998 A JP 2002-36653 A

However, the prior art has the following problems in accurately measuring the three-dimensional shape change corresponding to the temperature change of an object to be measured with a significant temperature change such as an electronic device.
For example, the technique disclosed in Patent Document 1 can measure the two-dimensional thermal deformation of the object to be measured by the digital image correlation method, but cannot measure the three-dimensional heat deformation of the object to be measured. was there.
Further, although the technique disclosed in Patent Document 2 can specify the temperature of the object to be measured existing at the position designated on the visible image in the thermal image, the temperature of the object to be measured which changes from time to time and the three-dimensional There is a problem in that the correspondence with the shape, that is, the three-dimensional thermal deformation characteristic of the object to be measured cannot be measured accurately.
Therefore, the present invention has been made in view of the above circumstances, and the object of the present invention is to provide a heat measurement characteristic capable of accurately measuring the three-dimensional thermal deformation characteristics of an object whose temperature and shape change in various patterns. An object of the present invention is to provide a deformation measuring method and a thermal deformation measuring apparatus.

In order to achieve the above object, a thermal deformation measuring method according to the present invention is a method for measuring three-dimensional deformation according to a temperature change of an object to be measured, and includes the following (1.1) to (1.5) It is a method which has each procedure shown by.
(1.1) A synchronization signal generation procedure for generating a synchronization signal in time series by the signal generation means.
(1.2) The imaging time point is the same from a thermo camera that captures a thermal image of the object to be measured and a plurality of visible cameras that capture visible images of the object to be measured at different positions in synchronization with the synchronization signal. An image data sampling procedure for sampling a combination of data of the thermal image and the plurality of visible images.
(1.3) A three-dimensional shape data calculation procedure for calculating three-dimensional shape data for each imaging time point through stereo image processing based on a combination of a plurality of visible image data for each imaging time point by an arithmetic means.
(1.4) Through the pattern tracking process in the digital image correlation method based on the data of a plurality of the visible images obtained from the specific visible camera by the calculation unit, the visible at each imaging time A displacement amount calculation procedure for calculating a displacement amount on the visible image of an image of one or a plurality of observation parts of the object to be measured in the image.
(1.5) Information of coordinate conversion between the preset three-dimensional shape and the thermal image, data of the thermal image at each imaging time point obtained by the image data sampling procedure, and the data Displacement amount on the visible image of the three-dimensional shape data for each imaging time point obtained by the three-dimensional shape data calculation procedure and the image of the observation site for each imaging time point obtained by the displacement amount calculation procedure Based on the above, a thermal deformation data recording procedure for storing in the storage means the three-dimensional thermal deformation data in which the position and temperature data in the three-dimensional coordinate system at each imaging time point of the observation site are associated.
Note that either the three-dimensional shape data calculation procedure or the displacement amount calculation procedure may be executed first or in parallel.

In the thermal deformation measurement method according to the present invention, a combination of a plurality of visible image data having the same imaging time point is sequentially sampled based on the synchronization signal, and stereo image processing and digital image correlation based on the visible image data are performed. With this pattern tracking process, the position of the observation part of the object to be measured in the three-dimensional coordinate system at each imaging time point is calculated. Here, the information on the position of the observation part in the three-dimensional coordinate system for each imaging time point is information on the three-dimensional shape change of the object to be measured according to the transition of the imaging time point.
Furthermore, in the thermal deformation measuring method according to the present invention, the thermal image data representing the temperature distribution of the object to be measured is also sampled at the same timing as the sampling of the visible image data based on the synchronization signal. The coordinate conversion information between the three-dimensional shape and the thermal image is represented by a coordinate conversion matrix called homography, and the homography can be obtained in advance by calculation.
Therefore, based on the known information, the temperature corresponding to the position of the observation part in the three-dimensional shape at the corresponding imaging time point from the data of the thermal image for each imaging time point obtained by the image data sampling procedure. Data can be identified. That is, the three-dimensional thermal deformation data obtained in the thermal deformation data recording procedure is an accurate correspondence between the temperature of the observation site and the three-dimensional position of the object to be measured, which changes every moment, that is, the measured object. The data shows the exact thermal deformation characteristics of the object.

In the thermal deformation measurement method according to the present invention, the synchronization signal generation procedure may be any one of the procedures shown in the following (1.6) to (1.9).
(1.6) A procedure for generating the synchronization signal in time series using an operation input through a predetermined operation unit as a trigger. Hereinafter, this procedure is referred to as a first synchronization signal generation procedure.
(1.7) A procedure for generating the synchronization signal in time series triggered by the elapse of a scheduled time. Hereinafter, this procedure is referred to as a second synchronization signal generation procedure.
(1.8) A procedure for generating the synchronization signal in time series by using the thermal image data sequentially obtained from the thermocamera as a trigger when reaching a predetermined temperature state. Hereinafter, this procedure is referred to as a third synchronization signal generation procedure.
(1.9) The synchronization signal is generated in time series triggered by a change in the shape of the object to be measured, which is calculated based on the data of the visible image sequentially obtained from the visible camera, reaching a predetermined deformation state. Procedure to make. Hereinafter, this procedure is referred to as a fourth synchronization signal generation procedure.
Whether the object to be measured is heated or cooled from the outside, self-heating or self-cooling, if the first synchronization signal generation procedure is adopted, the operator of the operation input can arbitrarily The thermal deformation of the DUT 1 can be measured at the timing.
On the other hand, there is a case where it is desired to measure the thermal deformation of the measurement object when the measurement object is heated or cooled from the outside according to a predetermined time schedule. In this case, in the thermal deformation measurement method according to the present invention, if the second synchronization signal generation procedure is adopted, the thermal deformation of the object to be measured can be measured every time a specific time point in the time schedule elapses.
In some cases, it is desired to measure the thermal deformation of the object to be measured every time a predetermined temperature change occurs. In some cases, it is desired to measure the thermal deformation of the object to be measured with different temperature resolutions for each temperature range. In such a case, in the thermal deformation measurement method according to the present invention, if the third synchronization signal generation procedure is employed, only necessary measurement data can be efficiently obtained.
Further, in the measurement of the thermal deformation of the object to be measured, there are cases where it is desired to measure the data of the heat deformation every time a predetermined shape change occurs in the object to be measured. In some cases, it is desired to measure the thermal deformation of the object to be measured with different resolutions depending on the magnitude of the shape change. In such a case, in the thermal deformation measurement method according to the present invention, if the fourth synchronization signal generation procedure is employed, only necessary measurement data can be efficiently obtained.

Further, the present invention can also be regarded as a thermal deformation measuring device that realizes the thermal deformation measuring method according to the present invention described above.
That is, the thermal deformation measuring apparatus according to the present invention is an apparatus for measuring three-dimensional deformation in accordance with a temperature change of an object to be measured, and each configuration shown in the following (2.1) to (2.7) A device comprising elements.
(2.1) Synchronization signal generation means for generating a synchronization signal in time series.
(2.2) A thermo camera that captures a thermal image of the object to be measured.
(2.3) A plurality of visible cameras that capture visible images of the measurement object at different positions.
(2.4) Image data sampling means for sampling a combination of data of the thermal image and the plurality of visible images having the same imaging time point from the thermo camera and the plurality of visible cameras in synchronization with the synchronization signal.
(2.5) Three-dimensional shape data calculating means for calculating three-dimensional shape data for each imaging time point through stereo image processing based on a combination of a plurality of visible image data for each imaging time point.
(2.6) Through the pattern tracking process in the digital image correlation method based on the data of the plurality of visible images having different imaging time points obtained from the specific visible camera, the coverage in the visible image at each imaging time point Displacement amount calculation means for calculating a displacement amount of the image of the measurement object on the visible image.
(2.7) Information of coordinate conversion between the preset three-dimensional shape and the thermal image, the thermal image data and the three-dimensional shape data for each imaging time point obtained by the image data sampling means Based on the data of the three-dimensional shape for each imaging time point obtained by the calculation means and the displacement amount on the visible image of the image of the observation site for each imaging time point obtained by the displacement amount calculation means. Thermal deformation data recording means for storing in the storage means three-dimensional thermal deformation data in which the position and temperature data in the three-dimensional coordinate system for each imaging time point of the observation site are associated.
The synchronization signal generating means in the thermal deformation measuring apparatus according to the present invention includes the first synchronization signal generation procedure, the second synchronization signal generation procedure, the third synchronization signal generation procedure, or the fourth synchronization signal generation procedure. One that performs any of the synchronization signal generation procedures is conceivable.
The thermal deformation measuring apparatus according to the present invention can provide the same effects as the above-described thermal deformation measuring method according to the present invention.

  According to the present invention, it is possible to accurately measure the three-dimensional thermal deformation characteristics of an object to be measured whose temperature and shape change in various patterns.

The schematic block diagram of the thermal deformation measuring apparatus X1 which concerns on the 1st Embodiment of this invention. An example of the time chart of the synchronizing signal in the thermal deformation measuring apparatus X1. The flowchart showing an example of the three-dimensional thermal deformation measurement process by the thermal deformation measuring apparatus X1. The schematic block diagram of the thermal deformation measuring apparatus X2 which concerns on the 2nd Embodiment of this invention. An example of the time chart of the synchronizing signal in the thermal deformation measuring apparatus X2. The schematic block diagram of the thermal deformation measuring apparatus X3 which concerns on the 3rd Embodiment of this invention. An example of the time chart of the synchronizing signal in the thermal deformation measuring apparatus X3.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings so that the present invention can be understood. The following embodiment is an example embodying the present invention, and does not limit the technical scope of the present invention.

The thermal deformation measuring devices X1 to X3 according to the embodiment of the present invention are devices that measure the deformation of a three-dimensional shape according to a temperature change of an object to be measured such as an electronic device whose temperature and shape change in various patterns. .
First, the configuration and function of the thermal deformation measuring apparatus X1 according to the first embodiment of the present invention will be described with reference to FIG.
As shown in FIG. 1, the thermal deformation measuring device X1 includes a heating / cooling plate 11, a temperature control device 12, a thermo camera 20, two visible cameras 21, a trigger signal generation timing external input device 30, and a time monitoring device. 31, a visible image sampling device 41, a visible image display 42, a three-dimensional shape calculation device 43, a thermal image sampling device 51, a thermal image display 52, a thermal deformation analysis computer 61, a data input device 62, an analysis image display 63, and the like. ing.
The visible image sampling device 41, the thermal image sampling device 51, and the three-dimensional shape calculation device 43 are realized by calculation means such as a DSP (Digital Signal Processor) or a computer such as a personal computer.

The heating / cooling plate 11 is a device for heating or cooling the DUT 1 placed thereon. The heating / cooling plate 11 includes, for example, a heater such as a heater and a cooler such as a Peltier cooler.
The temperature control device 12 is a device that controls the amount of heating or cooling of the DUT 1 by the heating / cooling plate 11 according to a preset time schedule of heating amount or cooling amount.
When it is desired to measure the thermal deformation of the DUT 1 when the DUT 1 is heated or cooled from the outside according to a predetermined time schedule, or the ambient temperature environment is maintained in a predetermined state, When it is desired to measure the thermal deformation of the DUT 1, the heating / cooling plate 11 and the temperature control device 12 are used.

The thermo camera 20 is an infrared camera that captures a thermal image of the DUT 1. The thermal image data obtained by the thermo camera 20 is data in which temperature data is set for each pixel. That is, the thermo camera 20 is a temperature sensor that measures the temperature distribution of the DUT 1.
The two visible cameras 21 are normal cameras that are arranged at different positions and capture visible images of the device under test 1 from different directions. The visible image data obtained by the visible camera 21 is normal image data in which luminance data is set for each pixel. The visible camera 21 is, for example, a CCD camera or a CMOS camera.
The imaging ranges of the thermocamera 20 and the two visible cameras 21 are set so as to include temperature and shape change observation sites in the DUT 1 in each imaging range. Therefore, in addition to the case where the thermo camera 20 and the two visible cameras 21 are arranged so that the entire object to be measured 1 is included in each imaging range, the object to be measured including the observation part. There is a case where only a part of 1 is arranged so as to be included in each imaging range in common.
For convenience, when distinguishing each of the two visible cameras 21, one is referred to as a first visible camera 21a and the other is referred to as a second visible camera 21b.

The time monitoring device 31 is a device that generates a predetermined synchronization signal Strg in time series by using a predetermined time as a trigger according to a preset time schedule. The visible image sampling device 41 is an example of the synchronization signal generating unit.
FIG. 2 is an example of a time chart of the synchronization signal Strg generated by the time monitoring device 31.
The example shown in FIG. 2 is an example in which the time monitoring device 31 generates the synchronization signal Strg every time a preset time Δτ elapses. In addition to the case where the time Δτ is constant every time, it is also conceivable that a different time is set according to the number of generations of the synchronization signal Strg.
The time monitoring device 31 is connected to the trigger signal generation timing external input device 30. Using this trigger signal generation timing external input device 30, for example, a human can generate a trigger at an arbitrary timing separately from the time schedule or by canceling the time schedule. As the trigger signal generation timing external input device 30, for example, a device such as a button-type switch or a keyboard for manually inputting the generation timing of the synchronization signal Strg is adopted.

The visible image sampling device 41 and the thermal image sampling device 51 are synchronized with the synchronization signal Strg, and the two visible images having the same imaging time point from the two visible cameras 21 and the one thermocamera 20. And a combination of the thermal image data.
That is, the visible image sampling device 41 samples data of two types of visible images having the same imaging time point from the two visible cameras 21 in synchronization with the synchronization signal Strg generated in time series. Is output to the three-dimensional shape calculation device 43. For example, the visible image sampling device 41 sequentially acquires the visible image data from the two visible cameras 21 with a very short period, and only the visible image data acquired when the synchronization signal Strg is generated. The result is output to the three-dimensional shape calculation device 43. Alternatively, the visible image sampling device 41 acquires two types of visible image data simultaneously from the two visible cameras 21 every time the synchronization signal Strg is generated, and uses the data as the three-dimensional shape calculation device 43. Output to.
In addition, the thermal image sampling device 51 samples data of the thermal image from the thermo camera 20 at the same time as the two types of visible images in synchronization with the synchronization signal Strg generated in time series. The data is output to the thermal deformation analysis computer 61. The thermal image sampling device 51 also samples the thermal image data in the same manner as the visible image sampling device 41.
Usually, the sampling of the visible image data and the thermal image data is performed simultaneously. However, if there is a time difference between imaging and data output between the visible camera 21 and the thermocamera 20, the visible image sampling device 41 and the thermal image sampling device 51 are canceled so that the time difference is canceled out. The timing of data sampling is adjusted in advance.
As described above, in the thermal deformation measuring device X1, a plurality of combinations of the visible image data and the thermal image data having the same imaging time point are sequentially sampled based on the synchronization signal Strg.

The visible image display 42 is a display that displays an image (the visible image) based on the data of the visible image acquired from the visible camera 21 by the visible image sampling device 41.
The thermal image display 52 is a display that displays an image (the thermal image) based on the thermal image data acquired from the thermo camera 20 by the thermal image sampling device 51.

In addition, the three-dimensional shape calculation device 43 is configured so that the position of one or a plurality of observation parts on the surface of the measurement object 1 in the three-dimensional coordinate system at each imaging time point, that is, the measurement object 1 that changes as needed. This is a device for calculating the three-dimensional shape. The calculation is performed by performing stereo image processing and digital image correlation method image processing based on the two types of visible image data sampled by the visible image sampling device 41.
The thermal deformation analysis computer 61 generates three-dimensional thermal deformation data in which the position and temperature data of the observation region in the three-dimensional coordinate system for each imaging time point are associated, and the data is stored in the hard disk drive. Or the like.
More specific contents of the processing executed by the three-dimensional shape calculation device 43 and the thermal deformation analysis computer 61 will be described later.
The data input device 62 is an operation input unit such as a keyboard and a mouse connected to the thermal deformation analysis computer 61.
The analysis image display 63 is a display for displaying data input to the thermal deformation analysis computer 61 and the calculation result of the thermal deformation analysis computer 61.

Next, an example of the three-dimensional thermal deformation measurement process executed by the three-dimensional shape calculation device 43 and the thermal deformation analysis computer 61 will be described with reference to the flowchart shown in FIG. Steps S1, S2,... Shown in FIG. 3 and below are identification codes of a plurality of procedures (steps) executed in the three-dimensional thermal deformation measurement process.
In the three-dimensional thermal deformation measurement process, first, the thermal deformation analysis computer 61 obtains two pieces of visible image data before starting measurement performed in response to the generation of the synchronization signal Strg. The observation site setting process and the marking process for each observation site are executed (S1).

That is, in step S1, the thermal deformation analysis computer 61 sets one or a plurality of observation parts in the object to be measured 1 with respect to the data of the visible image obtained from the predetermined specific visible camera 21. Processing and marking processing for each observation site are executed. Here, the marking process is a process of setting identification information of the observation site for the coordinate data corresponding to each of the observation sites in the visible image data.
In the following description, for the sake of convenience, it is assumed that the specific visible camera 21 is the first visible camera 21a. Further, the visible image obtained from the first visible camera 21a is referred to as a main visible image, and the visible image obtained from the second visible camera 21b is referred to as a sub-visible image. It is assumed that the setting process and the marking process are performed. For example, the thermal deformation analysis computer 61 sets the observation site coordinate information in accordance with an operation input to the data input device 62 by the user while displaying the main visible image on the analysis display 63. For example, the thermal deformation analysis computer 61 sets one or more representative points on the surface of the DUT 1 as the observation site in accordance with an operation input from a mouse or the like. By setting a large number of observation sites in a range over the entire image of the device under test 1 in the main visible image, the entire device under test 1 becomes the observation site.
The main visible image data to be displayed on the analysis display 63 is transmitted to the thermal deformation analysis computer 61 through the visible image sampling device 41 and the three-dimensional shape calculation device 43.

On the other hand, the three-dimensional shape calculation device 43 manages the data of the visible image by dividing it into subsets that are data in a small region in which the entire region of the visible image is divided into a large number. Therefore, in step S1, the three-dimensional shape calculation device 43 performs automatic classification processing of the subset on the data of the main visible image. The size of the subset and the amount of overlap between the subsets are set in advance in the three-dimensional shape calculation device 43. For example, the size of the subset is about 21 pixels in the X-axis direction × 21 pixels in the Y-axis direction, and the overlap amount is about 50% of the size of the subset.
After the process of step S1, the three-dimensional shape calculation device 43 and the thermal deformation analysis computer 61 perform the following measurement processes (S2 to S9) according to the generation of the synchronization signal Strg.
First, the three-dimensional shape calculation device 43 and the thermal deformation analysis computer 61 obtain the data of the two visible images and the thermal images having the same imaging time point (S2). The data of the visible image and the thermal image acquired in step S2 are data sampled by the visible image sampling device 41 and the thermal image sampling device 51 in response to the first generation of the synchronization signal Strg.

（１）式において，ｎ及びｍは，前記サブセットＷ_L及び前記サブセットＷ_RのＹ軸方向及びＸ軸方向各々の画素数，Ｉ_L(ｉ，ｊ)及びＩ_R(ｉ，ｊ)は，前記サブセットＷ_L及び前記サブセットＷ_R各々における点（ｉ，ｊ）の輝度である。
さらに，前記相互相関値Ｃ１を両サブセットの画像の明るさの平均振幅によって正規化した正規化値Ｃ₂は，次の（２）式により求められる。

一方，次の（３）式は，前記サブセットＷ_Lと前記サブセットＷ_Rとの間の相互相関値を明るさの平均値によって正規化した正規化値Ｃ₃を算出する式である。

（３）式において，μ_L，μ_Rは，前記サブセットＷ_L及び前記サブセットＷ_R各々の領域内の輝度の平均値である。
前記三次元形状演算装置４３は，相互相関値の正規化値Ｃ₂又はＣ₃を，前記サブセットＷ_Lと前記サブセットＷ_Rとの輝度相関値として算出し，その相互相関値が最大となる前記サブセットＷ_Rの位置を求める。
前記サブセットＷ_Rの移動範囲は，予め設定された制約条件の範囲内とする。その制約条件は，エピポーラ拘束条件や視差の連続性に基づく拘束条件等の周知の条件が設定されるが，ここでは説明を省略する。
また，画素単位を下回る高精度の対応点位置算出を行なうため，前記サブセットＷ_Rをまず粗く移動させてそれぞれの位置で前記輝度相関値を求めた後，２次曲面フィッティングやニュートン・ラフソン法などのよく知られた近似計算手法により，前記輝度相関値が最大となる前記サブセットＷ_Rの位置を求める方法もある。
以上のようにして前記輝度相関値が最大となる前記サブセットＷ_Rの位置が決定されれば，その中央点を前記対応点Ｐ2として決定する。
なお、前記輝度相関値Ｃ₂又はＣ₃を精度よく求めるために，通常，前記被測定物１の表面には，予め不規則な模様，例えば，斑点模様が形成されている。
前記主可視画像上の点Ｐ1及びそれに対応する前記副可視画像上の点Ｐ2が求められれば，三角測量の原理を適用することにより，前記被測定物１上の点Ｒの三次元座標が求められる。これを，前記ステップＳ１においてマーキング処理を施された前記主可視画像上のサブセットＷ_Lの全てに対して実行すれば，前記被測定物１における１つ又は複数の前記観測部位の三次元形状が求まる。
また，前記三次元形状は，前記第１可視カメラ２１ａから得られる前記主可視画像との間で座標の対応関係が既知である。そこで，ステップＳ３において，前記三次元形状演算装置４３は，前記主可視画像上の座標と前記三次元形状との関係を，前記熱変形解析用計算機６１へ伝送する。 Next, the three-dimensional shape calculation device 43 performs a three-dimensional shape for an image region including at least the observation site in the main visible image by stereo image processing based on the data of the two visible images obtained in step S2. (S3: Three-dimensional shape data calculation procedure).
A method of calculating three-dimensional shape data by the stereo image processing is well known, but the processing procedure will be briefly described below.
The three-dimensional shape calculation device 43 obtains the three-dimensional shape of the DUT 1 from the two types of visible images based on the principle of triangulation. For the visible camera 21 for obtaining the visible image, the camera internal variables such as the direction of the optical axis and the focal length and the distance between the two visible cameras 21a and 21b (referred to as the baseline distance) are calibrated in advance. Sought by. At this time, for the point R on the DUT 1, the point P1 on the main visible image captured by the first visible camera 21a and the sub-visible image captured by the second visible camera 21b. By associating with the point P2 on the image, the three-dimensional coordinates of the point R on the DUT 1 are obtained by applying the principle of triangulation.
The association between the point P1 on the main visible image and the point P2 on the sub-visible image is performed by calculating a luminance correlation value between the subsets. That is, the point P1 on the main visible image is set as the center point of the subset W _L arranged on the main visible image, and the luminance of both is sequentially changed while moving the subset W _R of the same size on the sub-visible image. calculating a correlation value, the center point of said subset W _R of the brightness correlation value is maximized, the corresponding point P2 on the secondary visual images.
The luminance correlation value between the subset W _L and the subset W _R is calculated based on, for example, the following expressions (1) to (3). Here, it is assumed that the subset W _L and the subset W _R are both rectangular regions in the range of vertical n pixels × horizontal m pixels.
First, the cross-correlation value C _{1 that} is an index of the degree of similarity between the subset W _L and the subset W _R is obtained by the following equation (1).

In the equation (1), n and m are the numbers of pixels in the Y-axis direction and the X-axis direction of the subset W _L and the subset W _R , and I _L (i, j) and I _R (i, j) are: The luminance of the point (i, j) in each of the subset W _L and the subset W _R.
Further, the normalized value C ₂ normalized by the mean amplitude of the brightness of the cross-correlation value C1 between both subsets image is obtained by the following equation (2).

Meanwhile, the following equation (3) is an equation for calculating a normalized value C ₃ normalized by the cross-correlation value average brightness value between the subset W _R and the subset W _L.

In the equation (3), μ _L and μ _R are average values of luminance in the respective regions of the subset W _L and the subset W _R.
The three-dimensional shape calculation device 43 calculates the normalized value C ₂ or C ₃ of the cross-correlation value as the luminance correlation value between the subset W _L and the subset W _R, and the cross-correlation value is maximized. The position of the subset W _R is obtained.
Moving range of said subset W _R is within a range of preset constraints. As the constraint conditions, known conditions such as epipolar constraint conditions and constraint conditions based on continuity of parallax are set, but description thereof is omitted here.
Moreover, to perform the corresponding point position calculation precision below the pixel unit, wherein after obtaining the subset W _R is first coarsely moving said luminance correlation value at each position by, quadratic surface fitting and Newton-Raphson method or the like by well-known approximation method, the luminance correlation value is also a method for determining the position of said subset W _R becomes maximum.
If the subset W position of _R is determined that the luminance correlation value as described above is maximized, it determines its center point as the corresponding point P2.
In order to obtain the luminance correlation value C ₂ or C ₃ with high accuracy, an irregular pattern, for example, a spot pattern is usually formed on the surface of the DUT 1 in advance.
If the point P1 on the main visible image and the corresponding point P2 on the subvisible image are obtained, the three-dimensional coordinates of the point R on the object to be measured 1 are obtained by applying the principle of triangulation. It is done. This, if executed for all the subsets W _L on the main visible image or marking process in step S1, one or more three-dimensional shape of the observation site in the DUT 1 I want.
The three-dimensional shape has a known coordinate correspondence with the main visible image obtained from the first visible camera 21a. Therefore, in step S3, the three-dimensional shape calculation device 43 transmits the relationship between the coordinates on the main visible image and the three-dimensional shape to the thermal deformation analysis computer 61.

By the way, the shape measuring method by the stereo image processing can be applied to measuring the three-dimensional shape of the DUT 1 using three or more visible images.
That is, when using three or more visible images, the above-described subsets are associated for each combination of one main visible image and two or more sub-visible images, and the result of the association is obtained. The three-dimensional shape data is calculated by integration. Various methods of integration are known.
Even when the observation site is in a blind spot region from one of the visual field directions of the two visible cameras 21 and correct shape measurement cannot be performed, the correct shape can be obtained by using three or more visible cameras 21. Measurement is possible. Further, even when an error may occur in the association of the subsets between the two visible images, the subsets are associated for each combination of two or more of the visible images and the results are integrated. Thus, it is possible to eliminate or alleviate errors in association of the subsets with respect to some combinations of the visible images.
However, normally, the apparatus can often be adjusted so that the observation site does not enter the blind spot of the two visible cameras 21. It is also possible to give an irregular pattern to the surface of the DUT 1 so that no error occurs in the association of the subsets. Therefore, usually, two visible cameras 21 are often sufficient.

By the way, the correspondence relationship between the three-dimensional shape and the thermal image obtained from the thermo camera 20 is determined by the coordinate between the main visible image and the thermal image by the thermal deformation analysis computer 61. It is calculated based on the correspondence relationship. The coordinate relationship between the main visible image and the thermal image can be generally expressed by a coordinate transformation matrix called homography. This homography can be calculated by calculation based on the result of a prior calibration measurement.
The calibration measurement is performed as follows, for example.
First, a plurality of temperature differences whose positions can be recognized through image processing or vision are arranged in the imaging range of the thermo camera 20 and the first visible camera 21a. This temperature difference is a point where the temperature is locally different from the surroundings, but the temperature may be higher or lower than the surroundings. This temperature difference is a point having a temperature difference from the surroundings to such an extent that it can be sufficiently visually recognized by the thermo camera 20. Further, the temperature difference is that the luminance or color in the visible light region is different from the surroundings so that the first visible camera can sufficiently recognize the temperature. As a temperature difference with the above conditions, for example, a sufficiently heated nichrome wire, miniature light bulb, or the like is arranged at a predetermined position.
Next, a plurality of the temperature differences are imaged by the thermo camera 20 and the first visible camera 21a, and each of the temperature difference images is obtained between the obtained thermal image and the main visible image. Is associated. This association may be performed manually through human vision or automatically by predetermined image processing.
The value of each element of the homography matrix is calculated from the result of the association. The calculation method is well-known and will not be described here. In order to calculate the homography matrix, the number of temperature differences should be at least 8 or more.
The information of the homography matrix obtained as described above is set and stored in advance in the thermal deformation analysis computer 61.

In step S4, the following processing is executed by the thermal deformation analysis computer 61.
That is, in step S4, the thermal deformation analysis computer 61 specifies the observation site on the main visible image based on the result of the marking process performed in step S1, and the three-dimensional shape calculation device 43. The three-dimensional shape of the observation site is calculated based on the data transmitted from the.
Next, the thermal deformation analysis computer 61 uses the homography matrix to convert the position of each point on the thermal image obtained in step S2 to the position on the main visible image, and then the third order. Based on the data transmitted from the original shape arithmetic unit 43, the three-dimensional shape is associated with the temperature data obtained from the thermal image. The three-dimensional shape / temperature correspondence data thus obtained is recorded in a data storage device such as a hard disk drive.

  After the processes of steps S1 to S4 described above are performed, the three-dimensional shape calculation device 43 and the thermal deformation analysis computer 61 determine whether a predetermined termination condition is satisfied (S9), and terminate it. Steps S5 to S8 shown below are executed until the condition is satisfied. Steps S5 to S8 are executed each time the second and subsequent synchronization signals Strg are generated and the thermal image and the visible image data are sampled.

  First, the three-dimensional shape calculation device 43 and the thermal deformation analysis computer 61 respectively acquire the two visible images and the thermal image data at new imaging points in the same manner as in step S2 (S5). The data of the visible image and the thermal image acquired in step S5 is data that is sequentially sampled in accordance with the generation of the second and subsequent synchronization signals Strg.

（４）式において，ｎ及びｍは，前記サブセットＷ_L1及び前記サブセットＷ_L2のＹ軸方向及びＸ軸方向各々の画素数，Ｉ_L1(ｉ，ｊ)及びＩ_L2(ｉ，ｊ)は，前記サブセットＷ_L1及び前記サブセットＷ_L2各々における点（ｉ，ｊ）の輝度，ｕ_L1，ｕ_L2は，前記サブセットＷ_L1及び前記サブセットＷ_L2各々の領域内の輝度の平均値である。
前記三次元形状演算装置４３は，（４）式に基づく相互相関値Ｃ₄を，前記サブセットＷ_L1と前記サブセットＷ_L2との輝度相関値として算出し，その輝度相関値が最大となる前記サブセットＷ_L2の位置を求める。ここで，前記ステップＳ３と同様に，２次曲面フィッティング法やニュートンラフソン法などの既知の近似計算手法を用いて，前記輝度相関値が最大となる前記サブセットＷ_L2の位置を高精度で求めることも考えられる。
前記輝度相関値が最大となる前記サブセットＷ_L2の位置が求まると，前記サブセットＷ_L1と前記サブセットＷ_L2との画像上の座標の差を求める。通常は，前記サブセットＷ_L1の中央点の座標と，前記サブセットＷ_L2の中央点の座標との差を求める。即ち，前回撮像時点から今回撮像時点までの間に前記サブセットＷ_L1に対応する前記観測部位が変位したものとして，前記サブセットＷ_L1と前記サブセットＷ_L2との画像上の座標の差を画像座標上の変位量として算出する。 Next, the three-dimensional shape calculation device 43 performs a pattern tracking process by the digital image correlation method based on the data of the main visible image at the previous and current imaging times obtained from the first visible camera 21a. Thus, the displacement amount on the visible image of the image of the observation site in the main visible image at each imaging time point is calculated (S6: displacement amount calculating procedure).
The pattern tracking process by the digital image correlation method is well known, but the process procedure will be briefly described below.
The three-dimensional shape calculation device 43 associates the subsets having a high luminance correlation value between the main visible image data at the previous and current imaging times.
That is, the three-dimensional shape operation unit 43, with respect to the subset W _L1 on the main visible image obtained by the imaging time of the latest, the subset of the resulting the main visible image in this imaging point W _L2 the while moving in an image, the brightness correlation value between the subset W _L1 and the subset W _L2, is calculated based on the following equation (4) for obtaining the cross-correlation value C _4. Here, it is assumed that each of the subset _WL1 and the subset _WL2 is a rectangular region having a range of vertical n pixels x horizontal m pixels.

In equation (4), n and m are the numbers of pixels in the Y-axis direction and the X-axis direction of the subset W _L1 and the subset W _L2 , and I _L1 (i, j) and I _L2 (i, j) are the brightness of the subset W _L1 and the point in the subset W _L2 each _{(i, j), u L1} , u L2 is the average value of the luminance of said subset W _L1 and the subset W _L2 each region.
The three-dimensional shape calculation device 43 calculates the cross-correlation value C ₄ based on the equation (4) as a luminance correlation value between the subset W _L1 and the subset W _L2, and the subset having the maximum luminance correlation value is calculated. _Find the position of WL2. Here, similarly to the step S3, using the known approximation method such as quadratic surface fitting method and Newton Raphson method, we obtain the position of said subset W _L2 of the luminance correlation value is maximized with high precision that Is also possible.
The position of said subset W _L2 of the luminance correlation value is maximized is obtained, obtaining the difference between the coordinates of the image of the subset W _L2 and the subset W _L1. Typically, obtains the central coordinates of the subset W _L1, the difference between the coordinates of the center point of said subset W _L2. That is, as the observation region corresponding to said subset W _L1 until this imaging point from the previous imaging time is displaced, said subset W _L1 and the subset W image on the coordinate difference of the image coordinates of the _L2 The amount of displacement is calculated.

Further, the three-dimensional shape calculation device 43, like step S3, is a stereo image based on the data of the two visible images (the main visible image and the sub-visible image) at the time of the current imaging obtained in step S5. By processing, three-dimensional shape data is calculated for an image region including at least the observation site in the main visible image (S7: three-dimensional shape data calculation procedure).
In step S7, the three-dimensional shape calculation device 43 associates the position of each point on the main visible image with the three-dimensional shape data obtained in step S7 in the same manner as in step S3. It transmits to the computer 61 for analysis. The three-dimensional shape calculation device 43 also transmits the displacement amount on the visible image of the image of the observation site obtained in step S6 to the thermal deformation analysis computer 61.

Next, similarly to step S4, the thermal deformation analysis computer 61 performs the processing from the three-dimensional shape calculation device 43 based on the result of the marking process obtained in step S1 for the image region including the observation region. The observation site is extracted from the transmitted main visible image data, and the corresponding three-dimensional shape is extracted.
Next, the thermal deformation analysis computer 61 calculates the observation site from the amount of displacement of the image of the observation site on the visible image, which is another information transmitted from the three-dimensional shape calculation device 43. Find the 3D displacement.
Further, the thermal deformation analysis computer 61 converts the temperature data of each point on the thermal image obtained in step S5 into temperature data at a position on the main visible image using the homography matrix. Further, the temperature data is made to correspond to the three-dimensional shape data. These series of data, that is, the three-dimensional shape data, the three-dimensional displacement amount, and the temperature data of the observation region are the shapes and three-dimensional positions associated by the thermal deformation analysis computer 61. It is recorded in a data storage device such as a hard disk drive as displacement / temperature correspondence data (S8: thermal deformation data recording procedure).
As described above, the three-dimensional shape calculation device 43 and the thermal deformation analysis computer 61 each time data of the new thermal image and visible image are sampled until the termination condition is satisfied. The processes in steps S5 to S8 are repeated for new sampling data.
The end condition is, for example, that the sampled data of the thermal image and the visible image cannot be obtained for a predetermined time-up time or more, and the number of times the processes in steps S5 to S8 are set in advance. It is thought that it was repeated only.

Here, the procedure of step S8 performed by the thermal deformation analysis computer 61, which is an example of the calculation means, is referred to as a thermal deformation data recording procedure, and the data recorded in the data storage device at that time is referred to as three-dimensional thermal deformation data. .
The thermal deformation data recording procedure includes the preset homography, the data of the thermal image for each imaging time point sampled in step S5, and the three-dimensional shape for each imaging time point calculated in step S7. The three-dimensional thermal deformation data is stored in the data storage device on the basis of the above-described data and the displacement amount on the visible image of the image of the observation site calculated at the imaging time point calculated in step S6. is there. Further, the three-dimensional thermal deformation data is data in which position, displacement amount, and temperature data in the three-dimensional coordinate system at each imaging time point of the observation site are associated with each other.
As described above, the homography is an example of coordinate conversion information between the three-dimensional shape and the thermal image.

As described above, the thermal deformation measuring apparatus X1 sequentially samples combinations of data of two visible images having the same imaging time point and different viewing directions based on the synchronization signal Strg (S2, S5). Further, the thermal deformation measuring device X1 performs the main visible image for each imaging time point by stereo image processing (S3, S7) based on the data of the visible image and pattern tracking processing (S6) by the digital image correlation method. Data in which the position of each point above is associated with the three-dimensional coordinates, and displacement amount data on the main visible image of the image of the observation site are calculated. Both data are information on the three-dimensional shape change of the DUT 1 according to the transition at the time of imaging.
Further, the thermal deformation measuring device X1 samples the thermal image data representing the temperature distribution of the DUT 1 at the same timing as the sampling of the visible image data based on the synchronization signal Strg ( S2, S5). The correspondence relationship between the three-dimensional shape and the thermal image can be obtained by calculation using the homography.
Then, the thermal deformation measuring device X1 calculates temperature data corresponding to the position of the observation part in the three-dimensional shape at the corresponding imaging time point from the data of the thermal image at each imaging time point based on known information. Is specified (S4, S8). In addition, the three-dimensional displacement amount of the observation site is also associated (S8). In other words, the three-dimensional thermal deformation data recorded by the processes of steps S4 and S8 is an accurate correspondence between the temperature of the observation site and the three-dimensional position and displacement amount of the DUT 1 that changes every moment. That is, the data shows the accurate thermal deformation characteristics of the DUT 1.

The thermal deformation analysis computer 61 has various thermal deformation analysis functions related to the device under test 1 based on the three-dimensional thermal deformation data.
For example, the computer 61 for thermal deformation analysis may use the analytical image display to show how the three-dimensional shape or cross-sectional shape of the DUT 1 changes with the progress of the imaging based on the three-dimensional thermal deformation data. 63 has a function of displaying an animation. At that time, in the animation display, the thermal deformation analysis computer 61 colors a portion corresponding to the observation site according to the temperature. The animation display screen is a screen on which the relationship between the temperature of the DUT 1 and the three-dimensional deformation can be grasped at a glance.
The thermal deformation analysis computer 61 also has a function of calculating a linear expansion coefficient for each observation region from the relationship between the temperature of the observation region and the displacement from the original shape in the three-dimensional heat deformation data. .
The thermal deformation analysis function described above has the effect of improving the efficiency of designing products such as electronic devices whose temperature and shape change in various patterns.

Next, the thermal deformation measuring device X2 according to the second embodiment of the present invention will be described with reference to FIGS. The thermal deformation measuring device X2 is an application example of the thermal deformation measuring device X1, and has the same configuration and function as the thermal deformation measuring device X1 except that the generation of the synchronization signal Strg is different. Hereinafter, only the portions of the thermal deformation measuring device X2 that are different from the thermal deformation measuring device X1 will be described. In FIG. 4, the same components as those shown in FIG. 1 are given the same reference numerals as those shown in FIG.
As shown in FIG. 4, the thermal deformation measuring device X2 includes a temperature monitoring device 32 as a device for generating the synchronization signal Strg, instead of the time monitoring device 31 in the thermal deformation measuring device X1.
The temperature monitoring device 32 sequentially acquires part or all of the thermal image data sequentially obtained from the thermo camera 20 with a sufficiently short period via the thermal image sampling device 51. Further, the temperature monitoring device 32 generates the synchronization signal Strg in time series, triggered by the fact that the thermal image data obtained in a short cycle reaches a predetermined temperature state.

FIG. 5 is an example of a time chart of the synchronization signal Strg generated by the temperature monitoring device 32.
In the example shown in FIG. 5, each time the predetermined representative temperature Tx based on the thermal image data is changed by the preset temperature change width ΔT by the temperature monitoring device 32, the synchronization signal Strg is changed. This is an example.
It should be noted that the temperature change width ΔT may be set differently depending on the number of generations of the synchronization signal Strg, in addition to the case where the temperature change width ΔT is constant every time. It is also conceivable that the synchronization signal Strg is generated every time the representative temperature Tx reaches one of a plurality of preset absolute temperatures.

  In some cases, it is desired to measure the thermal deformation of the DUT 1 every time a predetermined temperature change occurs. In some cases, it is desired to measure the thermal deformation of the DUT 1 with different temperature resolution for each temperature range. If the thermal deformation measuring device X2 is used in such a case, only necessary measurement data can be efficiently obtained.

Next, a thermal deformation measuring device X3 according to a third embodiment of the present invention will be described with reference to FIGS. The thermal deformation measuring device X3 is an application example of the thermal deformation measuring device X1, and has the same configuration and function as the thermal deformation measuring device X1 except that the method of generating the synchronization signal Strg is different. Hereinafter, only the portions of the thermal deformation measuring device X3 that are different from the thermal deformation measuring device X1 will be described. In FIG. 6, the same constituent elements as those shown in FIG. 1 are denoted by the same reference numerals as those shown in FIG.
As shown in FIG. 6, the thermal deformation measuring device X3 includes a deformation monitoring device 33 as a device for generating the synchronization signal Strg instead of the time monitoring device 31 in the thermal deformation measuring device X1.
The deformation monitoring device 33 sequentially acquires a part or all of the three-dimensional deformation amount of the observation site sequentially obtained from the thermal deformation analysis computer 61 at a sufficiently short period. Further, the deformation monitoring device 33 sequentially calculates a predetermined deformation amount based on the data of the three-dimensional deformation amount obtained in a short cycle, and the synchronization signal is triggered when the deformation amount reaches a predetermined deformation state. Strg is generated in time series.

FIG. 7 is an example of a time chart of the synchronization signal Strg generated by the deformation monitoring device 33.
In the example shown in FIG. 7, the deformation amount α sequentially calculated based on the three-dimensional displacement amount data obtained by the thermal deformation analysis computer 61 is changed by a preset deformation amount change width Δα. In this example, the synchronization signal Strg is generated every time.
The deformation monitoring device 33 calculates, for example, a change width of the position of one observation portion set in advance in the three-dimensional direction as the deformation amount α. In that case, the deformation monitoring device 33 extracts only the data of one specific part out of the three-dimensional displacement amount of the observation part obtained from the thermal deformation analysis computer 61 and uses it as the deformation quantity α. calculate.
It is also conceivable that the deformation monitoring device 33 calculates, as the displacement amount α, an average value, a maximum value, or the like of a change width of a plurality of preset positions of the observation parts in the three-dimensional direction.
Further, the deformation monitoring device 33 may include a displacement amount on the main visible image of one preset image of the observed region or a plurality of preset images of the observed region on the main visible image. It is also conceivable to calculate an average value or maximum value of the displacement amount as the deformation amount α.
In addition to the case where the deformation amount variation width Δα is constant each time, it is also conceivable that different values are set according to the number of generations of the synchronization signal Strg. It is also conceivable that the synchronization signal Strg is generated every time the deformation amount α reaches one of a plurality of preset absolute values.

In the measurement of the thermal deformation of the device under test 1, it is sometimes desired to measure the data of the heat deformation every time a predetermined shape change occurs in the device under test 1. In some cases, it is desired to measure the thermal deformation of the DUT 1 with different resolutions depending on the magnitude of the shape change. If the thermal deformation measuring device X3 is used in such a case, only necessary measurement data can be efficiently obtained.
In the measurement of the thermal deformation of the device under test 1, the operator may want to measure the data of the heat deformation at an arbitrary timing while monitoring the temperature or shape change state of the device under test 1.
In such a case, for example, in the thermal deformation measuring device X1, instead of the time monitoring device 31, the thermal deformation analysis computer 61 responds to an operation input for generating a predetermined signal to the data input device 62. It is also conceivable to generate the synchronization signal Strg. The operation input is, for example, a mouse button click operation or a predetermined operation key press operation on the keyboard.

The function sharing by each component in each embodiment described above is merely an example, and various other embodiments can be considered.
For example, the function of the visible image sampling device 41 is incorporated in the three-dimensional shape calculation device 43, and the function of the time monitoring device 31 is incorporated in the three-dimensional shape calculation device 43 or the thermal analysis computer 61. Etc. are considered.
It is also conceivable that the function of the temperature monitoring device 32 is incorporated in the thermal image sampling device 51, and that the functions of the visible image sampling device 41 and the deformation monitoring device 33 are incorporated in the three-dimensional shape calculation device 43. It is done.
It is also conceivable that three or more visible cameras 21 are provided, and that the thermo camera 20 and the plurality of visible cameras 21 are arranged in a positional relationship different from the state shown in FIG.

  The present invention can be used in a method and apparatus for measuring a three-dimensional shape change according to a temperature change of an object to be measured.

X1, X2, X3: Thermal deformation measuring device 1: Object to be measured 11: Heating / cooling plate 12: Temperature control device 20: Thermo camera 21: Visible camera 30: Trigger signal generation timing external input device 31: Time monitoring device 32: Temperature monitoring device 33: Deformation monitoring device 41: Visible image sampling device 42: Visible image display 43: Three-dimensional shape calculation device 51: Thermal image sampling device 52: Thermal image display 61: Thermal deformation analysis computer 62: Data input device 63 : Analysis image display S1, S2,...: Processing procedure (step)

Claims (7)

A thermal deformation measurement method for measuring a three-dimensional deformation according to a temperature change of an object to be measured,
A synchronization signal generation procedure for generating a synchronization signal in time series by the signal generation means;
In synchronization with the synchronization signal, a thermal camera that captures a thermal image of the object to be measured and a plurality of visible cameras that capture a visible image of the object to be measured at different positions, respectively, An image data sampling procedure for sampling a combination of the visible image data of
A three-dimensional shape data calculation procedure for calculating three-dimensional shape data for each imaging time point through stereo image processing based on a combination of a plurality of the visible image data for each imaging time point by an arithmetic means;
Through the pattern tracking process in the digital image correlation method based on the data of the plurality of visible images with different imaging time points obtained from the specific visible camera by the calculation means, the measured object in the visible image at each imaging time point A displacement amount calculating procedure for calculating a displacement amount on the visible image of an image of one or a plurality of observation parts of the object;
Information on the coordinate conversion between the preset three-dimensional shape and the thermal image, the thermal image data and the three-dimensional shape data calculated at each imaging time point obtained by the image data sampling procedure by the calculation means Based on the three-dimensional shape data for each imaging time point obtained by the procedure and the displacement amount on the visible image of the image of the observation site for each imaging time point obtained by the displacement amount calculation procedure, A thermal deformation data recording procedure for storing, in a storage means, three-dimensional thermal deformation data in which data of a position and temperature in a three-dimensional coordinate system for each imaging time point of the observation site are associated;
A method of measuring thermal deformation, comprising:   The thermal deformation measurement method according to claim 1, wherein the synchronization signal generation procedure is a procedure for generating the synchronization signal in time series with an operation input through a predetermined operation unit as a trigger.   The thermal deformation measurement method according to claim 1, wherein the synchronization signal generation procedure is a procedure for generating the synchronization signal in time series with a predetermined time elapses as a trigger.   2. The thermal signal according to claim 1, wherein the synchronization signal generation procedure is a procedure for generating the synchronization signal in time series triggered by the arrival of the thermal image data sequentially obtained from the thermocamera at a predetermined temperature state. Deformation measurement method.   The synchronization signal is time-sequentially triggered by a change in the shape of the object to be measured calculated based on the data of the visible image sequentially obtained from the visible camera reaching a predetermined deformation state. The method for measuring thermal deformation according to claim 1, wherein the thermal deformation measurement method is a procedure generated in step 1. A thermal deformation measuring device for measuring a three-dimensional deformation according to a temperature change of an object to be measured,
Synchronization signal generating means for generating the synchronization signal in time series;
A thermo camera for capturing a thermal image of the object to be measured;
A plurality of visible cameras that capture visible images of the object to be measured at different positions;
Image data sampling means for sampling a combination of the thermal image and the data of the plurality of visible images having the same imaging time point from the thermo camera and the plurality of visible cameras in synchronization with the synchronization signal;
Three-dimensional shape data calculation means for calculating three-dimensional shape data for each imaging time point through stereo image processing based on a combination of data of the plurality of visible images for each imaging time point;
Through the pattern tracking process in the digital image correlation method based on the data of the plurality of visible images with different imaging time points obtained from the specific visible camera, the image of the object to be measured in the visible image at each imaging time point A displacement amount calculating means for calculating a displacement amount on the visible image;
Information on the coordinate conversion between the preset three-dimensional shape and the thermal image, the thermal image data obtained by the image data sampling means, and the three-dimensional shape data calculation means at each imaging time point. Further, based on the data of the three-dimensional shape for each imaging time point and the displacement amount on the visible image of the image of the observation site for each imaging time point obtained by the displacement amount calculation means, Thermal deformation data recording means for storing in the storage means three-dimensional thermal deformation data associated with position and temperature data in the three-dimensional coordinate system for each imaging time point;
A thermal deformation measuring device comprising:   Means for generating the synchronization signal in time series triggered by an operation input through a predetermined operation unit; means for generating the synchronization signal in time series triggered by elapse of a predetermined time Based on the data of the visible image sequentially obtained from the visible camera and means for generating the synchronization signal in time series triggered by the arrival of the thermal image data sequentially obtained from the thermo camera reaching a predetermined temperature state The thermal deformation measuring apparatus according to claim 6, wherein the apparatus is one of means for generating the synchronization signal in time series triggered by a calculated change in the shape of the object to be measured reaching a predetermined deformation state. .

Images