JP2023055585A - Information processing device, information processing system, information output method and program - Google Patents

Abstract

To specify an internal structure of an object without breaking the object.SOLUTION: An information processing device includes an image acquisition unit for acquiring a response image showing a time response of a change in a temperature distribution in an area including a place heated by irradiating an object with periodical light, and an output unit for outputting an image of a cross section structure of the object estimated by using the response image acquired by the image acquisition unit. Then, the information processing device can specify an internal structure of the object without breaking the object.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing device, an information processing system, an information output method, and a program.

In Patent Document 1, a power spectrum image obtained by binarizing a slice image of a resin molded product obtained by an X-ray CT method and applying a Fourier transform to this binarized image is used to perform resin molding. It is disclosed to analyze the orientation trend of the filler material in a portion of the article.

JP 2012-2547 A

For example, to identify the internal structure of an object such as carbon fiber reinforced plastic, destructive inspection such as X-ray CT, which requires cutting out, may be used. However, destructive inspection, for example, may require a long time and require a large-scale inspection apparatus.
An object of the present invention is to identify the internal structure of an object without destroying the object.

For this purpose, the technology disclosed in the present specification generates a response image showing the time response of changes in temperature distribution in a region including a portion heated by periodically irradiating an object with light. and an output unit configured to output an image of the cross-sectional structure of the object estimated using the response image acquired by the image acquisition unit.

Viewed from another point of view, the technology disclosed in this specification provides a structural image showing the internal structure of an object, and temperature in a region including a portion heated by periodically irradiating the object with light. an acquisition unit that acquires learning data that is a combination with a response image that indicates a time response of changes in distribution; a learning unit that learns a machine learning model using the learning data acquired by the acquisition unit; and an output unit that inputs the response image of the object to the learned machine learning model and outputs an estimated image of the internal structure of the object.
Here, the object may have a specific material and a base material that supports the material, and the estimated image may be an image showing the position of the material in the cross section of the object.
Also, the learning data may include a plurality of response images with different heating conditions for one object.
Also, the heating condition is preferably a heating frequency for heating one object.
Also, the heating condition is preferably a heating position for one target object.
Further, the heating condition is preferably a heating target surface of one target.
Also, the heating condition is preferably a heating area for one object.
Further, the learning data may include a plurality of response images in which temperature detection surfaces of one object are different.
Further, the machine learning model has a neural network structure including an encoder section having a plurality of convolution layers and receiving the response image, and a decoder section having a plurality of deconvolution layers and outputting the estimated image. , the features extracted in the convolutional layer should not be sent to the deconvolutional layer corresponding to the convolutional layer.

Viewed from another point of view, the technology disclosed in this specification provides a structural image showing the internal structure of an object, and temperature in a region including a portion heated by periodically irradiating the object with light. a storage unit that stores a machine learning model trained using learning data that is a combination of a response image showing a time response of a change in distribution, and inputting the response image of one target to the machine learning model, and an output unit that outputs information about the internal structure of the one object.
Here, a heating unit for periodically irradiating an object with light to heat the object and a time response of a change in temperature distribution in an area including a portion of the object heated by the heating unit are detected. another output unit that outputs information indicating the orientation of at least one material constituting the object based on the time response of the change in the temperature distribution detected by the detection unit; It is preferable to include a reception unit that receives, from a user, selection of information to be output from the information regarding the internal structure of the output unit and the information indicating the orientation of the other output unit.

Viewed from another point of view, the technology disclosed in this specification provides a response image showing the time response of changes in temperature distribution in a region including a portion heated by irradiating an object with periodic light. and outputting an image of the cross-sectional structure of the object estimated using the obtained response image.

Viewed from another point of view, the technology disclosed in this specification provides a structural image showing the internal structure of an object, and temperature in a region including a portion heated by periodically irradiating the object with light. a step of acquiring learning data that is a combination of a response image showing a time response of a change in distribution; a step of training a machine learning model using the acquired learning data; and outputting an estimated image of the internal structure of the object.

Viewed from another point of view, the technology disclosed in this specification provides a structural image showing the internal structure of an object, and temperature in a region including a portion heated by periodically irradiating the object with light. a step of storing a machine learning model trained using learning data that is a combination of a response image showing a time response of a change in distribution; and outputting information about the internal structure of one object.

Viewed from another point of view, the technology disclosed in this specification provides a computer with a time response of changes in temperature distribution in an area including a portion heated by irradiating an object with periodic light. and a function of outputting an image of the cross-sectional structure of the object estimated using the obtained response image.

Viewed from another point of view, the technology disclosed in this specification includes, in a computer, a structural image showing the internal structure of an object, and a portion heated by periodically irradiating the object with light. A function of acquiring learning data that is a combination with a response image that shows the time response of changes in temperature distribution in an area, a function of learning a machine learning model using the acquired learning data, and a function of learning the machine learning model and a function of inputting the response image and outputting an estimated image of the internal structure of the object.

Viewed from another point of view, the technology disclosed in this specification includes, in a computer, a structural image showing the internal structure of an object, and a portion heated by periodically irradiating the object with light. A function of storing a machine learning model trained using learning data that is a combination of a response image showing a time response of changes in temperature distribution in an area, and inputting the response image of one object into the machine learning model. and a function of outputting information about the internal structure of the one object.

According to the technology disclosed in this specification, it is possible to identify the internal structure of an object without destroying the object.

It is a schematic block diagram which shows the internal structure evaluation apparatus of this embodiment. It is a functional block diagram of a computer. It is the figure which showed the hardware configuration example of a computer. It is explanatory drawing which shows the measurement principle of an in-plane thermal diffusivity. It is explanatory drawing which shows the measurement principle of a thickness direction thermal diffusivity. (a) shows the direction of thermal diffusion in the object 1, and (b) shows the relationship between the thermal diffusion angle, thermal diffusivity, and fiber orientation density. It is a flowchart explaining operation|movement of an internal-structure evaluation apparatus. FIG. 4 is a diagram showing an example of a cyclic heating response image measured by a lock-in thermography cyclic heating method; It is a figure which shows an example of the target object used by this embodiment. 4A and 4B are a periodic heating response image and a cross-sectional structure image of an object according to the present embodiment; 4 is a conceptual diagram of a neural network structure used by the machine learning unit of this embodiment; FIG. FIG. 4 is an explanatory diagram of learning in the machine learning unit of the embodiment; It is an example of the estimation result by the machine learning part of this embodiment. FIG. 11 is an explanatory diagram of a heating surface and a detection surface of a modified example of the object 1; FIG. 11 is an explanatory diagram of a heating surface and a detection surface of a modified example of the object 1; It is explanatory drawing of the internal-structure evaluation apparatus of a modification.

Hereinafter, this embodiment will be described in detail with reference to the accompanying drawings.
<Configuration of internal structure evaluation device 100>

FIG. 1 is a schematic configuration diagram showing an internal structure evaluation device 100 of this embodiment.
First, the configuration of an internal structure evaluation device 100 to which the present embodiment is applied will be described with reference to FIG.

As shown in FIG. 1, the internal structure evaluation apparatus 100 of this embodiment includes a diode laser 10 that functions as a light source for heating the object 1, and a light guide section 20 that guides the laser light from the diode laser 10 to the object 1. , an infrared thermography (lock-in thermography) 30 provided facing the object 1, a computer 50 that receives a signal from the infrared thermography 30, and a period that generates a periodic signal and outputs it to the diode laser 10 and the computer 50 and a target signal generator 70 .

The light guide section 20 also includes a mirror 21 that reflects the laser light emitted from the diode laser 10, a beam expander 23 that expands the beam diameter of the laser light from the mirror 21, and a holder that holds the object 1. 24 and.
Then, in the internal structure evaluation apparatus 100 , the laser light emitted from the diode laser 10 is applied to the object 1 through the mirror 21 and the beam expander 23 . In this object 1, the region irradiated with the laser light is periodically heated. That is, a specific point (position) on the surface of the object 1 is spot-periodically heated.

The temperature of the object 1 periodically heated by the laser light of the diode laser 10 is measured from the rear surface of the object 1 by the infrared thermography 30 . Note that the infrared thermography 30 takes (measures) an infrared image of a predetermined range including a specific point that is periodically spot-heated by the diode laser 10 . That is, the infrared thermography 30 two-dimensionally measures the temperature response of the back surface of the object 1 . A periodic signal is input from the periodic signal generator 70 to the infrared thermography 30 . Temperature distribution data, which is temperature data measured by the infrared thermography 30 , is output to the computer 50 .

Together with the infrared thermography 30, the computer 50 continuously executes acquisition and calculation of infrared images based on a frame rate at predetermined intervals, and averages image data from the amount of temperature change that changes with the passage of time. create (lock-in method). To explain further, the data obtained by the infrared thermography 30 are arithmetically processed by the computer 50, and the direction (angle) from the heating point Hp (see FIG. 6(a) described later), thermal diffusivity, and orientation are determined. Calculated.
It should be noted that performing arithmetic processing by the computer 50 using the image data obtained by the infrared thermography 30, and calculating the direction (angle) from the heating point Hp of the object 1, thermal diffusivity, and orientation is referred to as "internal It is sometimes called "structural measurement".

In addition, the computer 50 of the present embodiment uses image data obtained by the infrared thermography 30 (for example, a periodic heating response image Ti described later) and a learned machine learning model (hereinafter referred to as a learned model). It is also possible to output the result of estimating the internal structure of the object 1. This content will be described in detail later.
Note that estimating the internal structure of the object 1 based on machine learning by the computer 50 using image data obtained by the infrared thermography 30 may be called "internal structure estimation".

In the following description, one direction along the surface of the object 1 in FIG. 1, that is, the horizontal direction in the drawing may be referred to as the x direction. Also, the vertical direction in FIG. 1 is sometimes referred to as the z-direction. Also, the depth direction of the paper surface of FIG. 1 is sometimes referred to as the y direction.

<Functional Configuration of Computer 50>
FIG. 2 is a functional configuration diagram of the computer 50. As shown in FIG.

Next, the functional configuration of the computer 50 to which this embodiment is applied will be described.
As shown in FIG. 2, the computer 50 to which the present embodiment is applied measures the phase lag distribution based on the temperature distribution data and the periodic signal input from the infrared thermography 30 (see FIG. 1). It includes a measurement unit 51 and a thermal diffusivity distribution calculator 52 that calculates the thermal diffusivity distribution based on the measured phase lag. Further, the computer 50 includes an orientation calculator 53 that calculates the orientation (described later) of the object 1 (see FIG. 1) based on the calculated thermal diffusivity distribution, and the calculation result of the thermal diffusivity distribution and the orientation. and a calculation result display unit 54 that displays the . The computer 50 also includes a machine learning unit 55 for estimating the internal structure of the object 1 based on the time response (phase lag distribution) of changes in the measured temperature distribution, and selection of output contents of the evaluation result of the object 1. and a selection accepting unit 56 that accepts.

<Hardware Configuration of Computer 50>
FIG. 3 is a diagram showing a hardware configuration example of the computer 50. As shown in FIG.

As shown in FIG. 3, the computer 50 includes a CPU (Central Processing Unit) 501 as computing means, and a main memory 503 and HDD (Hard Disk Drive) 505 as storage means. Here, the CPU 501 executes various programs such as an OS (Operating System) and application software. The main memory 503 is a storage area for storing various programs and data used for executing them. The HDD 505 is a storage area that stores input data for various programs, output data from various programs, and the like. These components of the computer 50 execute the functional configurations described with reference to FIG. 2 and the like.

The computer 50 has a communication interface (communication I/F) 507 for communicating with the outside such as the infrared thermography 30 . Programs executed by the CPU 501 (for example, programs for measuring the internal structure and estimating the internal structure) are stored in advance in the main memory 503, or stored in a storage medium such as a CD-ROM. It is also possible to provide it to the CPU 501 or provide it to the CPU 501 via a network (not shown).

<Configuration around object 1>
Next, the configuration of the object 1 and the peripheral configuration of the object 1 in the internal structure evaluation apparatus 100 will be described.

First, the configuration of the object 1 will be described. The object 1 is a plate-like member. Although the material of the object 1 is not particularly limited, it is composed of a composite material such as carbon fiber reinforced plastics (CFRP). As shown in FIG. 1, the object 1 has a side irradiated with laser light from a diode laser 10, that is, an upper surface 11, which is the upper side in FIG. 1, and a side irradiated with laser light from the diode laser 10. It has an opposite side, namely a lower surface 13, which is the lower surface in FIG. This lower surface 13 is a surface facing the infrared thermography 30 . Additionally, the infrared thermography 30 measures the heat distribution on the underside 13 of the object 1 .

Then, the computer 50 of the present embodiment identifies (specifies) the orientation of the object 1 based on the time response of the change in the temperature distribution of the object 1 (see FIG. 1) detected by the infrared thermography 30. . To explain further, the computer 50 identifies the orientation of the object 1 based on the delay in response to changes in the temperature distribution of the object 1 . Additionally, here, the thermal properties of the object 1 are hypothesized as the orientation of the object 1 .

Here, the object 1 in the illustrated example is a carbon-based composite material, more specifically carbon fiber reinforced plastics (CFRP). To explain further, the object 1 is made of a pitch-based carbon fiber reinforced resin obtained by impregnating a so-called pitch-based carbon fiber (reinforcement material) manufactured from pitch with a resin (base material) such as epoxy resin. That is, in the object 1, resin (base material) such as epoxy resin supports carbon resin (material).

The pitch-based carbon fiber has a higher thermal conductivity than the resin impregnated in the pitch-based carbon fiber. In addition, the pitch-based carbon fiber is significantly different in thermal conductivity (thermal diffusivity) from relatively low thermal conductivity resin. Here, the pitch-based carbon fiber reinforced resin is merely an example, and it is sufficient if there is a difference in thermal conductivity (thermal diffusivity) between the carbon fiber and the resin. of carbon fiber reinforced resin may be used.
Further, the carbon fiber reinforced resin in the illustrated example is a so-called discontinuous fiber composite material, and for example, the length of the pitch-based carbon fiber is, for example, about 0.1 mm to 10 mm, more specifically about 1 mm to 5 mm. . Here, the discontinuous fiber composite material is merely an example and may be a continuous fiber composite material. For example, the object 1 may be carbon fiber reinforced resin in which continuous fiber prepreg is laminated.

The orientation of the object 1 is an index indicating the orientation distribution of fibers (carbon fibers) contained in the object 1, which is an anisotropic material. In the present embodiment, it is indicated by the orientation angle (orientation direction) and orientation angle dispersion (orientation angle variation). Also, the orientation angle is an angle indicating the direction in which the fibers tend to be oriented in the object 1 . In other words, the orientation angle is the preferred degree of orientation of the fibers.

<Measurement principle>
Next, the principle of the measurement method according to this embodiment will be described.
4(a) and 4(b) are explanatory diagrams showing the principle of measuring the in-plane thermal diffusivity.

First, the principle of measuring the in-plane thermal diffusivity of the object 1 will be described with reference to FIGS. 4(a) and 4(b).

Here, the surface of the object 1 is irradiated with heating light (laser light) of a certain frequency, and the infrared thermography 30 is used to measure from the back side of the object 1 . In addition, the distance dependency of the phase lag is detected here.
Now, assuming that the area irradiated with the heating light is a point heat source, the _AC temperature Tac at a position a distance r away from this point heat source is expressed by the following equation (1).

Here,
T ₀ ... constant (Km)
f ... heating frequency (Hz)
t time (s)
r ... distance (m)

Moreover, the phase difference θ between the point heat source and the _AC temperature Tac is represented by the following equation (2).

Here,
f ₁ heating frequency (constant) (Hz)
D: thermal diffusivity (mm ² /s)

The in-plane thermal diffusivity D of the object 1 is represented by the following equation (3).

5(a) and 5(b) are explanatory diagrams showing the principle of measuring the thermal diffusivity in the thickness direction.
Next, the principle of measuring the thermal diffusivity in the thickness direction of the object 1 will be described with reference to FIGS. 5(a) and 5(b). Here, the measurement is performed while changing the frequency of the heating light irradiated to the object 1 having a constant thickness d. Additionally, the frequency dependence of the phase lag is detected here.

A thermal diffusivity D in the thickness direction of the object 1 is represented by the following equation (4).

Here,
d: Thickness of object to be measured (constant) (mm)

FIG. 6(a) shows the direction of thermal diffusion in the object 1, and FIG. 6(b) is a diagram showing the relationship between the thermal diffusion angle, thermal diffusivity, and fiber orientation density.
Next, the measurement principle of the orientation distribution of the object 1 will be described with reference to FIGS. 6(a) and 6(b).

Now, as shown in FIG. 6A, the thermal diffusion phenomenon from the heating point Hp changes according to the orientation of the carbon fibers contained in the object 1 . That is, the heat from the heating point Hp is easily transferred in the direction along the direction of the carbon fibers included in the object 1, and the heat from the heating point Hp is difficult to be transferred in the direction crossing the direction of the carbon fibers. As a result, the speed of thermal diffusion differs depending on the angle around the heating point Hp on the object 1 . In other words, the thermal diffusivity changes according to the angle around the heating point Hp in the object 1 .

Here, as shown in FIG. 6B, as a result of measuring the thermal diffusivity in multiple directions from the heating point Hp in the object 1 by the internal structure evaluation device 100, the thermal diffusivity changes according to the angle. It was confirmed that
Further, in the present embodiment, the average and variance of the fiber orientation distribution are calculated based on the obtained angular distribution of thermal diffusivity. To explain further, the average and variance of the fiber orientation distribution are calculated from the Fiber Orientation Distribution density function obtained based on the obtained thermal diffusivity angular distribution. The fiber orientation distribution density function is obtained, for example, by dividing the thermal diffusivity angular distribution including a plurality of peaks into respective peak sections and performing fitting using the least squares method for each peak.

Here, the fiber orientation distribution density function is represented by Equation (5) shown below.

Here,
η・・・offset angle (rad)
ξ・・・Parameter that determines distribution size (dimensionless number)
P: first fitting parameter Q: second fitting parameter

Also, η≦θ _a ≦θ≦θ _b ≦180°, and P≧½, Q≧½, ξ≧2, and η≧0.
Then, based on P and Q, the mean value μ and variance σ ² (standard deviation σ) of the distribution are obtained by the following equation (6).

Then, based on the average value μ of the distribution, the fiber orientation direction θ ₀ is obtained by Equation (7) shown below. Note that the section including the peak in Equation (7) corresponds to the division angle.

In addition, the variance ^σ2 represents the degree of concentration in the orientation direction, so it can also be called the orientation strength (Degree of fiber orientation).

<Action>
FIG. 7 is a flowchart for explaining the operation of the internal structure evaluation device 100. As shown in FIG.
Next, operation of the internal structure evaluation apparatus 100 according to the present embodiment will be described with reference to FIGS. 1, 2 and 7. FIG.

First, the laser light emitted from the diode laser 10 in the internal structure evaluation apparatus 100 causes spot periodic heating of the surface of the object 1 (step 701).
Based on the temperature distribution measured by the infrared thermography 30, the phase delay distribution measuring section 51 measures the phase delay distribution (step 702).
Then, the thermal diffusivity distribution calculator 52 calculates the thermal diffusivity angular distribution based on the measured phase delay distribution (step 703). Based on the calculated angular distribution of thermal diffusivity, the orientation calculator 53 calculates the orientation direction and dispersion (step 704). Then, the calculation result display unit 54 displays the calculation result of the orientation direction and dispersion on display means (not shown) (step 705).

In the internal structure evaluation apparatus 100 (see FIG. 1) of the present embodiment described above, the upper surface 11 of the object 1 is used as a heating surface by the diode laser 10, and the lower surface 13 is used as a temperature detection surface by the infrared thermography 30. It is not limited to this aspect. The internal structure evaluation apparatus 100 may use the upper surface 11 of the object 1 as a heating surface by the diode laser 10 and use the same upper surface 11 as a temperature detection surface by the infrared thermography 30 .

Next, the machine learning section 55 will be described in detail.
FIG. 8 is a diagram showing an example of a cyclic heating response image measured by a lock-in thermography cyclic heating method.

As described above, the internal structure evaluation apparatus 100 of the present embodiment is capable of identifying fiber orientation using thermal diffusivity measurement by the lock-in thermography periodic heating method. The lock-in thermographic periodic heating method can obtain detailed information as amplitude and phase delay in a non-destructive manner by lock-in analysis of the thermal response. In other words, the internal structure evaluation apparatus 100 can acquire the state of heat diffusion as the amplitude and phase delay of minute temperature fluctuations.

When a certain anisotropic discontinuous fiber CFRP material is measured by a lock-in thermographic periodic heating method using spot heating, a periodic heating response image, which is a phase lag image as shown in FIG. 8, is obtained. This periodic heating response image does not have a perfect circular distribution, but has an elliptical distribution extending in the left-right direction under the influence of the carbon fibers inside. The cyclic heating response image thus contains internal structural information of the material. Therefore, it is possible to estimate the structure by quantifying the relationship between the internal structure and the periodic heating response image.

Then, the machine learning unit 55 uses a periodic heating response image that can be acquired by the lock-in thermography type periodic heating method and that indicates the time response of the change in temperature distribution. The machine learning unit 55 converts the periodic heating response image into an internal structure. Also, the machine learning unit 55 uses deep learning for conversion to an internal structure.
In this embodiment, learning data used for deep learning was prepared using virtual material construction and heat transfer simulation analysis. As training data, a data set consisting of a large number of pairs of image data of virtual materials with various structures and periodic heating response image data obtained by cyclic heating of the virtual materials by simulation is used.

The network used in the machine learning unit 55 has a structure that combines convolution and deconvolution to convert a periodic heating response image (input) into an estimated internal structure image (output). The machine learning unit 55 builds a trained model by making the network learn using the created data set. Then, the machine learning unit 55 makes it possible to estimate the internal structure from the periodic heating response image by using the learned model.

In this embodiment, a large amount of combinations of "known internal structures" and "periodic heating response images" are prepared as learning data. Materials development simulation software capable of creating virtual materials is used to create the known internal structure. On the other hand, the created periodic heating response image of the known internal structure uses software capable of heat transfer simulation analysis. In the present embodiment, a large number of combinations of data images of "known internal structures" and data images of "periodic heating response images" created and analyzed by these softwares were created.

FIG. 9 is a diagram showing an example of the target object 1 used in this embodiment.
FIG. 10 shows a periodic heating response image Ti and a cross-sectional structure image Si of the object 1 of this embodiment.

The object 1 shown in FIG. 9 is a virtual material. A unidirectional laminated CFRP is used for the object 1 . As shown in FIG. 9, a model is used in which bundles of φ200 μm carbon fibers 1c are distributed in one direction in an area of 2 mm×2 mm×1 mm in the center of a resin body of resin 1r of 3 mm×3 mm×1 mm. The distribution of the carbon fibers 1c is randomly arranged by software based on predetermined conditions.

In the heat transfer simulation analysis, periodic point heating was performed by heating the heating point Hp, which is one point in the central portion of the upper surface 11 of the object 1 . A 2 mm×2 mm area on the lower surface 13 of the object 1 was used as the measurement surface MA. Here, the size of the heating point Hp in this embodiment is assumed to be 100 μm in diameter. The periodic heating conditions were a heat flux of 500,000 W/m ² , a heating frequency of 0.1 Hz, a measurement frequency of 2 Hz, and 10 cycles.

Then, as shown in FIG. 10(a), a periodic heating response image is obtained as a result of the heat transfer simulation analysis. This periodic heating response image Ti is a periodic heating response image Ti showing the phase delay distribution viewed from the lower surface 13 (xy plane) of the object 1 shown in FIG. In the periodic heating response image Ti, the phase lag is represented by black and white shading. The phase lag distribution differs depending on the internal structure of the object 1. FIG. That is, the contents of the periodic heating response image Ti differ depending on the internal structure of the object 1, which is represented by black and white shading. In other words, the periodic heating response image Ti differs in the content of the image represented by black and white shading according to the arrangement of the carbon fibers 1c in the cross-sectional structure of the object 1 .

Also, as shown in FIG. 10B, a cross-sectional structure image Si showing the fiber distribution of the cross section of the object 1 passing through the heating point Hp shown in FIG. 9 is obtained. This cross-sectional structure image Si is an image of the yz cross section shown in FIG. The position of the carbon fiber 1c and the position of the resin 1r in the cross section are specified in the cross-sectional structure image Si. Note that the cross-sectional structure image Si of the present embodiment is coordinate-transformed into a monochrome square so that it can be easily used for deep learning, which will be described later.

Then, the machine learning unit 55 of the present embodiment uses a combination of the periodic heating response image Ti and the cross-sectional structure image Si as one data, and uses a data set in which a plurality of data are collected as learning data (teacher data) for deep learning. use.

<Conditions for machine learning>
FIG. 11 is a conceptual diagram of the neural network structure used by the machine learning unit of this embodiment.

The machine learning unit 55 uses a convolution neural network (CNN) as an example of a machine learning model. A convolutional neural network is a network structure that can reduce parameters while capturing image features by connecting each layer by convolution processing.

The machine learning unit 55 of this embodiment employs a network structure different from that of U-Net while referring to the structure of U-Net.
Here, U-Net is a network structure in which features are extracted from an input image using convolution in the encoder section, and an image having the same size as the input image is output by upsampling the features extracted in the decoder section. In particular, U-Net is mainly used for object detection, and the encoder and decoder sections are combined to stabilize the upsampling. U-net employs a skip structure in which, for example, a feature map (feature quantity) extracted in a specific convolution layer is sent to a deconvolution layer corresponding to the specific convolution layer.

In the present embodiment, the features of the input periodic heating response image Ti are extracted, and the feature is converted into an image that generates an output estimated image representing the internal structure. Therefore, as shown in FIG. 11, in this embodiment, referring to the structure of U-Net, a network structure is created in which the coupling between the encoder and the decoder is removed. The machine learning unit 55 of this embodiment uses a neural network including an encoder unit having five convolution layers and a decoder unit having five deconvolution layers. The encoder section is provided with three layers consisting of a convolutional layer and a pooling layer, and two convolutional layers only. Also, the decoder section is provided with three layers consisting of a deconvolution layer and an unpooling layer, and two deconvolution layers only. The neural network of this embodiment does not have a skip connection that sends a feature map (feature amount) extracted in a specific convolution layer to the deconvolution layer corresponding to the specific convolution layer.

In the internal structure evaluation device 100 of this embodiment, the information output method is realized by the following steps.
First, the machine learning unit 55 shows the cross-sectional structure image Si showing the internal structure of the object and the time response of the change in the temperature distribution in the region including the part heated by irradiating the object with periodic light. Acquire learning data that is a combination with the periodic heating response image Ti.
Next, the machine learning unit 55 learns a machine learning model using the acquired learning data.
Furthermore, the machine learning unit 55 inputs a periodic heating response image Ti of an object whose internal structure is unknown to the machine learning model, and outputs a structure estimation image Ei of the internal structure of the object.

FIG. 12 is an explanatory diagram of learning in the machine learning unit 55 of this embodiment.

The machine learning unit 55 of the present embodiment performs deep learning for learning a neural network with the above paired data set. In this embodiment, the data sets created are 122 pairs in total, 99 pairs for training data, 20 pairs for evaluation data within learning (validation data), and 3 pairs for test data. was randomly assigned. That is, a total of 119 pairs of learning data and in-learning evaluation data were used for learning.

As the error function, the mean squared error E, which is highly versatile in the regression problem shown in Equation (8), is used. As an optimization method, Adam (Adaptive moment estimation) is used and learning is performed in a direction in which the error function becomes smaller.

where y _i is the correct value and y _l ^ is the output. First, Model A was learned with hyperparameters set to 100 maximum epochs and 16 batches. The learning calculation time for the CPU was 6 minutes and 29 seconds. As a result, the value of the training error represented by the magnitude of the error function of the learning data was 0.054, and the value of the validation error represented by the magnitude of the error function of the validation data was 0.093. As shown in FIG. 12, the Error converges downward to the right, indicating that learning is progressing normally without over-learning.

Next, in order to increase the number of epochs while avoiding over-learning, learning is performed using E′ obtained by applying L2 regularization to the error function shown in Equation (9).

A regularization parameter λ was set to 0.001. Also, β is a regression coefficient vector. Model B was trained with 200 maximum epochs and 16 batches. The learning calculation time for the CPU was 10 minutes and 41 seconds. As a result, the value of the training error represented by the magnitude of the error function of the learning data was 0.043, and the value of the validation error represented by the magnitude of the error function of the validation data was 0.077.

FIG. 13 shows an example of estimation results by the machine learning unit 55 of this embodiment. 13(a) shows the results of Model A, and FIG. 13(b) shows the results of Model B. As shown in FIG.

The machine learning unit 55 of the present embodiment inputs the periodic heating response image Ti, which is test data (data 1, data 2, and data 3), into the learned model, and estimates the internal structure. 13A and 13B show, from the left, a periodic heating response image Ti as test data, an estimated structure image Ei as an estimation result, and a cross-sectional structure image Si as correct data. there is

A periodic heating response image Ti as each test data shown in FIG. 13 is created by heat transfer simulation analysis of a virtual material having the cross-sectional structure shown in the cross-sectional structure image Si.
The structure estimation image Ei, which is the estimation (output) result, is an image showing the positions of the carbon fibers 1c in the cross section of the object 1. FIG. Thus, the machine learning unit 55 can output the structure estimation image Ei, which is the image of the cross-sectional structure of the object 1, from the periodic heating response image Ti of the object 1. FIG.

As shown in FIGS. 13(a) and 13(b), qualitatively, both Model A and Model B perform structural estimation with a certain degree of accuracy. That is, it can be seen that the estimated structure image Ei is an output that has received the features of the cross-sectional structure image Si, which is the correct structure.

Next, the correct answer rate by the machine learning unit 55 of this embodiment will be described.
The correct answer rate is obtained by evaluating the estimated structure image Ei, which is the image output by the learned model, with the cross-sectional structure image Si, which is the image of the cross-sectional view of the object 1, as the correct answer.

Two evaluation methods were used in this embodiment. Specifically, there is an evaluation method called ALL and an evaluation method called Balanced accuracy (hereinafter referred to as BA). ALL is the positive percentage of the total, also called accuracy rate. In BA, it is a ratio obtained by averaging the difference in the number of correct and incorrect numbers.
In the evaluation, each pixel in the cross-sectional structure image Si of the virtual material was compared with the pixel at the corresponding position in the estimated structure image Ei obtained as the output image.

Here, in the pixels at the corresponding positions, the carbon fiber 1c and other than the carbon fiber 1c in the cross-sectional structure image Si (resin 1r in this example) and the carbon fiber 1c and other than the carbon fiber 1c in the estimated structure image Ei (resin in this example) 1r) and the correct/wrong relationship is as follows.
For example, when the cross-sectional structure image Si indicates the carbon fiber 1c, the estimated structure image Ei predicts the carbon fiber 1c as a true positive (hereinafter referred to as TP). When the cross-sectional structure image Si indicates a carbon fiber other than the carbon fiber 1c, a false positive (hereinafter referred to as FP) is obtained when the estimated structure image Ei predicts the carbon fiber 1c. When the cross-sectional structure image Si indicates the carbon fiber 1c, the estimated structure image Ei predicts that the carbon fiber is not the carbon fiber 1c. When the cross-sectional structure image Si indicates that the carbon fiber is other than the carbon fiber 1c, the estimated structure image Ei predicts that the carbon fiber is other than the carbon fiber 1c.

ALL is represented by Formula (10).

The ALL value was about 65% for Model A (data 1), about 66% for Model A (data 2), and about 57% for Model A (data 3). The ALL value was about 63% for Model B (data 1), about 65% for Model B (data 2), and about 62% for Model B (data 3). Thus, the value of ALL was about 57% to 66% of correct answers.

BA is represented by Formula (11).

The BA value was about 56% for Model A (data 1), about 57% for Model A (data 2), and about 50% for Model A (data 3). The BA value is about 53% for Model B (data 1), about 52% for Model B (data 2), and about 55% for Model B (data 3). Thus, the BA value was about 50% to 57% of correct answers.

In this embodiment, attention is focused on thermophysical properties unique to the inside of the object 1 (for example, CFRP, which is a composite material containing carbon fibers). That is, when the object 1 is heated using the lock-in thermography periodic heating method, the distribution of the state of heat propagation (phase delay) from the upper surface 11 to the lower surface 13 of the object 1 can be identified. For example, there is a tendency that the phase lag is small in the fiber direction of carbon fibers, and the phase lag is large in directions different from the fiber direction of carbon fibers. Therefore, the feature quantity of the internal structure of the object 1 is reflected in the periodic heating response image Ti showing the phase delay distribution obtained by measuring the object 1 . Therefore, in the machine learning unit 55 of the present embodiment, the feature amount is extracted by machine learning using a machine learning model. Then, by inputting the periodic heating response image Ti of the object 1 to the learned machine learning model, the structure estimation image Ei for estimating the internal structure of the object 1 is output.

As described above, the internal structure evaluation apparatus 100 of the present embodiment can estimate the cross-sectional structure of the object 1 from the periodic heating response image Ti using the learned model. The internal structure evaluation apparatus 100 can three-dimensionally identify the three-dimensional orientation of the internal structure of the object 1, for example, carbon fibers, from the two-dimensional periodic heating response image Ti. That is, the internal structure evaluation apparatus 100 can estimate the three-dimensional internal structure of the object 1 using the periodic heating response image Ti.

Next, the selection reception unit 56 shown in FIG. 2 will be described.
The internal structure evaluation apparatus 100 of the present embodiment performs arithmetic processing by the computer 50 using the image data obtained from the object 1 by the infrared thermography 30, and the measurement result of the internal structure for calculating the orientation of the object 1 is obtained. Output is enabled. Further, the internal structure evaluation apparatus 100 of the present embodiment can output the result of estimating the internal structure of the object 1 using a learned model based on the image data obtained by the infrared thermography 30 . Then, the selection accepting unit 56 can accept a selection from the user as to whether to output the result of measuring the internal structure or the result of estimating the internal structure.

Next, variations of learning data when learning a machine learning model in the machine learning unit 55 will be described.

<Modification>
For example, as described with reference to FIG. 1, the machine learning unit 55 of the present embodiment obtains the upper surface 11 of the object 1 as the heating surface by the diode laser 10 and the lower surface 13 as the temperature detection surface by the infrared thermography 30. The obtained periodic heating response image Ti is used as learning data. Then, the machine learning unit 55 learned a machine learning model using the periodic heating response image Ti obtained under the same heating conditions for the object 1 as learning data.

On the other hand, the learning data may include a plurality of periodic heating response images Ti with different heating conditions for the same object 1 . In this way, the accuracy of estimating the internal structure of the object 1 can be improved by making the machine learning model learn using learning data containing a plurality of different periodic heating response images Ti for the same object 1. becomes.

For example, the machine learning unit 55 may use the periodic heating response image Ti obtained by using the upper surface 11 of the object 1 as the heating surface and using the same upper surface 11 as the detection surface as learning data.
Then, with respect to the cross-sectional structure image Si showing the cross section of the object 1, a periodic heating response image Ti obtained by using the upper surface 11 of the object 1 as a heating surface and the lower surface 13 as a detection surface (see FIG. 1). and a periodic heating response image Ti obtained by using the upper surface 11 of one object 1 as a heating surface and the upper surface 11 as a detection surface to create learning data. Then, the machine learning unit 55 may make the neural network learn using the created learning data.

<Modification>
FIG. 14 is an explanatory diagram of a modified heating surface and detection surface of the object 1 .

In the present embodiment, for example, the heating surface and the detection surface of the object 1 are a combination of two surfaces, such as an upper surface 11 (xy plane) and a lower surface 13 (xy plane), which are in a substantially parallel relationship, so that periodic heating response can be achieved. An image Ti is obtained, but is not limited to this aspect.

As shown in FIG. 14, a heating point Hp is provided on the first side surface 14 (xz plane) of the object 1, and the first side surface 14 is used as a heating surface. A periodic heating response image Ti may be obtained using the third side surface 16 (xz plane) as a detection plane. Further, a heating point Hp is provided on the second side surface 15 (yz plane) of the object 1 to use the second side surface 15 as a heating surface, and a fourth side surface 17 ( yz plane) may be obtained as a detection plane.

Then, a periodic heating response image Ti obtained by using the lower surface 13 as a detection surface and a periodic heating response image Ti obtained by using the third side surface 16 as a detection surface for the cross-sectional structure image Si showing the cross section of the object 1. , and the periodic heating response image Ti obtained by using the fourth side surface 17 as a detection surface to create learning data. In this way, the learning data may be composed of a plurality of periodic heating response images Ti obtained by changing the heating target surface for one target 1 . Then, the machine learning unit 55 may make the neural network learn using this learning data.

<Modification>
15A and 15B are explanatory diagrams of a modified heating surface and detection surface of the object 1. FIG.

In the present embodiment, for example, the upper surface 11 of the object 1 is heated by performing point heating at one heating point Hp (see, for example, FIG. 9) on the upper surface 11 of the object 1, but the present invention is not limited to this example. .
As shown in FIG. 15 , point heating may be performed at a plurality of locations on the upper surface 11 of the object 1 . In the example shown in FIG. 15 , point heating is performed by irradiating the first heating point Hp1, the second heating point Hp2, the third heating point Hp3, and the fourth heating point Hp4 on the upper surface 11 with light from the diode laser 10. implement. Then, each time point heating is performed at each point on the upper surface 11 of the object 1, a periodic heating response image Ti is obtained using the lower surface 13 as a detection surface.

Learning data is created by combining a plurality of periodic heating response images Ti with different heating positions on the upper surface 11 side with respect to the cross-sectional structure image Si showing the cross section of the object 1 . In this way, the learning data may be composed of a plurality of periodic heating response images Ti obtained by varying the heating positions on one object 1 . Then, the machine learning unit 55 may make the neural network learn using this learning data.

<Modification>
FIG. 16 is an explanatory diagram of the internal structure evaluation device 100 of the modification.

As shown in FIG. 16, the basic configuration of the internal structure evaluation device 100 of the modification is the same as the internal structure evaluation device 100 of the present embodiment described above. The internal structure evaluation apparatus 100 of the modified example has a cylindrical lens 25 that changes laser light emitted from the diode laser 10 (see FIG. 1) into sheet light. Then, as shown in FIG. 16 , a light irradiation area HA is formed on the upper surface 11 of the object 1 . The illustrated light irradiation area HA has a substantially elliptical or substantially rectangular shape with the major axis extending in the x-direction. In addition, the light irradiation area HA has a shape elongated in one direction on the upper surface 11 of the object 1 . This light irradiation area HA enables line heating (line heating) on the upper surface 11 of the object 1 . In addition, the light irradiation area HA can be regarded as a linear heat source.

Then, line heating is performed on the upper surface 11 of the object 1, and a periodic heating response image Ti is obtained using the lower surface 13 as a detection surface. Also, learning data is created by combining a cross-sectional structure image Si representing a cross section of the object 1 with a periodic heating response image Ti obtained by performing line heating. Then, the machine learning unit 55 may make the neural network learn using this learning data.

Moreover, in the internal structure evaluation apparatus 100 shown in FIG. 16, the object 1 and the cylindrical lens 25 may be moved relatively. For example, the cylindrical lens 25 is fixed and the object 1 is moved in the y direction with respect to the cylindrical lens 25 . Thereby, line heating may be performed at different locations on the object 1 . Each time line heating is performed at each location on the upper surface 11 of the object 1, a periodic heating response image Ti can be obtained using the lower surface 13 as a detection surface.
Learning data is created by combining a plurality of periodic heating response images Ti with different line heating positions on the upper surface 11 side with respect to the cross-sectional structure image Si representing the cross section of the object 1 . Then, the machine learning unit 55 may make the neural network learn using this learning data.

Furthermore, for the cross-sectional structure image Si showing the cross section of the object 1, a periodic heating response image Ti obtained by performing point heating on one object 1, and a line A data set is created by combining periodic heating response images Ti obtained by heating. In this way, the learning data may be composed of a plurality of periodic heating response images Ti obtained by varying the heating area for one object 1 . Then, the machine learning unit 55 may make the neural network learn using this learning data.

Note that the internal structure evaluation apparatus 100 may perform surface heating such that the entire upper surface 11 of the object 1 is heated, and acquire the periodic heating response image Ti using the lower surface 13 as a detection surface.

<Modification>
Although the machine learning unit 55 of the present embodiment uses the periodic heating response image Ti obtained by heating the object 1 with one heating frequency as learning data, the machine learning unit 55 is not limited to this aspect. For example, a plurality of periodic heating response images Ti obtained by heating one object 1 at different heating frequencies may be used as learning data.

In particular, when the upper surface 11 of the object 1 is used as a heating surface and the upper surface 11 is used as a detection surface, by obtaining a plurality of periodic heating response images Ti with different heating frequencies, the thickness direction (z direction) of the object 1 can be obtained. ) can be selectively acquired.

Here, when the heating frequency is low, the heat wavelength becomes long, and the heating range becomes longer from the upper surface 11 to the lower surface 13 . In this case, information on the thermal properties up to the lower surface 13 in the thickness direction of the object 1 can be obtained. On the other hand, when the heating frequency is high, the heat wavelength becomes short, and the heating range from the upper surface 11 to the lower surface 13 becomes short. In this case, information on the thermal characteristics of only the upper surface 11 side in the thickness direction of the object 1 can be obtained.

By varying the heating frequency for the object 1 in this way, it is possible to obtain information on thermal characteristics that differ in the thickness direction of the object 1 . A neural network can be trained using a plurality of periodic heating response images Ti obtained by different heating frequencies when heating one object 1 as learning data.

Note that the diode laser 10 is an example of a heating unit. The infrared thermography 30 is an example of a detector. The calculation result display section 54 is an example of another output section. The machine learning unit 55 is an example of an acquisition unit, an image acquisition unit, a learning unit, an output unit, and a storage unit. The selection reception unit 56 is an example of a reception unit. The internal structure evaluation device 100 is an example of an information processing device and an information processing system. The periodic heating response image Ti is an example of a response image. The cross-sectional structure image Si is an example of a structure image. The estimated structure image Ei is an example of an estimated image.

In addition, although the internal structure evaluation apparatus 100 of this embodiment uses the learned model of the machine learning unit 55 to estimate the internal structure of the object 1, it is not limited to this aspect. The internal structure evaluation apparatus 100 only needs to be able to estimate the internal structure (cross-sectional structure) of the object 1 based on the periodic heating response image Ti of the object 1, and can use image processing techniques based on pattern matching, for example.

Further, as described with reference to FIG. 9, for example, in the present embodiment, an example in which machine learning is performed using a model in which carbon fiber bundles are distributed in one direction as the target object 1 is described. It is not limited to the mode. The machine learning unit 55 of the internal structure evaluation apparatus 100 of the present embodiment can be similarly applied to, for example, the target object 1 in which the fiber directions of the carbon fiber bundle are oriented in various directions (multidirectional).

Further, the target object 1 to be evaluated by the internal structure evaluation apparatus 100 of this embodiment is not limited to composite materials. The object 1 may for example consist of a single material. In this case, for example, when voids are present in the material, heat conduction is inhibited in the void portions. Therefore, the presence or absence of voids in the material is reflected in the periodic heating response image Ti as a time response of changes in temperature distribution. The internal structure evaluation apparatus 100 can also measure the internal structure and estimate the internal structure of the object 1 made of a single material.

Various embodiments and modifications have been described above, but these embodiments and modifications may of course be combined.
In addition, the present disclosure is not limited to the above embodiments, and can be embodied in various forms without departing from the gist of the present disclosure.

REFERENCE SIGNS LIST 1 object, 1r resin, 1c carbon fiber, 10 diode laser, 30 infrared thermography, 50 computer, 51 distribution measurement unit, 52 thermal diffusivity distribution calculation unit, 53 orientation calculation unit, 54... Calculation result display unit, 55... Machine learning unit, 56... Selection reception unit, Si... Cross-sectional structure image, Ti... Periodic heating response image, Ei... Structure estimation image

Claims (18)

an image acquisition unit that acquires a response image showing a time response of a change in temperature distribution in a region including a portion that is heated by periodically irradiating the object with light;
an output unit that outputs an image of the cross-sectional structure of the object estimated using the response image acquired by the image acquisition unit;
An information processing device comprising: Learning data that is a combination of a structure image that shows the internal structure of an object and a response image that shows the time response of changes in temperature distribution in an area that includes a portion that is heated by periodically irradiating the object with light. an acquisition unit that acquires
a learning unit for learning a machine learning model using the learning data acquired by the acquisition unit;
an output unit that inputs the response image of the object to the machine learning model trained by the learning unit and outputs an estimated image of the internal structure of the object;
An information processing system comprising: The object has a specific material and a base material that supports the material,
3. The information processing system according to claim 2, wherein said estimated image is an image indicating the position of said material in a cross section of said object. 3. The information processing system according to claim 2, wherein said learning data includes a plurality of said response images with different heating conditions for one object. 5. An information processing system according to claim 4, wherein said heating condition is a heating frequency for heating one object. 5. The information processing system according to claim 4, wherein the heating condition is a heating position for one object. 7. The information processing system according to claim 6, wherein the heating condition is a heating target surface of one object. 7. The information processing system according to claim 6, wherein said heating condition is a heating area for one object. 3. The information processing system according to claim 2, wherein said learning data includes a plurality of said response images having different temperature detection surfaces in one object. The machine learning model comprises a neural network structure including an encoder unit having a plurality of convolution layers and receiving the response image, and a decoder unit having a plurality of deconvolution layers and outputting the estimated image,
3. An information processing system according to claim 2, wherein features extracted in said convolutional layer are not sent to said deconvolutional layer corresponding to said convolutional layer. Learning data that is a combination of a structural image showing the internal structure of an object and a response image showing the time response of changes in the temperature distribution in an area that includes a portion heated by periodically irradiating the object with light. A storage unit that stores a machine learning model learned using
an output unit that inputs the response image of one target to the machine learning model and outputs information about the internal structure of the one target;
An information processing device comprising: a heating unit that periodically irradiates an object with light to heat the object;
a detection unit that detects a time response of a change in temperature distribution in a region including a portion of the object heated by the heating unit;
another output unit that outputs information indicating the orientation of at least one material that constitutes the object based on the time response of the change in temperature distribution detected by the detection unit;
a reception unit that receives, from a user, a selection of information to be output from information about the internal structure of the output unit and information indicating the orientation of the other output unit;
12. The information processing apparatus according to claim 11, comprising: acquiring a response image showing a time response of a change in temperature distribution in a region including a portion heated by irradiating the object with periodic light;
outputting an image of the cross-sectional structure of the object estimated using the acquired response image;
An information output method characterized by comprising: Learning data that is a combination of a structure image that shows the internal structure of an object and a response image that shows the time response of changes in temperature distribution in an area that includes a portion that is heated by periodically irradiating the object with light. and obtaining
training a machine learning model using the acquired learning data;
inputting the response image of an object into the machine learning model and outputting an estimated image of the internal structure of the object;
An information output method characterized by comprising: Learning data that is a combination of a structural image showing the internal structure of an object and a response image showing the time response of changes in the temperature distribution in an area that includes a portion heated by periodically irradiating the object with light. storing the machine learning model trained using
inputting the response image of an object into the machine learning model and outputting information about the internal structure of the object;
An information output method characterized by comprising: to the computer,
A function of acquiring a response image showing a time response of a change in temperature distribution in an area including a portion heated by periodically irradiating an object with light;
a function of outputting an image of the cross-sectional structure of the object estimated using the acquired response image;
program to run. to the computer,
Learning data that is a combination of a structure image that shows the internal structure of an object and a response image that shows the time response of changes in temperature distribution in an area that includes a portion that is heated by periodically irradiating the object with light. and the ability to get
A function to train a machine learning model using the acquired learning data,
a function of inputting the response image of the object into the machine learning model and outputting an estimated image of the internal structure of the object;
program to run. to the computer,
Learning data that is a combination of a structural image showing the internal structure of an object and a response image showing the time response of changes in the temperature distribution in an area that includes a portion heated by periodically irradiating the object with light. A function to store a machine learning model trained using
A function of inputting the response image of one object into the machine learning model and outputting information about the internal structure of the one object;
program to run.

Images