JP7481185B2 - Machine learning method, machine learning device, machine learning program, communication method, and film formation device - Google Patents

Description

This disclosure relates to a technology that uses machine learning to learn film formation conditions.

In recent years, hard coatings have been formed on the surfaces of engine component substrates in order to improve the hardness of sliding members such as engine parts such as pistons and cylinders. Hereinafter, a coating formed to improve the hardness of the surface of a sliding member substrate is referred to as a sliding coating. For example, Patent Document 1 discloses a technology for forming a carbon film by arc ion plating while reducing the supply amount of hydrocarbon gas, thereby controlling the internal stress and hardness of the carbon film so that the hardness of the formed carbon film is low on the substrate surface and increases with increasing distance from the substrate.

JP 2016-56435 A

However, in the past, the deposition conditions for such sliding films were determined by skilled engineers based on their many years of experience, and were difficult to determine easily.

The present invention has been made to solve the above-mentioned problems, and aims to provide a machine learning device etc. that can easily determine the deposition conditions for properly depositing a sliding film without relying on the many years of experience of a skilled technician.

In recent years, various services related to machine learning, including deep learning, have been provided on the cloud, and users have been able to easily use these services. The inventors of the present invention came up with the idea that appropriate deposition conditions for a sliding film can be easily determined by machine learning the deposition conditions and physical quantities related to the performance evaluation of the sliding film.

A machine learning method according to one aspect of the present invention is a machine learning method in which a machine learning device determines deposition conditions for a deposition device that deposits a slide coating on a substrate, the deposition device including a vacuum exhaust system for evacuating a chamber, a heating and cooling system for heating and cooling the chamber, an evaporation source system for evaporating a target, a table system for placing a workpiece, a process gas system for introducing a process gas into the chamber, and an etching system, and the method obtains state variables including at least one physical quantity related to a performance evaluation of the slide coating and at least one deposition condition, and calculates a result of the determination of the at least one deposition condition based on the state variables. A reward is calculated, a function for determining the at least one film forming condition from the state variables is updated based on the reward, and the film forming condition that obtains the greatest reward is determined by repeating the updating of the function, the at least one film forming condition being at least one of a first parameter related to the vacuum exhaust system, a second parameter related to the heating and cooling system, a third parameter related to the evaporation source system, a fourth parameter related to the table system, and a fifth parameter related to the process gas system, and the at least one physical quantity being at least one of a film quality characteristic, a mechanical characteristic, and a physical characteristic.

According to this configuration, at least one of the deposition conditions among the first parameter related to the vacuum exhaust system, the second parameter related to the heating and cooling system, the third parameter related to the evaporation source system, the fourth parameter related to the table system, and the fifth parameter related to the process gas system, and at least one physical quantity among the film quality characteristics, mechanical characteristics, and physical characteristics related to the performance evaluation of the slide coating are observed as state variables. Then, based on the observed state variables, a reward for the determination result of the deposition condition is calculated, and based on the calculated reward, a function for determining the deposition condition from the state variables is updated, and this update is repeated to learn the deposition condition that obtains the most reward. Furthermore, in this configuration, machine learning is performed using the film quality characteristics, mechanical characteristics, and physical characteristics as physical quantities related to the evaluation of the slide coating. Therefore, this configuration can easily determine appropriate deposition conditions for the slide coating.

In the above configuration, the first parameter may be at least one of the exhaust speed, ultimate pressure, residual gas type, residual gas partial pressure, and P-Q characteristic.

According to this configuration, at least one of the exhaust speed, ultimate pressure, residual gas type, residual gas partial pressure, and P-Q characteristics is set as the film formation conditions related to the vacuum exhaust system and machine learning is performed, so that appropriate film formation conditions can be determined taking into account the state of the vacuum exhaust system.

In the above configuration, the second parameter may be at least one of the heater temperature of a heater constituting the heating and cooling system, the work temperature which is the temperature of the work, the heating rate of the heater, the heating rate of the work, the output of the heater, the temperature accuracy of the heater, the temperature accuracy of the work, the response characteristics of the heater temperature and the work temperature, the temperature distribution of the heater, and the temperature distribution of the work.

According to this configuration, machine learning is performed on at least one of the heater temperature, workpiece temperature, heater heating rate, workpiece heating rate, heater output, heater temperature accuracy, workpiece temperature accuracy, heater temperature response characteristics, workpiece temperature response characteristics, heater temperature distribution, and workpiece temperature distribution as film formation conditions related to the heating and cooling system, so that appropriate film formation conditions can be determined taking into account the state of the heating and cooling system.

In the above configuration, the third parameter may be at least one of the composition of the target, the thickness of the target, the manufacturing method of the target, the arc discharge voltage, the arc discharge current, the evaporation source magnetic field, the evaporation source coil current, and the arc ignition characteristics.

According to this configuration, machine learning is performed on at least one of the target composition, target thickness, target manufacturing method, arc discharge voltage, arc discharge current, evaporation source magnetic field, evaporation source coil current, and arc ignition characteristics as film formation conditions related to the evaporation source system, so that appropriate film formation conditions can be determined taking into account the state of the evaporation source system.

In the above configuration, the fourth parameter may include at least one of a bias voltage for the workpiece, a bias current for the workpiece, the number of abnormal discharges, a change in the abnormal discharge over time, a waveform of the bias voltage, a waveform of the bias current, a rotation speed of the workpiece, a shape of the workpiece, a loading amount of the workpiece, a loading method of the workpiece, and a material of the workpiece.

According to this configuration, at least one of the bias voltage, bias current, number of abnormal discharges, time change of abnormal discharges, bias voltage waveform, bias current waveform, workpiece rotation speed, workpiece shape, workpiece loading amount, workpiece loading method, and workpiece material is machine-learned as a deposition condition related to the table system, so that appropriate deposition conditions can be determined taking into account the state of the table system.

In the above configuration, the fifth parameter may be at least one of the flow rate of the process gas, the type of the process gas, and the pressure of the process gas.

According to this configuration, at least one of the process gas flow rate, the process gas type, and the process gas pressure is learned by machine learning as a film formation condition related to the process gas system, so that appropriate film formation conditions can be determined taking into account the state of the process gas system.

In the above configuration, the at least one film formation condition may further include a sixth parameter related to the etching system.

With this configuration, machine learning is performed taking into account the deposition conditions related to the etching system, so appropriate deposition conditions can be determined taking into account the state of the etching system.

In the above configuration, the sixth parameter may be at least one of a heating current for heating a filament of the etching system, a heating voltage for heating the filament, a diameter of the filament, a discharge current of the filament, and a discharge voltage of the filament.

According to this configuration, at least one of the filament heating current, filament heating voltage, filament diameter, filament discharge current, and filament discharge voltage is machine-learned as a deposition condition for the etching system, so that appropriate deposition conditions can be determined taking into account the state of the etching system.

In the above configuration, the film quality characteristics may include at least one of the film thickness, roughness, surface properties, composition, crystal structure, film microstructure, crystallinity, crystal grain size, residual stress, density, defect amount, and defect size of the sliding film.

According to this configuration, at least one of the film thickness, roughness, surface properties, composition, crystal structure, film microstructure, crystallinity, crystal grain size, residual stress, density, defect amount, and defect size is adopted as the film quality characteristics of the sliding film. Therefore, it is easy to obtain film formation conditions that can obtain a sliding film that satisfies these film quality characteristics.

In the above configuration, the mechanical properties may include at least one of the hardness, elastic modulus, and wear resistance of the sliding film.

According to this configuration, at least one of hardness, elastic modulus, and wear resistance is adopted as the mechanical property of the sliding film. Therefore, it is easy to obtain film formation conditions that can obtain a sliding film that satisfies these mechanical properties.

In the above configuration, the physical properties may include at least one of the friction coefficient, oxidation resistance, adhesion, thermal conductivity, adhesion, corrosion resistance, and surface chemical affinity of the sliding film.

According to this configuration, at least one of the friction coefficient, oxidation resistance, adhesion, thermal conductivity, adhesion, corrosion resistance, and surface chemical affinity is adopted as the physical property of the sliding film. Therefore, it is easy to obtain film formation conditions that can obtain a sliding film that satisfies these physical properties.

In the above configuration, the function may be updated in real time using deep reinforcement learning.

According to this embodiment, the function is updated in real time using deep reinforcement learning, so the function can be updated accurately and quickly.

In the above configuration, in calculating the reward, if the at least one physical quantity is approaching a predetermined reference value corresponding to each physical quantity, the reward may be increased, and if the at least one physical quantity is not approaching the reference value corresponding to each physical quantity, the reward may be decreased.

According to this embodiment, the reward increases as the physical quantity approaches the reference value, so the physical quantity can be made to reach the reference value quickly.

Each process of the above-mentioned machine learning method may be implemented by a machine learning device, or may be implemented in a machine learning program and distributed. This machine learning device may be configured as a server, or may be configured as a film forming device.

A communication method according to another aspect of the present invention is a communication method of a film-forming device that forms a slide coating on a substrate when the film-forming conditions of the film-forming device are learned by machine learning, the film-forming device includes a vacuum exhaust system for evacuating a chamber, a heating and cooling system for heating and cooling the chamber, an evaporation source system for evaporating a target, a table system for placing a workpiece, a process gas system for introducing a process gas into the chamber, an etching system, and a communication unit, and observes state variables including at least one physical quantity related to performance evaluation of the slide coating and at least one film-forming condition, transmits the state variables onto a network, and receives at least one film-forming condition that has been learned by machine learning, the at least one film-forming condition being at least one of a first parameter related to the vacuum exhaust system, a second parameter related to the heating and cooling system, a third parameter related to the evaporation source system, a fourth parameter related to the table system, and a fifth parameter related to the process gas system, and the at least one physical quantity being at least one of a film quality characteristic, a mechanical characteristic, and a physical characteristic.

This configuration provides the information necessary for machine learning of film formation conditions. This type of communication method can also be implemented in film formation devices.

According to the present invention, skilled engineers can easily determine the appropriate film formation conditions for the sliding film without relying on years of experience.

FIG. 1 is an overall configuration diagram of a film forming apparatus applied to a machine learning system according to an embodiment. FIG. 1 is an overall configuration diagram of a machine learning system according to an embodiment. 3 is a flowchart illustrating an example of processing in the machine learning system illustrated in FIG. 2 . FIG. 4 is a diagram showing an example of film formation conditions. FIG. 1 is a diagram illustrating an example of a physical quantity. FIG. 1 is an overall configuration diagram of a machine learning system according to a modified example of the present invention.

Figure 1 is an overall configuration diagram of a film forming apparatus applied to a machine learning system according to an embodiment. The film forming apparatus 30 is an apparatus that forms a hard film (hereinafter, a sliding film) on the surface of a workpiece (substrate to be coated) by an arc ion plating method in order to improve the hardness of the surface of the workpiece (substrate to be coated), which is a substrate. For example, sliding parts such as engines and pistons can be used as the workpiece. The arc ion plating method is a type of ion plating method that evaporates a solid material by using a vacuum arc discharge. The arc ion plating method is suitable for forming a sliding film because the ionization rate of the evaporated material is high and a film with excellent adhesion can be formed. The sliding film is, for example, TiN, TiAlN, TiCN, CrN, DLC (diamond-like carbon), etc.

The film forming apparatus 30 includes a vacuum exhaust system 510, a heating and cooling system 520, an evaporation source system 530, a table system 540, a process gas system 550, an etching system 560, and a chamber 570.

The vacuum exhaust system 510 includes an exhaust device 511 and creates a vacuum inside the chamber 570. The exhaust device 511 includes a pump for exhausting air from the chamber 570.

The heating and cooling system 520 includes a heater power supply unit 521 and a heater 522, and heats the workpiece 545. The heater power supply unit 521 is a power supply circuit that supplies power to the heater 522. The heater 522 is provided in the chamber 570, and generates heat using power supplied from the heater power supply unit 521. The heating and cooling system 520 also cools the workpiece 545 by stopping the heater 522 from generating heat.

The evaporation source system 530 is a system that evaporates a target (film-forming material). The evaporation source system 530 includes an arc cathode 531 and an arc power supply unit 532. The arc power supply unit 532 is a power supply circuit that supplies a discharge current to the arc cathode 531. The arc cathode 531 includes a target, and generates a vacuum arc discharge between the target and the inner wall of the chamber 570 by the power supplied from the arc power supply unit 532. When the vacuum arc discharge starts, a molten area called an arc spot with a diameter of several μm is generated on the cathode surface. A high-density current is concentrated in the arc spot, and the cathode surface is instantly melted and evaporated. This vacuum arc discharge forms a film on the surface of the workpiece 545.

In the example of FIG. 1, two pairs of arc cathodes 531 and arc power supplies 532 are shown, but this is just one example, and there may be one pair of arc cathodes 531 and arc power supplies 532, or three or more pairs.

The table system 540 is a rotating table on which the workpiece 545 is mounted. The table system 540 includes a table 541, a table driving unit 542, and a bias power supply unit 543. The table 541 is provided in the chamber 570. The workpiece 545 is placed on the table 541. The table driving unit 542 includes a motor and the like, and rotates the table 541. The bias power supply unit 543 applies a negative potential to the workpiece 545 via the table 541.

The process gas system 550 introduces process gases into the chamber 570 to form a reactive film.

The etching system 560 includes a discharge power supply unit 561, a pair of filament electrodes 562, and a filament (not shown) provided between the pair of filament electrodes 562. The discharge power supply unit 561 is a power supply circuit that supplies a discharge current to the filament via the pair of filament electrodes 562. The etching system 560 generates argon plasma between the arc cathode 531 and the filament, and between the inner wall of the chamber 570 and the filament. The surface of the workpiece 545 is cleaned by the generation of this argon plasma. In this cleaning, the arc cathode 531 and the inner wall of the chamber 570 function as an anode, and the filament functions as a cathode.

The chamber 570 is a container that houses the workpiece 545. The inside of the chamber 570 is evacuated by the vacuum exhaust system 510, and the vacuum state is maintained.

Figure 2 is an overall configuration diagram of a machine learning system in an embodiment. The machine learning system includes a server 10, a communication device 20, and a film forming device 30. The server 10 and the communication device 20 are connected to each other via a network 40 so that they can communicate with each other. The communication device 20 and the film forming device 30 are connected to each other via a network 50 so that they can communicate with each other. The network 40 is, for example, a wide area communication network such as the Internet. The network 50 is, for example, a local area network. The server 10 is, for example, a cloud server composed of one or more computers. The communication device 20 is, for example, a computer owned by a user who uses the film forming device 30. The communication device 20 functions as a gateway that connects the film forming device 30 to the network 40. The communication device 20 is realized by installing dedicated application software on a computer owned by the user himself. Alternatively, the communication device 20 may be a dedicated device provided to the user by the manufacturer of the film forming device 30. The film forming device 30 is the film forming device described in Figure 1.

The configuration of each device will be described in detail below. The server 10 includes a processor 100 and a communication unit 101. The processor 100 is a control device including a CPU and the like. The processor 100 includes a reward calculation unit 110, an update unit 120, a determination unit 130, and a learning control unit 140. Each block of the processor 100 may be realized by the processor 100 executing a machine learning program that causes a computer to function as the server 10 in the machine learning system, or may be realized by a dedicated electrical circuit.

The reward calculation unit 110 calculates the reward for the determination result of at least one film formation condition based on the state variables observed by the state observation unit 321.

The update unit 120 updates a function for determining at least one film formation condition from the state variables observed by the state observation unit 321, based on the reward calculated by the reward calculation unit 110. The action value function described below is used as the function.

The determination unit 130 determines at least one film formation condition that will yield the greatest reward by repeatedly updating the function while changing at least one film formation condition.

The learning control unit 140 is responsible for the overall control of machine learning. The machine learning system of this embodiment learns the film formation conditions by reinforcement learning. Reinforcement learning is a machine learning technique in which an agent (subject of action) selects an action based on the state of the environment, changes the environment based on the selected action, and gives the agent a reward associated with the change in the environment, thereby making the agent learn to select better actions. Q-learning and TD-learning can be adopted as reinforcement learning. In the following explanation, Q-learning is taken as an example. In this embodiment, the reward calculation unit 110, the update unit 120, the decision unit 130, the learning control unit 140, and the state observation unit 321 described later correspond to the agents.

The communication unit 101 is composed of a communication circuit that connects the server 10 to the network 40. The communication unit 101 receives the state variables observed by the state observation unit 321 via the communication device 20. The communication unit 101 transmits the film formation conditions determined by the determination unit 130 to the film formation device 30 via the communication device 20. In this embodiment, the communication unit 101 is an example of a state acquisition unit that acquires state variables.

The communication device 20 includes a transmitter 201 and a receiver 202. The transmitter 201 transmits state variables transmitted from the film forming device 30 to the server 10, and transmits film forming conditions transmitted from the server 10 to the film forming device 30. The receiver 202 receives state variables transmitted from the film forming device 30, and receives film forming conditions transmitted from the server 10.

In addition to the components shown in FIG. 1, the film forming apparatus 30 includes a communication unit 310, a processor 320, a memory 330, a sensor unit 340, and an input unit 350.

The communication unit 310 is a communication circuit for connecting the film forming apparatus 30 to the network 50. The communication unit 310 transmits state variables observed by the state observation unit 321 to the server 10. The communication unit 310 receives the film forming conditions determined by the determination unit 130 of the server 10. The communication unit 310 receives a film forming execution command (described later) determined by the learning control unit 140.

The processor 320 is a control device including a CPU and the like. The processor 320 includes a state observation unit 321, a film formation execution unit 322, and an input determination unit 323. The communication unit 310 transmits the state variables acquired by the state observation unit 321 to the server 10. Each block of the processor 320 is realized, for example, by the CPU executing a machine learning program that causes the processor 320 to function as the film formation device 30 of the machine learning system.

The state observation unit 321 acquires the physical quantity detected by the sensor unit 340 after the film formation is performed. The state observation unit 321 observes state variables including at least one physical quantity related to the performance evaluation of the film formation and at least one film formation condition after the film formation is performed. Specifically, the state observation unit 321 acquires the film formation condition based on the measurement value of the sensor unit 340. The state observation unit 321 also acquires the physical quantity based on the measurement value of the sensor unit 340, etc.

Figure 4 shows an example of film formation conditions. The film formation conditions are roughly classified into medium categories. The medium categories include at least one of a first parameter related to the vacuum exhaust system 510, a second parameter related to the heating and cooling system 520, a third parameter related to the evaporation source system 530, a fourth parameter related to the table system 540, and a fifth parameter related to the process gas system 550. Furthermore, the medium categories may also include a sixth parameter related to the etching system 560.

The first parameter includes at least one of the exhaust speed, the ultimate pressure, the residual gas species, the residual gas partial pressure, and the P-Q characteristic. The exhaust speed is the speed at which the vacuum exhaust system 510 exhausts the air, the residual gas, and the introduced process gas in the chamber 570. The exhaust speed is obtained, for example, by calculation from the performance value of the pump constituting the vacuum exhaust system 510. Alternatively, the exhaust speed may be a measured value calculated from a pressure sensor and an exhaust time. The ultimate pressure is the pressure in the chamber 570 before the start of the film formation process. The ultimate pressure is obtained, for example, by calculation from the performance value of the pump constituting the vacuum exhaust system 510. Alternatively, the ultimate pressure may be a measured value of a pressure sensor. The residual gas species is gas remaining in the chamber 570 and is an impurity. The residual gas species is, for example, nitrogen, oxygen, moisture, and hydrogen. The residual gas species is determined based on the partial pressure of the residual gas described later. The residual gas partial pressure is the partial pressure of multiple residual gases remaining in the chamber 570. The residual gas partial pressure is obtained by measurement using a vacuum residual gas monitor such as a quadrupole mass spectrometer. The P-Q characteristic is a characteristic that shows the relationship between the pressure inside the chamber (P) and the flow rate (Q). The P-Q characteristic is obtained by calculation from, for example, the flow rate of the gas inside the chamber 570 detected by a flow rate sensor and the measurement value of the pressure sensor.

The second parameters include at least one of the heater temperature, workpiece temperature, heater heating rate, workpiece heating rate, heater output, heater temperature accuracy, workpiece temperature accuracy, heater temperature/workpiece temperature, heater temperature distribution, workpiece temperature distribution, cooling gas type, cooling gas pressure, and workpiece cooling rate.

The heater temperature is the temperature of the heater 522. The heater temperature is, for example, a measurement value of a temperature sensor (thermocouple). The work temperature is the temperature of the work 545. The work temperature is, for example, a measurement value of a temperature sensor provided near the work 545. The heater heating rate is the rate of change of the heater temperature when the heater 522 heats up. The heater heating rate is obtained from the time series change in the heater temperature. The work heating temperature is the rate of change of the work temperature when the work 545 heats up. The work heating rate is obtained from the time series change in the work temperature.

The heater output is the output of the heater 522. The heater output is obtained by calculation from the setting value of the heater power supply unit 521. The heater output may be calculated from the measured values of the current value and voltage value supplied to the heater by a sensor.

The heater temperature accuracy is a value that indicates the variation in the heater temperature. The heater temperature accuracy is calculated from past heater temperature measurements. The work temperature accuracy is a value that indicates the variation in the work temperature. The work temperature accuracy is calculated from past work temperature measurements. The heater temperature/work temperature is the response characteristic of the heater 522 to the work 545.

The heater temperature distribution is the temperature distribution of the heater 522. The heater temperature distribution is obtained from the measurement values of multiple temperature sensors provided around the heater 522. The workpiece temperature distribution is the temperature distribution of the workpiece 545. The workpiece temperature distribution is obtained from the measurement values of multiple temperature sensors provided around the workpiece 545.

The cooling gas type is information indicating the type of gas used to cool the inside of the chamber 570, and is a value that is input in advance. The cooling gas pressure is the pressure of the cooling gas. The cooling gas pressure is a value measured by a pressure sensor installed in the chamber 570. The workpiece cooling rate is the cooling rate of the workpiece 545. The workpiece cooling rate is obtained from the time series change in the workpiece temperature detected by a temperature sensor installed near the workpiece 545.

The third parameter includes at least one of the target composition, the target thickness, the target manufacturing method, the arc discharge voltage, the arc discharge current, the evaporation source magnetic field, the evaporation source coil current, and the arc ignition characteristics. The target composition is the composition of the material that constitutes the target. The target thickness is the thickness of the target. The target manufacturing method is the manufacturing method of the target. The target composition, the target thickness, and the target manufacturing method are input values that are input in advance.

The arc discharge voltage is the voltage supplied by the arc power supply unit 532 to the arc cathode 531, and is a value measured by a sensor. The arc discharge current is the current supplied by the arc power supply unit 532 to the arc cathode 531, and is a value measured by a sensor.

The source magnetic field is the position and strength of the magnetic field emitted by the permanent magnetic flux contained in the source system 530. The source magnetic field is an input value that is input in advance. The source coil current is the current flowing through the coil contained in the source system 530, and is a measured value by a sensor. The arc ignition characteristics are the behavior of the voltage and current on the arc surface when the arc ignites. The arc ignition characteristics are obtained from the measured values of the arc discharge voltage and arc discharge current at a certain timing.

The fourth parameter includes at least one of the bias voltage, bias current, number of OLs, OL time change, bias voltage waveform, bias current waveform, workpiece rotation speed, workpiece shape, workpiece loading amount, workpiece loading method, and workpiece material.

The bias voltage is the bias voltage supplied by the bias power supply unit 543 to the workpiece 545, and is a value measured by the sensor. The bias current is the bias current supplied by the bias power supply unit 543 to the workpiece 545, and is a value measured by the sensor.

The number of OL (overload) is the number of abnormal discharges in the table system or the workpiece, and is a measurement value by a sensor. The OL time change is the number of OLs per unit time. The bias voltage waveform is the waveform of the bias voltage, and is obtained from the measurement value by the sensor. The bias voltage waveform is a voltage waveform, particularly during pulse bias. The bias current waveform is the waveform of the bias current, and is obtained from the measurement value by the sensor. The workpiece rotation speed is the number of rotations per unit time of the workpiece 545, and includes the number of rotations per unit time of the table 541 and the number of rotations per unit time when the workpiece 545 rotates on the table 541. The workpiece rotation speed is, for example, a detection value by a sensor. The workpiece shape is a numerical value indicating the shape of the workpiece 545, and is an input value input in advance. The workpiece loading amount is the amount of the workpiece 545 loaded (for example, weight), and is an input value input in advance. The workpiece loading method is the method of loading the workpiece 545 on the table 541, and is an input value input in advance. The workpiece material is the material of the workpiece 545 and is a pre-entered input value.

The fifth parameter includes at least one of a gas flow rate, a gas type, and a gas pressure. The gas flow rate is the flow rate of the process gas. The gas type is information indicating the type of the process gas. The gas pressure is the pressure of the process gas. These are, for example, detection values of a sensor.

The sixth parameter includes at least one of a filament heating current, a filament heating voltage, a filament diameter, a discharge current, and a discharge voltage. The filament heating current is a heating current for heating a pair of filament electrodes 562 that constitute the etching system 560, and is a measured value by a sensor. The filament heating voltage is a heating voltage for heating a pair of filament electrodes 562, and is a measured value by a sensor.

The filament diameter is the diameter of each of the pair of filament electrodes 562, and is a value that is input in advance. The filament diameter may be calculated. The discharge current is the discharge current of the pair of filament electrodes 562, and is a value measured by a sensor. The discharge voltage is the discharge voltage of the pair of filament electrodes 562, and is a value measured by a sensor.

Figure 5 is a diagram showing an example of physical quantities. The physical quantities are roughly classified into intermediate categories. The intermediate categories include at least one of film quality characteristics, mechanical characteristics, and physical characteristics. The film quality characteristics include at least one of film thickness, roughness, surface properties, composition, crystal structure, film microstructure, crystallinity, crystal grain size, residual stress, density, particle amount, and particle size.

Film thickness is the thickness of the film. Surface texture is the surface morphology, including surface roughness. Composition is the composition of the film. Crystal structure is the crystal structure of the film. Film microstructure is the general meaning, and refers to the microstructure, such as crystal morphology and orientation. Crystallinity is the proportion that is crystalline. Crystal grain size is the size of the crystal grains. Residual stress is the internal stress of the film.

Film thickness is obtained by film thickness gauge. Roughness is obtained by roughness gauge. Surface texture is obtained by microscope or roughness gauge. Composition is obtained by X-ray spectroscopy. Crystal structure, film microstructure, crystallinity, grain size, and residual stress are obtained by X-ray diffraction or electron microscope.

Density is the density of the particles that make up the coating. Particle amount (defect amount) is the amount of dirt contained in the coating. Particle size (defect size) is the size of the dirt contained in the coating. Density is obtained by X-ray reflection method. Particle amount and particle size are obtained by microscope or image processing.

The mechanical properties include at least one of hardness, elastic modulus, and wear resistance. Hardness is measured using a hardness tester or a nanoindenter. Elastic modulus is measured using a nanoindenter. Wear resistance is measured using a sliding test or a wear resistance test.

The physical properties include at least one of the coefficient of friction, oxidation resistance, adhesion, thermal conductivity, adhesion, corrosion resistance, and surface chemical affinity. The coefficient of friction is obtained by a sliding test. The oxidation resistance is obtained by X-ray analysis or composition analysis. The adhesion indicates the degree of adhesion of the coating to the substrate, and is obtained by an indentation method or a scratch test. The thermal conductivity is obtained by a thermal conductivity measurement. The adhesion is obtained by a sliding test or a microscope observation. The corrosion resistance indicates the resistance of the coating to corrosion, and is obtained by a corrosive liquid spray test or an immersion test. The surface chemical affinity indicates the chemical affinity between external environmental substances and the coating surface, and is obtained by surface chemical analysis.

Refer back to FIG. 2. The film formation execution unit 322 controls the film formation operation of the film formation apparatus 30. The input determination unit 323 automatically or manually determines whether or not it is a mass production process. When the input determination unit 323 automatically determines whether or not it is a mass production process, it determines that the film formation apparatus 30 is in a mass production process if the number of inputs of the condition number input to the input unit 350 exceeds a reference number. The condition number is an identification number for identifying one film formation condition. The film formation conditions identified by the condition number include at least the film formation conditions indicated as Input among the film formation conditions shown in FIG. 4.

When manually determining whether or not a mass production process is in progress, the input determination unit 323 determines that the film forming apparatus 30 is in a mass production process if data indicating a mass production process is input to the input unit 350. If the film forming apparatus 30 is in a mass production process, the film forming apparatus 30 does not perform machine learning.

The memory 330 is, for example, a non-volatile storage device, and stores the finally determined optimal film formation conditions, etc. The sensor unit 340 is various sensors used to measure the film formation conditions exemplified in FIG. 4 and the physical quantities exemplified in FIG. 5. The input unit 350 is an input device such as a keyboard and a mouse.

Figure 3 is a flowchart showing an example of processing in the machine learning system shown in Figure 2. In step S1, the learning control unit 140 acquires the input values of the film formation conditions input by the user using the input unit 350. The input values acquired here are the input values for the film formation conditions listed as Input among the film formation conditions listed in Figure 4.

In step S2, the learning control unit 140 determines at least one film formation condition and a setting value for the film formation condition. Here, the film formation condition to be set is at least one film formation condition for which a setting value can be set, among the film formation conditions listed in FIG. 4, other than the film formation condition described as Input. Here, the setting value of the film formation condition to be determined corresponds to an action in reinforcement learning.

Specifically, the learning control unit 140 randomly selects a setting value for each of the film formation conditions to be set. Here, the setting value is randomly selected from within a predetermined range for each of the film formation conditions. As a method for selecting the setting value of the film formation condition, for example, the ε-greedy method can be adopted.

In step S3, the learning control unit 140 transmits a film formation execution command to the film formation device 30, thereby causing the film formation device 30 to start a film formation operation. When the film formation execution command is received by the communication unit 310, the film formation execution unit 322 sets the film formation conditions according to the film formation execution command and starts the film formation operation. The film formation execution command includes the input values of the film formation conditions set in step S1 and the setting values of the film formation conditions determined in step S2.

When the film formation operation is completed, the state observation unit 321 observes the state variables (step S4). Specifically, the state observation unit 321 acquires, as state variables, the physical quantities related to the film formation evaluation described in FIG. 5 and the film formation conditions described in FIG. 4 whose states are observed by a sensor or the like. The physical quantities may be input to the film formation apparatus 30, for example, by a user operating the input unit 350, or may be input to the film formation apparatus 30 by communication between the film formation apparatus 30 and a measuring instrument that measures the physical quantities. The state observation unit 321 transmits the acquired state variables to the server 10 via the communication unit 310.

In step S5, the determination unit 130 evaluates the physical quantity. Here, the determination unit 130 evaluates the physical quantity by determining whether or not the physical quantity to be evaluated (hereinafter referred to as the target physical quantity) among the physical quantities acquired in step S4 has reached a predetermined reference value. The target physical quantity is one or more of the physical quantities listed in FIG. 5. When there are multiple target physical quantities, there will be multiple reference values corresponding to the respective target physical quantities. For example, a predetermined value indicating that the coating has reached a certain standard can be used as the reference value.

The reference value may be a value including, for example, an upper limit value and a lower limit value. In this case, when the target physical quantity falls within the range between the upper limit value and the lower limit value, it is determined that the reference value has been reached. The reference value may be a single value. In this case, when the target physical quantity exceeds the reference value or falls below the reference value, it is determined that a certain criterion is satisfied.

When the determination unit 130 determines that the target physical quantity has reached the reference value (YES in step S6), it outputs the film formation conditions set in step S2 as the final film formation conditions (step S7). On the other hand, when the determination unit 130 determines that the physical quantity has not reached the reference value (NO in step S6), it proceeds to step S8. Note that when there are multiple target physical quantities, the determination unit 130 may determine YES in step S6 when all of the target physical quantities have reached the reference value.

In step S8, the reward calculation unit 110 determines whether the target physical quantity is approaching a reference value. If the target physical quantity is approaching a reference value (YES in step S8), the reward calculation unit 110 increases the reward for the agent (step S9). On the other hand, if the target physical quantity is not approaching the reference value (NO in step S8), the reward calculation unit 110 decreases the reward for the agent (step S10). In this case, the reward calculation unit 110 may increase or decrease the reward according to a predetermined increase or decrease value of the reward. Note that, if there are multiple target physical quantities, the reward calculation unit 110 may perform the determination in step S8 for each of the multiple target physical quantities. In this case, the reward calculation unit 110 may increase or decrease the reward for each of the multiple target physical quantities based on the determination result in step S8. Also, different values may be adopted for the increase or decrease value of the reward depending on the target physical quantity.

In step S11, the update unit 120 updates the action value function using the reward given to the agent. The Q-learning adopted in this embodiment is a method for learning a Q value (Q(s, a)), which is the value of selecting an action a under a certain environmental state s. The environmental state s _t corresponds to the state variable of the above flow. In Q-learning, an action a with the highest Q(s, a) is selected in a certain environmental state s. In Q-learning, various actions a are taken under a certain environmental state s by trial and error, and the correct Q(s, a) is learned using the reward at that time. The update formula for the action value function Q(s _t , a _t ) is shown in the following formula (1).

Here, s _t and a _t respectively represent the environmental state and the action at time t. The action a _t changes the environmental state to s _t+1 , and the reward r _t+1 is calculated based on the change in the environmental state. The term with max is the Q value (Q(s _{t+1 , a)) multiplied by γ when the most valuable action a known at that time is selected under the environmental state s t+1} . Here, γ is the discount rate and takes a value of 0<γ≦1 (usually 0.9 to 0.99). α is the learning coefficient and takes a value of 0<α≦1 (usually around 0.1).

This update formula increases Q(s _t , a _t ) if γ·maxQ( _{s t+1 , a ) based on the Q value when the best action is taken in the next environmental state s t+1} by action a is greater than Q(s _t , a _t ), which is the Q value of action a in state s. On the other hand, this update formula decreases Q(s _t , a _t ) if γ·maxQ(s t _{+ 1} , a ) is smaller than Q(s _t , a _t ). In other words, the value of a certain action a in a certain state s _t is made to approach the value of the best action in the next state s _t+1 resulting from that action. This determines the optimal state for forming a film on the workpiece 545, that is, at least one optimal film forming condition.

When the processing of step S11 is completed, the processing returns to step S2, where the setting value of the selected film formation condition is changed, or an unselected film formation condition is selected as the next film formation condition, and the action value function is updated in the same manner. Although the update unit 120 updates the action value function, the present invention is not limited to this, and the action value table may be updated.

Q(s, a) may be stored in table format with values for all state-action pairs (s, a). Alternatively, Q(s, a) may be represented by an approximation function that approximates values for all state-action pairs (s, a). This approximation function may be composed of a multi-layered neural network. In this case, the neural network may perform online learning by learning data obtained by actually operating the kneading device 300 in real time and reflecting the data in the next action. This realizes deep reinforcement learning.

Conventionally, in film-forming devices, film-forming conditions have been developed by changing the film-forming conditions so as to obtain a good sliding film. To obtain a good sliding film, it is necessary to find a relationship between the evaluation of the sliding film and the film-forming conditions. However, as shown in Figure 4, the number of types of film-forming conditions is enormous, so an extremely large number of physical models are required to define such relationships, and it has been found that it is difficult to describe such relationships using physical models. Furthermore, to construct such a physical model, it is necessary to artificially find which parameters affect the evaluation of which sliding film, and this construction is difficult.

According to this embodiment, at least one of the first to sixth parameters described above and at least one physical quantity of the film quality characteristics, mechanical characteristics, and physical characteristics related to the film formation performance evaluation are observed as state variables. Then, based on the observed state variables, a reward for the result of determining the film formation conditions is calculated, and based on the calculated reward, an action value function for determining the film formation conditions from the state variables is updated, and this updating is repeated to learn the film formation conditions that will obtain the most reward. In this way, in this embodiment, the film formation conditions are determined by machine learning without using the above-mentioned physical model. As a result, this embodiment can easily determine appropriate film formation conditions for the sliding film.

The present invention can be modified in the following ways:

(1) FIG. 6 is an overall configuration diagram of a machine learning system according to a modified example of the present invention. The machine learning system according to this modified example is composed of a single film forming apparatus 30A. The film forming apparatus 30A includes a processor 320A, an input unit 391, and a sensor unit 392. The processor 320A includes a machine learning unit 370 and a film forming unit 380. The machine learning unit 370 includes a reward calculation unit 371, an update unit 372, a determination unit 373, and a learning control unit 374. The reward calculation unit 371 to the learning control unit 374 are the same as the reward calculation unit 110 to the learning control unit 140 shown in FIG. 2, respectively. The state observation unit 381, the film formation execution unit 382, and the input judgment unit 383 are the same as the state observation unit 321, the film formation execution unit 322, and the input judgment unit 323 shown in FIG. 2, respectively. The input unit 391 and the sensor unit 392 are the same as the input unit 350 and the sensor unit 340 shown in FIG. 2, respectively. In this modified example, the status observation unit 381 is an example of a status acquisition unit that acquires status information.

In this way, the machine learning system according to this modified example can learn optimal film formation conditions using the film formation apparatus 30A alone.

(2) In the above flow, the state variables are observed after the film formation operation is completed, but this is just one example, and multiple state variables may be observed during one film formation operation. For example, if the state variables are composed only of parameters that can be measured instantly, multiple state variables can be observed during one film formation operation. This shortens the learning time.

(3) In the above embodiment, the film forming apparatus 30 is an apparatus that forms a film by the arc ion plating method, but the present invention is not limited to this, and may be an apparatus that forms a film by other physical vapor deposition methods such as evaporation.

10: Server 30: Film forming apparatus 100: Processor 110: Reward calculation unit 120: Update unit 130: Determination unit 140: Learning control unit 510: Vacuum exhaust system 520: Heating and cooling system 530: Evaporation source system 540: Table system 550: Process gas system 560: Etching system 570: Chamber 574: Learning control unit

Claims (16)

A machine learning method in which a machine learning device determines film formation conditions of a film formation device that forms a slide coating on a substrate, comprising:
The film forming apparatus includes a vacuum exhaust system for evacuating a chamber, a heating and cooling system for heating and cooling the chamber, an evaporation source system for evaporating a target, a table system for placing a workpiece thereon, a process gas system for introducing a process gas into the chamber, and an etching system;
acquiring state variables including at least one physical quantity related to a performance evaluation of the slide coating and at least one film-forming condition;
calculating a reward for a result of determining the at least one film formation condition based on the state variables;
updating a function for determining the at least one film forming condition from the state variables based on the reward;
By repeatedly updating the function, a film forming condition that can obtain the maximum reward is determined;
the at least one film formation condition is at least one of a first parameter related to the vacuum exhaust system, a second parameter related to the heating and cooling system, a third parameter related to the evaporation source system, a fourth parameter related to the table system, and a fifth parameter related to the process gas system;
the at least one physical quantity is at least one of a film quality characteristic, a mechanical characteristic, and a physical characteristic;
the fifth parameter is at least one of a flow rate of the process gas, a type of the process gas, and a pressure of the process gas;
the at least one physical quantity related to performance evaluation of the slide film is a performance evaluation result of a slide film formed on the basis of an input value of at least one film-forming condition and a setting value for the at least one film-forming condition determined by machine learning;
The set value is randomly selected from a predetermined range.
Machine learning methods. The first parameter is at least one of an exhaust speed, an ultimate pressure, a residual gas type, a residual gas partial pressure, and a PQ characteristic.
The machine learning method of claim 1. The second parameter is at least one of a heater temperature of a heater constituting the heating and cooling system, a work temperature which is the temperature of the work, a temperature rise rate of the heater, a temperature rise rate of the work, an output of the heater, a temperature accuracy of the heater, a temperature accuracy of the work, a response characteristic of the heater temperature and the work temperature, a temperature distribution of the heater, and a temperature distribution of the work.
The machine learning method according to claim 1 or 2. the third parameter is at least one of a composition of the target, a thickness of the target, a manufacturing method of the target, an arc discharge voltage, an arc discharge current, an evaporation source magnetic field, an evaporation source coil current, and an arc ignition characteristic;
The machine learning method according to any one of claims 1 to 3. The fourth parameter includes at least one of a bias voltage for the workpiece, a bias current for the workpiece, a number of abnormal discharges, a time change in the abnormal discharge, a waveform of the bias voltage, a waveform of the bias current, a rotation speed of the workpiece, a shape of the workpiece, a mounting amount of the workpiece, a mounting method of the workpiece, and a material of the workpiece.
The machine learning method according to any one of claims 1 to 4. The at least one deposition condition further includes a sixth parameter related to the etching system.
The machine learning method according to any one of claims 1 to 5. the sixth parameter being at least one of a heating current for heating a filament of the etching system, a heating voltage for heating the filament, a diameter of the filament, a discharge current of the filament, and a discharge voltage of the filament;
The machine learning method of claim 6. The film quality characteristics include at least one of a film thickness, a roughness, a surface condition, a composition, a crystal structure, a film microstructure, a crystallinity, a crystal grain size, a residual stress, a density, an amount of defects, and a size of defects in the slide coating.
The machine learning method according to any one of claims 1 to 7. The mechanical properties of the slide coating include at least one of hardness, elastic modulus, and wear resistance.
The machine learning method according to any one of claims 1 to 8. The physical properties of the slide coating include at least one of a friction coefficient, an oxidation resistance, an adhesion, a thermal conductivity, an adhesive property, a corrosion resistance, and a surface chemical affinity.
The machine learning method according to any one of claims 1 to 9. The function is updated in real time using deep reinforcement learning.
The machine learning method according to any one of claims 1 to 10. In the calculation of the reward, if the at least one physical quantity approaches a predetermined reference value corresponding to each physical quantity, the reward is increased, and if the at least one physical quantity does not approach a reference value corresponding to each physical quantity, the reward is decreased.
The machine learning method according to any one of claims 1 to 11. A machine learning device that determines film formation conditions for a film formation device that forms a slide coating on a substrate,
The film forming apparatus includes a vacuum exhaust system for evacuating a chamber, a heating and cooling system for heating and cooling the chamber, an evaporation source system for evaporating a target, a table system for placing a workpiece thereon, a process gas system for introducing a process gas into the chamber, and an etching system;
a state acquisition unit that observes state variables including at least one physical quantity related to a performance evaluation of the slide coating and at least one film formation condition;
a reward calculation unit that calculates a reward for a result of determining the at least one film formation condition based on the state variable;
an update unit that updates a function for determining the at least one film forming condition based on the state variable based on the reward;
a determination unit that determines a film forming condition under which the reward is most maximized by repeatedly updating the function;
the at least one film formation condition is at least one of a first parameter related to the vacuum exhaust system, a second parameter related to the heating and cooling system, a third parameter related to the evaporation source system, a fourth parameter related to the table system, and a fifth parameter related to the process gas system;
the at least one physical quantity is at least one of a film quality characteristic, a mechanical characteristic, and a physical characteristic;
the fifth parameter is at least one of a flow rate of the process gas, a type of the process gas, and a pressure of the process gas;
the at least one physical quantity related to performance evaluation of the slide film is a performance evaluation result of a slide film formed on the basis of an input value of at least one film-forming condition and a setting value for the at least one film-forming condition determined by machine learning;
The set value is randomly selected from a predetermined range.
Machine learning device. A computer-readable machine learning program that causes a computer to function as a machine learning device that determines film formation conditions for a film formation device that forms a slide coating on a substrate,
The film forming apparatus includes a vacuum exhaust system for evacuating a chamber, a heating and cooling system for heating and cooling the chamber, an evaporation source system for evaporating a target, a table system for placing a workpiece thereon, a process gas system for introducing a process gas into the chamber, and an etching system;
a state acquisition unit that observes state variables including at least one physical quantity related to a performance evaluation of the slide coating and at least one film formation condition;
a reward calculation unit that calculates a reward for a result of determining the at least one film formation condition based on the state variable;
an update unit that updates a function for determining the at least one film forming condition based on the state variable based on the reward;
a determination unit that determines a film forming condition under which the reward is most maximized by repeatedly updating the function;
the at least one film formation condition is at least one of a first parameter related to the vacuum exhaust system, a second parameter related to the heating and cooling system, a third parameter related to the evaporation source system, a fourth parameter related to the table system, and a fifth parameter related to the process gas system;
the at least one physical quantity is at least one of a film quality characteristic, a mechanical characteristic, and a physical characteristic;
the fifth parameter is at least one of a flow rate of the process gas, a type of the process gas, and a pressure of the process gas;
the at least one physical quantity related to performance evaluation of the slide film is a performance evaluation result of a slide film formed on the basis of an input value of at least one film-forming condition and a setting value for the at least one film-forming condition determined by machine learning;
The set value is randomly selected from a predetermined range.
Machine learning programs. A communication method for a film-forming device when machine learning film-forming conditions of the film-forming device that forms a slide coating on a substrate, comprising:
The film forming apparatus includes a vacuum exhaust system for evacuating a chamber, a heating and cooling system for heating and cooling the chamber, an evaporation source system for evaporating a target, a table system for placing a workpiece, a process gas system for introducing a process gas into the chamber, an etching system, and a communication unit;
observing state variables including at least one physical quantity related to a performance evaluation of the slide coating and at least one film-forming condition;
Transmitting the state variables onto a network and receiving at least one film formation condition that has been machine-learned;
the at least one film formation condition is at least one of a first parameter related to the vacuum exhaust system, a second parameter related to the heating and cooling system, a third parameter related to the evaporation source system, a fourth parameter related to the table system, and a fifth parameter related to the process gas system;
the at least one physical quantity is at least one of a film quality characteristic, a mechanical characteristic, and a physical characteristic;
the fifth parameter is at least one of a flow rate of the process gas, a type of the process gas, and a pressure of the process gas;
the at least one physical quantity related to performance evaluation of the slide film is a performance evaluation result of a slide film formed on the basis of an input value of at least one film-forming condition and a setting value for the at least one film-forming condition determined by machine learning;
The set value is randomly selected from a predetermined range.
Communication method. A film-forming apparatus for forming a slide film on a substrate, comprising:
a vacuum pumping system for evacuating the chamber;
a heating and cooling system for heating and cooling the chamber;
an evaporation source system for evaporating the target;
A table system on which a workpiece is placed;
a process gas system for introducing a process gas into the chamber;
An etching system;
a state observing unit that observes state variables including at least one physical quantity related to a performance evaluation of the slide coating and at least one film-forming condition;
a communication unit that transmits the state variables onto a network and receives the film formation conditions that have been machine-learned;
the at least one film formation condition is at least one of a first parameter related to the vacuum exhaust system, a second parameter related to the heating and cooling system, a third parameter related to the evaporation source system, a fourth parameter related to the table system, and a fifth parameter related to the process gas system;
the at least one physical quantity is at least one of a film quality characteristic, a mechanical characteristic, and a physical characteristic;
the fifth parameter is at least one of a flow rate of the process gas, a type of the process gas, and a pressure of the process gas;
the at least one physical quantity related to performance evaluation of the slide film is a performance evaluation result of a slide film formed on the basis of an input value of at least one film-forming condition and a setting value for the at least one film-forming condition determined by machine learning;
The set value is randomly selected from a predetermined range.
Film forming equipment.

Images