JP7401405B2 - Machine learning method, machine learning device, machine learning program, communication method, and film forming device - Google Patents

Description

The present disclosure relates to a technique for learning film forming conditions through machine learning.

BACKGROUND ART In recent years, in products such as wristwatches and mobile phones, a film has been formed on the surface of a base material for the purpose of increasing decorativeness. A film formed for this purpose is hereinafter referred to as a decorative film. For example, Patent Document 1 describes a decorative article having a black hard coating formed on a base material using a film forming method such as chemical vapor deposition (CVD) or physical vapor deposition (PVD). A decorative article is disclosed in which the black hard coating includes a gradient layer made of DLC (diamond-like carbon), and the hydrogen content in the gradient layer increases as the distance from the base material increases.

JP 2018-53365 Publication

However, conventionally, the conditions for forming such a decorative film have been determined by skilled engineers relying on years of experience, and it has been difficult to determine them easily.

The present invention was made in order to solve the above-mentioned problems, and is a machine that easily determines film forming conditions for appropriately forming a decorative film without relying on the years of experience of skilled engineers. The purpose is to provide learning devices, etc.

In recent years, various services related to machine learning, including deep learning, have been provided on the cloud, and it has become easier for users to use these services. Therefore, the present inventor conceived the present invention based on the knowledge that appropriate film forming conditions for the decorative film can be easily determined by machine learning the film forming conditions and the physical quantities related to performance evaluation of the decorative film. reached.

A machine learning method according to one aspect of the present invention is a machine learning method in which a machine learning device determines film forming conditions of a film forming apparatus that forms a decorative film on a base material, wherein the film forming apparatus has a chamber. An evacuation system for creating a vacuum, a heating and cooling system for heating and cooling the chamber, an evaporation source system for evaporating a target, a table system for placing a workpiece, and a process gas for introducing a process gas into the chamber. and an etching system, obtain a state variable including at least one physical quantity related to performance evaluation of the decorative film, and at least one film forming condition, and based on the state variable, perform the at least one film forming process. By calculating a reward for the result of determining the conditions, updating a function for determining the at least one film forming condition from the state variable based on the reward, and repeating the updating of the function, the reward is maximized. The at least one film forming condition includes a first parameter related to the vacuum evacuation system, a second parameter related to the heating/cooling system, a third parameter related to the evaporation source system, and a third parameter related to the evaporation source system. The at least one physical quantity is at least one of a fourth parameter related to the table system and a fifth parameter related to the process gas system, and the at least one physical quantity is at least one of a film quality property and a physical property.

According to this configuration, the first parameter related to the vacuum evacuation system, the second parameter related to the heating and cooling system, the third parameter related to the evaporation source system, the fourth parameter related to the table system, and the fifth parameter related to the process gas system. At least one of the film forming conditions and at least one physical quantity among film quality characteristics and physical characteristics related to performance evaluation of the decorative coating are observed as state variables. Then, based on the observed state variables, a reward for the determination result of the deposition conditions is calculated, and based on the calculated reward, the function for determining the deposition conditions from the state variables is updated. By repeating the process, the deposition conditions that yield the most rewards are learned. Further, in this configuration, machine learning is performed using at least one of film quality characteristics and physical characteristics as a physical quantity related to evaluation of the decorative film. Therefore, with this configuration, appropriate film forming conditions for the decorative film can be easily determined.

In the above configuration, the first parameter may be at least one of pumping speed, ultimate pressure, residual gas species, residual gas partial pressure, and PQ characteristic.

According to this configuration, machine learning is performed using at least one of the pumping speed, ultimate pressure, residual gas type, residual gas partial pressure, and PQ characteristic as a film forming condition related to the vacuum pumping system. Appropriate deposition conditions can be determined by taking into account the state of the exhaust system.

In the above configuration, the second parameter includes a heater temperature of a heater constituting the heating/cooling system, a workpiece temperature that is the temperature of the workpiece, a temperature increase rate of the heater, a temperature increase rate of the workpiece, an output of the heater, It may be at least one of temperature accuracy of the heater, temperature accuracy of the workpiece, response characteristics of the heater temperature and workpiece temperature, temperature distribution of the heater, and temperature distribution of the workpiece.

According to this configuration, heater temperature, workpiece temperature, heater heating rate, workpiece heating rate, heater output, heater temperature accuracy, workpiece temperature accuracy, heater temperature response characteristic, workpiece temperature response characteristic, Since machine learning is performed using at least one of the temperature distribution of the heater and the temperature distribution of the workpiece as the film formation conditions related to the heating and cooling system, it is necessary to take the state of the heating and cooling system into account when determining the appropriate film formation conditions. You can decide.

In the above configuration, the third parameter is at least one of the composition of the target, the thickness of the target, the manufacturing method of the target, the arc discharge voltage, the arc discharge current, the evaporation source magnetic field, the evaporation source coil current, and the arc ignition characteristic. It may be one.

According to this configuration, at least one of the target composition, target thickness, target manufacturing method, arc discharge voltage, arc discharge current, evaporation source magnetic field, evaporation source coil current, and arc ignition characteristics is related to the evaporation source system. Since machine learning is performed based on the conditions, it is possible to determine appropriate film-forming conditions by taking into consideration the state of the evaporation source system.

In the above configuration, the fourth parameter includes a bias voltage for the workpiece, a bias current for the workpiece, the number of abnormal discharges, a temporal change in the abnormal discharge, a waveform of the bias voltage, a waveform of the bias current, and a rotational speed of the workpiece. , the shape of the workpiece, the amount of the workpiece to be loaded, the method of mounting the workpiece, and the material of the workpiece.

According to this configuration, bias voltage, bias current, number of abnormal discharges, time change of abnormal discharge, waveform of bias voltage, waveform of bias current, workpiece rotation speed, workpiece shape, workpiece loading amount, workpiece loading method Since machine learning is performed on at least one of the film formation conditions related to the table system and the material of the workpiece, appropriate film formation conditions can be determined taking into consideration the state of the table system.

In the above configuration, the fifth parameter may be at least one of the flow rate of the process gas, the type of the process gas, and the pressure of the process gas.

According to this configuration, since machine learning is performed using at least one of the flow rate of the process gas, the type of process gas, and the pressure of the process gas as a film forming condition related to the process gas system, the state of the process gas system is taken into account. can be used to determine appropriate film-forming conditions.

In the above configuration, the at least one film forming condition may further include a sixth parameter related to the etching system.

According to this configuration, since machine learning is performed in consideration of the film formation conditions related to the etching system, appropriate film formation conditions can be determined by taking the state of the etching system into consideration.

In the above configuration, the sixth parameter includes a heating current for heating the filament of the etching system, a heating voltage for heating the filament, a diameter of the filament, a discharge current of the filament, and a discharge voltage of the filament. It may be at least one of the following.

According to this configuration, machine learning is performed using at least one of the filament heating current, the filament heating voltage, the filament diameter, the filament discharge current, and the filament discharge voltage as a film forming condition related to the etching system. Appropriate film formation conditions can be determined by taking into account the state of the etching system.

In the above configuration, the film quality characteristics include film thickness, roughness, surface texture, composition, crystal structure, film microstructure, crystallinity, crystal grain size, residual stress, density, defect amount, and defect size in the decorative film. The physical property may include an optical property of the decorative coating.

According to this configuration, at least one of the film thickness, roughness, surface texture, composition, crystal structure, film microstructure, crystallinity, crystal grain size, residual stress, density, defect amount, and defect size in the decorative film is It is adopted as a film quality characteristic. Furthermore, the optical properties of the decorative film are employed as physical properties. Therefore, it is possible to easily obtain film-forming conditions that allow obtaining a decorative film that satisfies these film quality characteristics and physical characteristics.

In the above configuration, the function may be updated in real time using deep reinforcement learning.

According to this aspect, since the function is updated in real time using deep reinforcement learning, the function can be updated accurately and quickly.

In the above configuration, in calculating the reward, if the at least one physical quantity approaches a predetermined reference value corresponding to each physical quantity, the reward is increased, and the at least one physical quantity is set to a reference value corresponding to each physical quantity. If it is not close to , the reward may be decreased.

According to this aspect, the reward is increased as the physical quantity approaches the reference value, so the physical quantity can quickly reach the reference value.

Each process of the machine learning method described above may be implemented by a machine learning device, or may be implemented in a machine learning program and distributed. This machine learning device may be configured with a server or a film forming device.

A communication method according to another aspect of the present invention is a communication method for a film forming apparatus when machine learning the film forming conditions of the film forming apparatus that forms a decorative film on a base material, The system includes an evacuation system to evacuate the chamber, a heating and cooling system to heat and cool the chamber, an evaporation source system to evaporate the target, a table system to place the workpiece, and a process gas to the chamber. It includes a process gas system to be introduced, an etching system, and a communication section, and a state variable including at least one physical quantity related to performance evaluation of the decorative film and at least one film forming condition is observed, and the state variable is connected to a network. at least one film-forming condition that has been machine learned, and the at least one film-forming condition includes a first parameter related to the vacuum evacuation system, a second parameter related to the heating and cooling system, and a second parameter related to the evaporation system. at least one of a third parameter related to the source system, a fourth parameter related to the table system, and a fifth parameter related to the process gas system, and the at least one physical quantity is selected from among a film quality property and a physical property. At least one of the following.

According to this configuration, information necessary for machine learning of film forming conditions is provided. Such a communication method can also be implemented in a film forming apparatus.

According to the present invention, appropriate film forming conditions for a decorative film can be easily determined by a skilled engineer without relying on many years of experience.

1 is an overall configuration diagram of a film forming apparatus applied to a machine learning system according to an embodiment. FIG. 1 is an overall configuration diagram of a machine learning system in an embodiment. 3 is a flowchart showing an example of processing in the machine learning system shown in FIG. 2. FIG. It is a figure showing an example of film-forming conditions. FIG. 3 is a diagram showing an example of physical quantities. FIG. 2 is an overall configuration diagram of a machine learning system according to a modification of the present invention.

FIG. 1 is an overall configuration diagram of a film forming apparatus applied to a machine learning system according to an embodiment. The film forming apparatus 30 is an apparatus that forms a film (hereinafter referred to as a decorative film) on a workpiece (object to be coated), which is a base material, for the purpose of enhancing decorativeness by using an arc ion plating method. Examples of the workpieces that can be used include ornaments such as wristwatches and necklaces, mobile phone casings, and automobile bumpers. Arc ion plating is a type of ion plating method that uses vacuum arc discharge to evaporate solid materials. The arc ion plating method has a high ionization rate of the evaporated material and can form a film with excellent adhesion, so it is suitable for forming decorative films. Examples of the decorative film include TiN, TiAlN, TiCN, CrN, and DLC.

The film forming apparatus 30 includes a vacuum evacuation system 510, a heating/cooling system 520, an evaporation source system 530, a table system 540, a process gas system 550, an etching system 560, and a chamber 570.

The evacuation system 510 includes an evacuation device 511 and evacuates the inside of the chamber 570. The exhaust device 511 includes a pump for exhausting the air in the chamber 570.

The heating and cooling system 520 includes a heater power supply section 521 and a heater 522, and heats the workpiece 545. The heater power supply unit 521 is a power supply circuit that supplies power to the heater 522. The heater 522 is provided in the chamber 570 and generates heat using electric power supplied from the heater power supply section 521. Further, the heating/cooling system 520 cools the workpiece 545 by stopping the heater 522 from generating heat.

The evaporation source system 530 is a system that evaporates a target (film forming material). Evaporation source system 530 includes an arc cathode 531 and an arc power source 532. The arc power supply section 532 is a power supply circuit that supplies discharge current to the arc cathode 531. The arc cathode 531 includes a target, and generates a vacuum arc discharge between it and the inner wall of the chamber 570 by electric power supplied from the arc power supply section 532. When vacuum arc discharge starts, a molten region called an arc spot with a diameter of several μm is generated on the cathode surface. A high-density current is concentrated at the arc spot, and the cathode surface is instantly melted and vaporized. A film is formed on the surface of the workpiece 545 by this vacuum arc discharge.

In the example of FIG. 1, two pairs of arc cathodes 531 and arc power supply units 532 are illustrated, but this is just an example, and there may be one pair of arc cathodes 531 and arc power supply units 532, or three pairs of arc cathodes 531 and arc power supply units 532. There may be more than one pair.

Table system 540 is a rotary table on which workpiece 545 is mounted. The table system 540 includes a table 541, a table driving section 542, and a bias power supply section 543. Table 541 is provided within chamber 570. A workpiece 545 is placed on the table 541. The table drive unit 542 includes a motor and the like, and rotates the table 541. The bias power supply section 543 applies a negative potential to the workpiece 545 via the table 541.

Process gas system 550 introduces process gas into chamber 570 to form a reactive coating.

The etching system 560 includes a discharge power source 561, a pair of filament electrodes 562, and a filament (not shown) provided between the pair of elongated filament electrodes 562. The discharge power supply section 561 is a power supply circuit that supplies discharge current to the filament via a pair of filament electrodes 562. Etching system 560 generates an argon plasma between arc cathode 531 and the filament and between the inner wall of chamber 570 and the filament. The surface of the workpiece 545 is cleaned by the generation of this argon plasma. In this cleaning, the arc cathode 531 and the inner wall of the chamber 570 function as an anode, and the filament functions as a cathode.

Chamber 570 is a container that accommodates workpiece 545. The interior of the chamber 570 is evacuated and maintained in a vacuum state by the evacuation system 510.

FIG. 2 is an overall configuration diagram of the machine learning system in the embodiment. The machine learning system includes a server 10, a communication device 20, and a film deposition device 30. The server 10 and the communication device 20 are connected via a network 40 so that they can communicate with each other. The communication device 20 and the film forming device 30 are connected to each other via a network 50 so that they can communicate with each other. The network 40 is, for example, a wide area communication network such as the Internet. Network 50 is, for example, a local area network. The server 10 is, for example, a cloud server composed of one or more computers. The communication device 20 is, for example, a computer owned by a user who uses the film forming apparatus 30. The communication device 20 functions as a gateway that connects the film forming device 30 to the network 40 . The communication device 20 is realized by installing dedicated application software on a computer owned by the user. Alternatively, the communication device 20 may be a dedicated device provided to the user by the manufacturer of the film forming device 30. The film forming apparatus 30 is the film forming apparatus described in FIG.

The configuration of each device will be specifically explained below. Server 10 includes a processor 100 and a communication unit 101. Processor 100 is a control device including a CPU and the like. The processor 100 includes a reward calculation section 110, an update section 120, a determination section 130, and a learning control section 140. Each block included in the processor 100 may be realized by the processor 100 executing a machine learning program that causes the computer to function as the server 10 in the machine learning system, or may be realized by a dedicated electric circuit.

The remuneration calculation unit 110 calculates a remuneration for the determination result of at least one film forming condition based on the state variables observed by the state observation unit 321.

The updating unit 120 updates a function for determining at least one film forming condition from the state variables observed by the state observation unit 321 based on the reward calculated by the reward calculation unit 110. As the function, an action value function described below is adopted.

The determining unit 130 repeatedly updates the function while changing at least one film-forming condition, thereby determining at least one film-forming condition that provides the most reward.

The learning control unit 140 manages overall control of machine learning. The machine learning system of this embodiment learns film forming conditions by reinforcement learning. Reinforcement learning is an agent (behavior) that selects a certain action based on the situation of the environment, changes the environment based on the selected action, and gives the agent a reward associated with the environmental change, so that the agent can take better actions. This is a machine learning method that learns the selection of As reinforcement learning, Q learning and TD learning can be adopted. In the following explanation, Q learning will be taken as an example. In this embodiment, the reward calculation unit 110, the update unit 120, the determination unit 130, the learning control unit 140, and the state observation unit 321 described below correspond to the agent.

The communication unit 101 is composed of a communication circuit that connects the server 10 to the network 40 . The communication unit 101 receives the state variable observed by the state observation unit 321 via the communication device 20. The communication unit 101 transmits the film forming conditions determined by the determining unit 130 to the film forming apparatus 30 via the communication device 20. In this embodiment, the communication unit 101 is an example of a state acquisition unit that acquires state variables.

Communication device 20 includes a transmitter 201 and a receiver 202. The transmitter 201 transmits the state variables transmitted from the film forming apparatus 30 to the server 10 and also transmits the film forming conditions transmitted from the server 10 to the film forming apparatus 30. The receiver 202 receives the state variables transmitted from the film deposition apparatus 30 and also receives the film deposition conditions transmitted from the server 10 .

The film forming apparatus 30 includes a communication section 310, a processor 320, a memory 330, a sensor section 340, and an input section 350 in addition to the configuration shown in FIG.

The communication unit 310 is a communication circuit for connecting the film forming apparatus 30 to the network 50. The communication unit 310 transmits the state variables observed by the state observation unit 321 to the server 10. The communication unit 310 receives the film forming conditions determined by the determining unit 130 of the server 10. The communication unit 310 receives a film formation execution command, which will be described later, determined by the learning control unit 140.

Processor 320 is a control device including a CPU and the like. The processor 320 includes a state observation section 321, a film formation execution section 322, and an input determination section 323. The communication unit 310 transmits the state variable acquired by the state observation unit 321 to the server 10. Each block included in the processor 320 is realized, for example, by the CPU executing a machine learning program that causes the CPU to function as the film forming apparatus 30 of the machine learning system.

The state observation unit 321 acquires the physical quantity detected by the sensor unit 340 after the film formation is performed. The state observation unit 321 observes a state variable including at least one physical quantity related to performance evaluation of film formation and at least one film formation condition after the film formation is performed. Specifically, the state observation unit 321 acquires the film forming conditions based on the measured value of the sensor unit 340. Further, the state observation unit 321 acquires physical quantities based on the measured values of the sensor unit 340 and the like.

FIG. 4 is a diagram showing an example of film forming conditions. Film forming conditions are broadly classified into medium categories. The middle classification includes a first parameter related to the evacuation system 510, a second parameter related to the heating and cooling system 520, a third parameter related to the evaporation source system 530, a fourth parameter related to the table system 540, and a fourth parameter related to the process gas system 550. 5 parameters are included. Furthermore, the intermediate classification may include a sixth parameter related to the etching system 560.

The first parameter includes at least one of pumping speed, ultimate pressure, residual gas species, residual gas partial pressure, and PQ characteristics. The pumping speed is the speed at which the vacuum pumping system 510 pumps out air, residual gas, and introduced process gas from the chamber 570. The pumping speed can be obtained, for example, by calculation from the performance values of the pumps that constitute the vacuum pumping system 510. Alternatively, the pumping speed may be a measured value calculated from a pressure sensor and pumping time. The ultimate pressure is the pressure within the chamber 570 before starting the film deposition process. The ultimate pressure is obtained, for example, by calculation from the performance values of the pumps that constitute the vacuum evacuation system 510. Alternatively, the ultimate pressure may be a measured value of a pressure sensor. The residual gas species is gas remaining in the chamber 570 and is an impurity. The residual gas species include, for example, nitrogen, oxygen, moisture, and hydrogen. The residual gas type is determined based on the partial pressure of residual gas, which will be described later. The residual gas partial pressure is the partial pressure of a plurality of residual gases remaining within the chamber 570. The residual gas partial pressure is obtained by measurement using a vacuum residual gas monitor such as a quadrupole mass spectrometer. The PQ characteristic is a characteristic indicating the relationship between the chamber internal pressure (P) and the flow rate (Q). The PQ characteristic is obtained by calculation from, for example, the flow rate of gas in the chamber 570 detected by a flow rate sensor and the measured value of a pressure sensor.

The second parameters include heater temperature, workpiece temperature, heater temperature increase rate, workpiece temperature increase rate, heater output, heater temperature accuracy, workpiece temperature accuracy, heater temperature/workpiece temperature, heater temperature distribution, workpiece temperature distribution, cooling gas type, It includes at least one of cooling gas pressure and workpiece cooling rate.

The heater temperature is the temperature of the heater 522. The heater temperature is, for example, a value measured by a temperature sensor (thermocouple). The workpiece temperature is the temperature of the workpiece 545. The workpiece temperature is, for example, a value measured by a temperature sensor provided near the workpiece 545. The heater temperature increase rate is the rate of change in heater temperature when the heater 522 is heated up. The heater temperature increase rate is obtained from the time-series change in heater temperature. The workpiece temperature increase is the rate of change in workpiece temperature when the workpiece 545 is heated up. The workpiece temperature increase rate is obtained from the time-series changes in the workpiece temperature.

The heater output is the output of the heater 522. The heater output is obtained by calculation from the set value of the heater power supply section 521. The heater output may be calculated from sensor-measured values of current and voltage supplied to the heater.

Heater temperature accuracy is a value indicating the variation in heater temperature. Heater temperature accuracy is calculated from past heater temperature measurements. The workpiece temperature accuracy is a value indicating the variation in the workpiece temperature. Workpiece temperature accuracy is calculated from past workpiece temperature measurements. Heater temperature/workpiece temperature is a response characteristic of the heater 522 to the workpiece 545.

The heater temperature distribution is the temperature distribution of the heater 522. The heater temperature distribution is obtained from the measured values of a plurality of temperature sensors provided around the heater 522. The workpiece temperature distribution is the temperature distribution of the workpiece 545. The workpiece temperature distribution is obtained from the measured values of a plurality of temperature sensors provided around the workpiece 545.

The cooling gas type is information indicating the type of gas that cools the inside of the chamber 570, and is an input value input in advance. Cooling gas pressure is the pressure of cooling gas. The cooling gas pressure is a value measured by a pressure sensor provided within the chamber 570. The workpiece cooling rate is the cooling rate of the workpiece 545. The workpiece cooling rate is obtained from a time-series change in the workpiece temperature detected by a temperature sensor provided near the workpiece 545.

The third parameter includes at least one of target composition, target thickness, target manufacturing method, arc discharge voltage, arc discharge current, evaporation source magnetic field, evaporation source coil current, and arc ignition characteristics. The target composition is the composition of the substances that make up the target. Target thickness is the thickness of the target. The target manufacturing method is a method for manufacturing a target. The target composition, target thickness, and target manufacturing method are input values input in advance.

The arc discharge voltage is a voltage that the arc power supply unit 532 supplies to the arc cathode 531, and is a value measured by a sensor. The arc discharge current is a current that the arc power supply section 532 supplies to the arc cathode 531, and is a value measured by a sensor.

The source magnetic field is the location and strength of the magnetic field emitted by the permanent magnetic flux contained in the source system 530. The evaporation source magnetic field is an input value input in advance. The evaporation source coil current is a current flowing through a coil included in the evaporation source system 530, and is a value measured by a sensor. Arc ignition characteristics are the behavior of voltage and current on the arc surface during arc ignition. Arc ignition characteristics are obtained from measured values of arc discharge voltage and arc discharge current at certain timings.

The fourth parameter includes at least one of bias voltage, bias current, OL number, OL time change, bias voltage waveform, bias current waveform, workpiece rotation speed, workpiece shape, workpiece loading amount, workpiece loading method, and workpiece material. .

The bias voltage is a bias voltage supplied to the workpiece 545 by the bias power supply unit 543, and is a value measured by a sensor. The bias current is a bias current supplied to the workpiece 545 by the bias power supply unit 543, and is a value measured by a sensor.

The OL (OverLoad) number is the number of abnormal discharges in the table system or the workpiece, and is a value measured by a sensor. The OL time change is the number of OLs per unit time. The bias voltage waveform is a waveform of a bias voltage, and is obtained from a measurement value by a sensor. The bias voltage waveform is particularly a voltage waveform during pulse bias. The bias current waveform is a waveform of a bias current, and is obtained from a value measured by a sensor. The work rotation speed is the rotation speed of the workpiece 545 per unit time, and includes the rotation speed of the table 541 per unit time and the rotation speed per unit time when the workpiece 545 rotates on the table 541. The work rotation speed is, for example, a value detected by a sensor. The workpiece shape is a numerical value indicating the shape of the workpiece 545, and is an input value input in advance. The workpiece loading amount is the loading amount (for example, weight) of the workpiece 545, and is an input value input in advance. The workpiece mounting method is a method of mounting the workpiece 545 on the table 541, and is an input value input in advance. The workpiece material is the material of the workpiece 545 and is an input value input in advance.

The fifth parameter includes at least one of gas flow rate, gas type, and gas pressure. The gas flow rate is the flow rate of the process gas. The gas type is information indicating the type of process gas. Gas pressure is the pressure of the process gas. These are, for example, sensor detection values.

The sixth parameter includes at least one of a filament heating current, a filament heating voltage, a filament diameter, a discharge current, and a discharge voltage. The filament heating current is a heating current for heating a pair of filament electrodes 562 that constitute the etching system 560, and is a value measured by a sensor. The filament heating voltage is a heating voltage for heating the pair of filament electrodes 562, and is a value measured by a sensor.

The filament diameter is the diameter of each of the pair of filament electrodes 562, and is an input value input in advance. Note that the filament diameter may be calculated by calculation. The discharge current is a discharge current of the pair of filament electrodes 562, and is a value measured by a sensor. The discharge voltage is the discharge voltage of the pair of filament electrodes 562, and is a value measured by a sensor.

FIG. 5 is a diagram showing an example of physical quantities. Physical quantities are broadly classified into medium categories. The middle classification includes at least one of membrane properties and physical properties. The film quality characteristics include at least one of film thickness, roughness, surface texture, composition, crystal structure, film microstructure, crystallinity, crystal grain size, residual stress, density, particle amount, and particle size.

The film thickness is the thickness of the film. Surface texture is the morphology of the surface including surface roughness. The composition is that of the film. Crystal structure is the crystal structure of the film. Membrane microstructure is a general meaning and refers to microstructural structure such as crystal morphology and orientation. Crystallinity is the percentage of crystals. Grain size is the size of crystal grains. Residual stress is internal stress in the film.

The film thickness is obtained using a film thickness measuring device. Roughness is obtained using a roughness meter. The surface texture is obtained using a microscope or a roughness meter. The composition is obtained by X-ray spectroscopy. The crystal structure, film microstructure, crystallinity, grain size, and residual stress are obtained by X-ray diffraction or electron microscopy.

Density is the density of the particles that make up the film. The amount of particles (defect amount) is the amount of dust contained in the film. Particle size (defect size) is the size of dust contained in the film. Density is obtained by X-ray reflection method. The particle amount and particle size can be obtained using a microscope or image processing.

The physical properties include at least one of adhesion and optical properties. Adhesion indicates the degree of adhesion of the film to the base material, and can be obtained by an indentation method or a scratch test. Optical properties indicate the color, gloss, or texture of the film. Optical properties are measured using a spectrophotometer.

Referring back to FIG. The film-forming execution unit 322 controls the film-forming operation of the film-forming apparatus 30. The input determination unit 323 automatically or manually determines whether the process is a mass production process. When automatically determining whether or not it is a mass production process, the input determination unit 323 determines that the film forming apparatus 30 is in a mass production process if the number of inputs of the condition number input to the input unit 350 exceeds the reference number of times. judge. The condition number is an identification number for specifying one film forming condition. The film forming conditions specified by the condition number include at least the film forming conditions described as "Input" among the film forming conditions shown in FIG.

When manually determining whether or not the process is a mass production process, the input determination unit 323 determines that the film forming apparatus 30 is in the mass production process if data indicating that the process is a mass production process is input to the input unit 350. . When in the mass production process, the film forming apparatus 30 does not perform machine learning.

The memory 330 is, for example, a nonvolatile storage device, and stores the finally determined optimal film forming conditions. The sensor unit 340 is various sensors used to measure the film forming conditions illustrated in FIG. 4 and the physical quantities illustrated in FIG. 5. The input unit 350 is an input device such as a keyboard and a mouse.

FIG. 3 is a flowchart showing an example of processing in the machine learning system shown in FIG. In step S<b>1 , the learning control unit 140 obtains the input value of the film forming conditions input by the user using the input unit 350 . The input value acquired here is the input value for the film-forming condition described as Input among the film-forming conditions listed in FIG.

In step S2, the learning control unit 140 determines at least one film forming condition and a set value for the film forming condition. Here, the film-forming conditions to be set are the film-forming conditions other than the film-forming conditions listed as Input among the film-forming conditions listed in FIG. These are the film forming conditions. Here, the set values of the film forming conditions that are determined correspond to actions in reinforcement learning.

Specifically, the learning control unit 140 randomly selects a set value for each of the film forming conditions to be set. Here, the set value is randomly selected from within a predetermined range for each film forming condition. As a method for selecting the set values of the film forming conditions, for example, the ε-greedy method can be adopted.

In step S3, the learning control unit 140 causes the film forming apparatus 30 to start a film forming operation by transmitting a film forming execution command to the film forming apparatus 30. When the film-forming execution command is received by the communication unit 310, the film-forming execution unit 322 sets film-forming conditions according to the film-forming execution command and starts the film-forming operation. The film-forming execution command includes the input value of the film-forming condition set in step S1, the set value of the film-forming condition determined in step S2, and the like.

When the film-forming operation ends, the state observation unit 321 observes the state variables (step S4). Specifically, the state observation unit 321 converts the physical quantities related to film formation evaluation shown in FIG. 5 and the film formation conditions whose state is observed by a sensor etc. out of the film formation conditions shown in FIG. Get as. The physical quantity may be input into the film forming apparatus 30 by, for example, a user operating the input unit 350, or may be input into the film forming apparatus 30 by communication between a measuring instrument that measures the physical quantity and the film forming apparatus 30. Good too. The state observation unit 321 transmits the acquired state variables to the server 10 via the communication unit 310.

In step S5, the determining unit 130 evaluates the physical quantity. Here, the determining unit 130 determines the physical quantity by determining whether the physical quantity to be evaluated (hereinafter referred to as the target physical quantity) among the physical quantities acquired in step S4 has reached a predetermined reference value. evaluate. The target physical quantity is one or more of the physical quantities listed in FIG. When there are multiple target physical quantities, there are multiple reference values corresponding to each target physical quantity. As the reference value, for example, a predetermined value indicating that the film has reached a certain standard can be adopted.

The reference value may be a value including, for example, an upper limit value and a lower limit value. In this case, when the target physical quantity falls within the range between the upper limit value and the lower limit value, it is determined that the reference value has been reached. The reference value may be one value. In this case, if the target physical quantity exceeds a reference value or falls below a reference value, it is determined that a certain criterion is met.

When determining that the target physical quantity has reached the reference value (YES in step S6), the determining unit 130 outputs the film forming conditions set in step S2 as the final film forming conditions (step S7). On the other hand, when determining that the physical quantity has not reached the reference value (NO in step S6), the determining unit 130 advances the process to step S8. Note that in the case where there are a plurality of target physical quantities, the determining unit 130 may determine YES in step S6 when all the target physical quantities have reached the reference value.

In step S8, the reward calculation unit 110 determines whether the target physical quantity is close to the reference value. When the target physical quantity approaches the reference value (YES in step S8), the reward calculation unit 110 increases the reward for the agent (step S9). On the other hand, if the target physical quantity is not close to the reference value (NO in step S8), the reward calculation unit 110 reduces the reward to the agent (step S10). In this case, the remuneration calculation unit 110 may increase or decrease the remuneration according to a predetermined increase or decrease value of the remuneration. Note that when there are a plurality of target physical quantities, the reward calculation unit 110 may perform the determination in step S8 for each of the plurality of target physical quantities. In this case, the reward calculation unit 110 may increase or decrease the reward for each of the plurality of target physical quantities based on the determination result in step S8. Further, different values may be adopted as the increase/decrease value of the reward depending on the target physical quantity.

In step S11, the updating unit 120 updates the action value function using the reward given to the agent. Q learning adopted in this embodiment is a method of learning a Q value (Q(s, a)) that is the value of selecting action a under a certain environmental state s. Note that the environmental state s _t corresponds to the state variable of the above flow. In Q learning, when a certain environmental state s occurs, the action a with the highest Q(s, a) is selected. In Q learning, various actions a are taken under a certain environmental state s through trial and error, and the correct Q(s, a) is learned using the reward at that time. The update formula for the action value function Q(s _t , a _t ) is shown by the following equation (1).

Here, s _t and a _t represent the environmental state and behavior at time t, respectively. Due to the action a _t , the environmental state changes to s _t+1 , and the reward r _t+1 is calculated based on the change in the environmental state. In addition, the term with max is the Q value (Q(s _t+1 , a)) multiplied by γ when the most valuable action a known at that time is selected under the environmental state s _t+1. It is. Here, γ is a discount rate and takes a value of 0<γ≦1 (usually 0.9 to 0.99). α is a learning coefficient and takes a value of 0<α≦1 (usually about 0.1).

This update formula is based on the Q value when action a takes the best action in the next environmental state s _t+1 , rather than Q(s _t , a _t ), which is the Q value of action a in state s. If maxQ(s _t+1 , a) is larger, Q(s _t , a _t ) is increased. On the other hand, this update formula reduces Q(s _t , at ₎ if γ·maxQ(s _t+1 , a) is smaller than Q(s _t , at ₎ . In other words, the value of a certain action a in a certain state s _t is brought closer to the value of the best action in the next state s _t+1 . This determines the optimum state for forming the workpiece 545, that is, at least one optimum film forming condition.

When the process in step S11 is completed, the process returns to step S2, and the setting values of the selected film forming conditions are changed, the unselected film forming conditions are selected as the next film forming conditions, and the same process is performed. The action value function is updated. Although the update unit 120 updates the action value function, the present invention is not limited thereto, and may update the action value table.

For Q(s, a), values for all state-action pairs (s, a) may be stored in a table format. Alternatively, Q(s, a) may be expressed by an approximation function that approximates the values for all state and action pairs (s, a). This approximation function may be constructed by a multilayer neural network. In this case, the neural network may perform online learning in which data obtained by actually operating the film forming apparatus 30 is learned in real time and reflected in the next action. This realizes deep reinforcement learning.

Conventionally, in film forming apparatuses, film forming conditions have been developed by changing the film forming conditions so as to obtain a good decorative film. In order to obtain a good decorative film, it is necessary to find a relationship between the evaluation of the decorative film and the film forming conditions. However, as shown in Figure 4, there are a huge number of types of film formation conditions, so defining such relationships requires an extremely large number of physical models. The findings showed that it is difficult to Furthermore, in order to construct such a physical model, it is also required to artificially find out which parameters influence the evaluation of which decorative coating, which is difficult to construct.

According to the present embodiment, at least one parameter among the first to sixth parameters described above and at least one physical quantity among film quality characteristics and physical characteristics related to performance evaluation of film formation are state variables. It is observed as Then, based on the observed state variables, a reward for the determination result of the film forming conditions is calculated, and based on the calculated reward, the action value function for determining the film forming conditions from the state variables is updated. Updates are repeated to learn the deposition conditions that yield the most rewards. In this way, in this embodiment, the film forming conditions are determined by machine learning without using the above-mentioned physical model. As a result, in this embodiment, appropriate film forming conditions for the decorative film can be easily determined.

Note that the following modifications can be adopted in the present invention.

(1) FIG. 6 is an overall configuration diagram of a machine learning system according to a modification of the present invention. The machine learning system according to this modification is configured by a single film forming apparatus 30A. The film forming apparatus 30A includes a processor 320A, an input section 391, and a sensor section 392. Processor 320A includes a machine learning section 370 and a film forming section 380. The machine learning section 370 includes a reward calculation section 371, an updating section 372, a determining section 373, and a learning control section 374. The remuneration calculation unit 371 to learning control unit 374 are the same as the remuneration calculation unit 110 to learning control unit 140 shown in FIG. 2, respectively. The state observation section 381, the film formation execution section 382, and the input determination section 383 are the same as the state observation section 321, the film formation execution section 322, and the input determination section 323 shown in FIG. 2, respectively. The input section 391 and the sensor section 392 are the same as the input section 350 and the sensor section 340 shown in FIG. 2, respectively. In this modification, the state observation unit 381 is an example of a state acquisition unit that acquires state information.

In this way, according to the machine learning system according to this modification, the optimal film forming conditions can be learned by the film forming apparatus 30A alone.

(2) In the above flow, the state variable is observed after the end of the film-forming operation, but this is just an example, and a plurality of state variables may be observed during one film-forming operation. For example, if the state variables consist only of parameters that can be measured instantaneously, a plurality of state variables can be observed during one film deposition operation. Thereby, learning time can be shortened.

(3) In the above embodiment, the film forming apparatus 30 is an apparatus that forms films by arc ion plating, but the present invention is not limited to this, and forms films by other physical vapor deposition methods such as vapor deposition. It may also be a device that coats the film.

10: Server 30: Film forming apparatus 100: Processor 110: Reward calculation unit 120: Updating unit 130: Determining unit 140: Learning control unit 510: Vacuum exhaust system 520: Heating cooling system 530: Evaporation source system 540: Table system 550: Process gas system 560: Etching system 570: Chamber 574: Learning control section

Claims (13)

A machine learning method in which a machine learning device determines film forming conditions of a film forming apparatus that forms a decorative film on a base material, the machine learning method comprising:
The film forming apparatus includes an evacuation system for evacuating the chamber, a heating and cooling system for heating and cooling the chamber, an evaporation source system for evaporating a target, a table system for placing a workpiece, and the chamber. An AIP apparatus including a process gas system for introducing a process gas into the process gas and an etching system,
Obtaining a state variable including at least one physical quantity related to performance evaluation of the decorative film and at least one film forming condition;
calculating a reward for the determination result of the at least one film forming condition based on the state variable;
updating a function for determining the at least one film formation condition from the state variable based on the reward;
By repeating the update of the function, determine the film forming conditions that can obtain the maximum amount of the reward,
The at least one film forming condition includes a first parameter related to the vacuum evacuation system, a second parameter related to the heating and cooling system, a third parameter related to the evaporation source system, a fourth parameter related to the table system, and the process. at least one of a fifth parameter related to the gas system;
The at least one physical quantity is at least one of a film property and a physical property,
The fifth parameter is at least one of the flow rate of the process gas, the type of the process gas, and the pressure of the process gas,
The film quality characteristic is at least one of film thickness, roughness, surface texture, composition, crystal structure, film microstructure, crystallinity, crystal grain size, residual stress, density, defect amount, and defect size in the decorative film. including;
The physical properties include adhesion and optical properties of the decorative coating,
Machine learning methods. The first parameter is at least one of pumping speed, ultimate pressure, residual gas species, residual gas partial pressure, and PQ characteristics.
The machine learning method according to claim 1. The second parameters include a heater temperature of a heater constituting the heating/cooling system, a workpiece temperature that is the temperature of the workpiece, a heating rate of the heater, a heating rate of the workpiece, an output of the heater, and a temperature of the heater. accuracy, temperature accuracy of the workpiece, response characteristics of the heater temperature and the workpiece temperature, temperature distribution of the heater, and temperature distribution of the workpiece,
The machine learning method according to claim 1 or 2. The third parameter is at least one of the composition of the target, the thickness of the target, the manufacturing method of the target, the arc discharge voltage, the arc discharge current, the evaporation source magnetic field, the evaporation source coil current, and the arc ignition characteristics.
The machine learning method according to any one of claims 1 to 3. The fourth parameter includes a bias voltage for the workpiece, a bias current for the workpiece, the number of times of abnormal discharge, a time change of the abnormal discharge, a waveform of the bias voltage, a waveform of the bias current, the rotation speed of the workpiece, and the number of rotations of the workpiece. including at least one of the shape, the amount of the workpiece to be loaded, the method of mounting the workpiece, and the material of the workpiece;
The machine learning method according to any one of claims 1 to 4. The at least one film forming condition further includes a sixth parameter regarding the etching system.
The machine learning method according to any one of claims 1 to 5 . The sixth parameter is at least one of a heating current for heating a filament of the etching system, a heating voltage for heating the filament, a diameter of the filament, a discharge current of the filament, and a discharge voltage of the filament. is,
The machine learning method according to claim 6 . the function is updated in real time using deep reinforcement learning;
The machine learning method according to any one of claims 1 to 7 . In calculating the reward, if the at least one physical quantity approaches a predetermined reference value corresponding to each physical quantity, the reward is increased, and the at least one physical quantity does not approach the reference value corresponding to each physical quantity. reduce said remuneration if
The machine learning method according to any one of claims 1 to 8 . A machine learning device that determines film forming conditions for a film forming apparatus that forms a decorative film on a base material,
The film forming apparatus includes an evacuation system for evacuating the chamber, a heating and cooling system for heating and cooling the chamber, an evaporation source system for evaporating a target, a table system for placing a workpiece, and the chamber. An AIP apparatus including a process gas system for introducing a process gas into the process gas and an etching system,
a state acquisition unit that observes a state variable including at least one physical quantity related to performance evaluation of the decorative film and at least one film forming condition;
a remuneration calculation unit that calculates remuneration for the determination result of the at least one film forming condition based on the state variable;
an updating unit that updates a function for determining the at least one film forming condition based on the state variable based on the reward;
a determining unit that determines a film forming condition that provides the largest amount of the reward by repeatedly updating the function;
The at least one film forming condition includes a first parameter related to the vacuum evacuation system, a second parameter related to the heating and cooling system, a third parameter related to the evaporation source system, a fourth parameter related to the table system, and the process. at least one of a fifth parameter related to the gas system;
The at least one physical quantity is at least one of a film property and a physical property,
The fifth parameter is at least one of the flow rate of the process gas, the type of the process gas, and the pressure of the process gas,
The film quality characteristic is at least one of film thickness, roughness, surface texture, composition, crystal structure, film microstructure, crystallinity, crystal grain size, residual stress, density, defect amount, and defect size in the decorative film. including;
The physical properties include adhesion and optical properties of the decorative coating,
Machine learning device. A computer-readable machine learning program that causes a computer to function as a machine learning device that determines film forming conditions for a film forming apparatus that forms a decorative film on a base material,
The film forming apparatus includes an evacuation system for evacuating the chamber, a heating and cooling system for heating and cooling the chamber, an evaporation source system for evaporating a target, a table system for placing a workpiece, and the chamber. An AIP apparatus including a process gas system for introducing a process gas into the process gas and an etching system,
a state acquisition unit that observes a state variable including at least one physical quantity related to performance evaluation of the decorative film and at least one film forming condition;
a remuneration calculation unit that calculates remuneration for the determination result of the at least one film forming condition based on the state variable;
an updating unit that updates a function for determining the at least one film forming condition based on the state variable based on the reward;
a determining unit that determines a film forming condition that provides the largest amount of the reward by repeatedly updating the function;
The at least one film forming condition includes a first parameter related to the vacuum evacuation system, a second parameter related to the heating and cooling system, a third parameter related to the evaporation source system, a fourth parameter related to the table system, and the process. at least one of a fifth parameter related to the gas system;
The at least one physical quantity is at least one of a film property and a physical property,
The fifth parameter is at least one of the flow rate of the process gas, the type of the process gas, and the pressure of the process gas,
The film quality characteristic is at least one of film thickness, roughness, surface texture, composition, crystal structure, film microstructure, crystallinity, crystal grain size, residual stress, density, defect amount, and defect size in the decorative film. including;
The physical properties include adhesion and optical properties of the decorative coating,
Machine learning program. A communication method for a film forming apparatus when machine learning the film forming conditions of the film forming apparatus that forms a decorative film on a base material, the method comprising:
The film forming apparatus includes an evacuation system for evacuating the chamber, a heating and cooling system for heating and cooling the chamber, an evaporation source system for evaporating a target, a table system for placing a workpiece, and the chamber. An AIP device including a process gas system for introducing process gas into the AIP system, an etching system, and a communication section,
Observing a state variable including at least one physical quantity and at least one film forming condition related to performance evaluation of the decorative film,
transmitting the state variable onto a network and receiving at least one machine-learned deposition condition;
The at least one film forming condition includes a first parameter related to the vacuum evacuation system, a second parameter related to the heating and cooling system, a third parameter related to the evaporation source system, a fourth parameter related to the table system, and the process. at least one of a fifth parameter related to the gas system;
The at least one physical quantity is at least one of a film property and a physical property,
The fifth parameter is at least one of the flow rate of the process gas, the type of the process gas, and the pressure of the process gas,
The film quality characteristic is at least one of film thickness, roughness, surface texture, composition, crystal structure, film microstructure, crystallinity, crystal grain size, residual stress, density, defect amount, and defect size in the decorative film. including;
The physical properties include adhesion and optical properties of the decorative coating,
Communication method. A film forming apparatus comprising an AIP apparatus and forming a decorative film on a base material,
A vacuum evacuation system to evacuate the chamber,
a heating and cooling system that heats and cools the chamber;
an evaporation source system that evaporates a target;
A table system on which the work is placed,
a process gas system that introduces a process gas into the chamber;
etching system,
a state observation unit that observes a state variable including at least one physical quantity related to performance evaluation of the decorative film and at least one film forming condition;
a communication unit that transmits the state variable onto the network and receives the machine-learned film forming conditions;
The at least one film forming condition includes a first parameter related to the vacuum evacuation system, a second parameter related to the heating and cooling system, a third parameter related to the evaporation source system, a fourth parameter related to the table system, and the process. at least one of a fifth parameter related to the gas system;
The at least one physical quantity is at least one of a film property and a physical property,
The fifth parameter is at least one of the flow rate of the process gas, the type of the process gas, and the pressure of the process gas,
The film quality characteristic is at least one of film thickness, roughness, surface texture, composition, crystal structure, film microstructure, crystallinity, crystal grain size, residual stress, density, defect amount, and defect size in the decorative film. including;
The physical properties include adhesion and optical properties of the decorative coating,
Film deposition equipment.

Images