JP2021147112A - Method of feed processing - Google Patents

Abstract

To provide a technology allowing for reducing the time required for machine learning and preventing damages to a real installation.SOLUTION: A method of feed processing includes ejecting a processing substance to a surface while feeding a base material. The method has a) a step of acquiring plural pieces of information related to a control value related to feeding including a rotational speed of a motor to feed a base material or to a control value related to ejection including ejection timing of a processing substance and a measured value related to a tension or feed rate of the base material, while actually feeding the base material, b) a step of preparing a simplified simulation model indicative of a relationship between the control value and the measured value on a computer, c) a step of preparing an in-learning model by conducting reinforcement learning to maintain an output value from the simplified simulation model in a predetermined range while updating the control value on the computer, and d) a step of conducting reinforcement learning to maintain the measured value in the predetermined range in the case of ejecting a processing substance to a surface while actually feeding the base material with updating the control value outputted from the in-learning model.SELECTED DRAWING: Figure 4

Description

The present invention relates to a transport processing method in which a processing substance is discharged onto the surface of a base material while transporting a long strip-shaped base material in the longitudinal direction.

Conventionally, there is known an inkjet type image recording device that records an image on a printing paper by ejecting ink from a plurality of recording heads while transporting a long strip-shaped printing paper in the longitudinal direction. The image recording device ejects inks of different colors from a plurality of heads. Then, a multicolor image is recorded on the surface of the printing paper by superimposing the monochromatic images formed by the inks of each color.

In this type of image recording apparatus, it is required that printing paper is conveyed at a constant transfer speed by a plurality of rollers. However, the transfer speed of the printing paper below the recording head may deviate from the ideal transfer speed due to the slip between the surface of the roller and the printing paper or the stretching of the printing paper due to the ink. Then, the ejection position of the ink of each color on the surface of the printing paper may shift in the transport direction. Therefore, for example, Patent Document 1 describes a method for compensating for such a deviation.

Japanese Unexamined Patent Publication No. 2019-166832

Patent Document 1 discloses a method of recording an image (9) by dropping ink droplets on a printing substrate while transporting the printing substrate by rotating a jetting drum. Then, the compensation torque is calculated based on the measurement result of the drive torque of the jetting drum and the grayscale value transition (10) of the image data recorded on the printing substrate, and the jetting drum is set in consideration of the compensation torque. The steps of drive control are disclosed. In addition, it is described that machine learning is performed when calculating the compensation torque.

However, when machine learning is performed from scratch using an actual device, it may take an enormous amount of time to reach a desired level. In addition, in the initial stage of learning, there is a risk of damaging the actual device by performing control that is far from the ideal operation.

The present invention has been made in view of such circumstances, and an object of the present invention is to provide a technique capable of shortening the time required for machine learning. It is also an object of the present invention to provide a technique capable of preventing damage to an actual device even in the initial stage of learning.

In order to solve the above problems, the first invention of the present application is a transport processing method in which a processing substance is discharged onto the surface of the base material while transporting the long strip-shaped base material in the longitudinal direction. While transporting the base material, the control value related to the transport including the rotation speed of the motor which is the drive source for transporting the base material, or the information related to the control value related to the discharge including the discharge timing of the processing substance. , A simple simulation model showing the relationship between the control value and the measured value based on the step of acquiring a plurality of measured values related to the tension or the transport speed of the base material and b) the acquisition result in the step a). On a computer, and c) Reinforcement learning for maintaining the output value from the simple simulation model within a predetermined range while updating the control value on the computer is being learned. The step of creating a model and d) the case where the processing substance is discharged to the surface of the base material while actually transporting the base material while updating the control value output from the learning model. It has a step of performing reinforcement learning for maintaining the measured value within a predetermined range.

The second invention of the present invention is the transport processing method of the first invention, in which the base material is hung on a plurality of rollers, and a drive roller which is at least one of the plurality of rollers is driven by the motor. It is conveyed by rotating, and the control value related to the transfer includes the rotation speed of the drive roller.

The third invention of the present application is the transport processing method of the first invention or the second invention, and the control value related to the discharge includes the discharge amount of the processed substance.

The fourth invention of the present application is the transport processing method of any one of the first to third inventions, and in the step a), information on the thickness or type of the base material is further acquired, and the above-mentioned In the step b), the simple simulation model showing the relationship between the control value, the information on the base material, and the measured value is created based on the acquisition result in the step a).

The fifth invention of the present application is the transport processing method of any one of the first to fourth inventions, and in the step a), information relating to the characteristics of the treated substance is further acquired, and the step b. ), The simple simulation model showing the relationship between the control value, the information of the processing substance, and the measured value is created based on the acquisition result in the step a).

The sixth invention of the present application is the transport processing method of any one of the first to fifth inventions, and in the step a), information relating to environmental conditions including the temperature or humidity around the base material. In the step b), the simple simulation model showing the relationship between the control value, the information related to the environmental condition, and the measured value is created based on the acquisition result in the step a).

The seventh invention of the present application is the transport processing method of any one invention from the first invention to the sixth invention, and in the step b), the simple simulation model includes a decision tree and is included in the decision tree. Parameters are adjusted.

The eighth invention of the present application is the transport processing method of any one invention from the first invention to the seventh invention, and the reinforcement learning performed in the step c) is a machine executed by the technique of PPO or DQN. Learning.

According to the first to eighth inventions of the present application, a simple simulation model is created on a computer using the result of collecting data in an actual device in advance, and reinforcement learning is performed on the computer using the simple simulation model. .. Then, the learning model obtained by performing reinforcement learning on the computer is transferred to the actual device again, and reinforcement learning is continuously performed. As a result, reinforcement learning can be started from a state advanced to some extent commensurate with the operation of the actual device. In addition, a considerable amount of reinforcement learning can be performed on a computer. As a result, the time required for machine learning can be significantly reduced. In addition, even if the control value becomes excessive in the initial stage of learning, it is possible to prevent damage to the actual device. Furthermore, by finally performing reinforcement learning using the actual device again, it is possible to create a trained model that outputs more accurate control values.

It is a figure which showed the structure of the image recording apparatus. It is a partial top view of the image recording apparatus in the vicinity of an image recording unit. It is a figure which showed typically the structure of the edge position detection part. It is a flowchart which shows the flow of the pre-learning. It is a figure which showed the example of the data aggregate conceptually. It is a figure which conceptually showed the example of the decision tree included in the simple simulation model. It is a block diagram conceptually showing the state of performing reinforcement learning on a computer. It is a flowchart which shows the detailed flow of reinforcement learning on a computer. It is a block diagram conceptually showing the state of performing reinforcement learning with an actual device. It is a flowchart which shows the detailed flow of reinforcement learning in an actual device. It is a graph which showed the example of the 1st edge signal and the example of the 2nd edge signal.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In one embodiment of the present invention, as an example of the transfer processing device, an image recording device that records an image on the transferred printing paper will be described as an example. Then, a method for maintaining the measured value of the transport speed or tension of the printing paper within a predetermined range will be described.

<1. First Embodiment>
<1-1. Image recording device configuration>
First, the overall configuration of the image recording apparatus 1 which is an example of the transport processing apparatus of the present invention will be described with reference to FIG. FIG. 1 is a diagram showing a configuration of an image recording device 1. The image recording device 1 transfers an image on the printing paper 9 by ejecting ink from a plurality of recording heads 21 to 24 toward the printing paper 9 while conveying the printing paper 9 which is a long strip-shaped base material. This is an inkjet printing device for recording. As shown in FIG. 1, the image recording device 1 includes a transport mechanism 10, an image recording unit 20, two edge position detection units 30, an encoder 40, a tension detection unit 50, an information acquisition unit 60, a control unit 80, and a simulation unit. It is equipped with a computer 90.

The transport mechanism 10 is a mechanism for transporting the printing paper 9 in the transport direction along the longitudinal direction thereof. The transport mechanism 10 of the present embodiment includes a plurality of rollers including a winding roller 11, a plurality of transport rollers 12, and a winding roller 13, and one or a plurality of (three in this embodiment) motors 14. .. The printing paper 9 is hung on the plurality of rollers. Further, in the present embodiment, the motor 14 is connected to the unwinding roller 11, one of the plurality of conveying rollers 12 (the conveying roller 121 in FIG. 1), and the take-up roller 13, respectively. The motor 14 is a drive source for conveying the printing paper 9. The unwinding roller 11, the transport roller 121, and the take-up roller 13 are driven by a motor 14 and rotate about a rotation axis, respectively. Each transfer roller 12 guides the printing paper 9 to the downstream side of the transfer path by rotating around the rotation axis. As a result, the printing paper 9 is unwound from the unwinding roller 11 and is conveyed along the conveying path composed of the plurality of conveying rollers 12. Further, the printed paper 9 after being conveyed is collected by the take-up roller 13.

That is, in the present embodiment, the unwinding roller 11, the transport roller 121, and the take-up roller 13 of the plurality of rollers are drive rollers connected to the motor 14. However, the roller to which the motor 14 is connected is not limited to this. The printing paper 9 may be conveyed by a drive roller, which is at least one of a plurality of rollers, driven by a motor 14 to rotate.

As shown in FIG. 1, the printing paper 9 moves below the plurality of recording heads 21 to 24, which will be described later, substantially parallel to the arrangement direction of the plurality of recording heads 21 to 24. At this time, the surface (recording surface) of the printing paper 9 is directed upward (recording heads 21 to 24 side). Further, the printing paper 9 is hung on a plurality of transport rollers 12 in a state where tension is applied. As a result, slack and wrinkles of the printing paper 9 during transportation are suppressed.

The image recording unit 20 is a processing unit that ejects ink droplets (hereinafter referred to as “ink droplets”) onto the printing paper 9 conveyed by the conveying mechanism 10. The image recording unit 20 of the present embodiment includes a first recording head 21, a second recording head 22, a third recording head 23, and a fourth recording head 24. The first recording head 21, the second recording head 22, the third recording head 23, and the fourth recording head 24 are arranged along the transport path of the printing paper 9.

FIG. 2 is a partial top view of the image recording device 1 in the vicinity of the image recording unit 20. Each of the four recording heads 21 to 24 covers the entire width direction of the printing paper 9. Further, as shown by a broken line in FIG. 2, a plurality of nozzles 250 arranged in parallel with the width direction of the printing paper 9 are provided on the lower surface of each of the recording heads 21 to 24. Each of the recording heads 21 to 24 is directed from the plurality of nozzles 250 toward the upper surface of the printing paper 9, and is composed of K (black), C (cyan), M (magenta), and Y (yellow), which are color components of the multicolor image. Ink droplets of each color are ejected.

That is, the first recording head 21 ejects K-color ink droplets onto the upper surface of the printing paper 9 at the first processing position P1 on the transport path. The second recording head 22 ejects C-color ink droplets onto the upper surface of the printing paper 9 at the second processing position P2 on the downstream side of the first processing position P1. The third recording head 23 ejects M-color ink droplets onto the upper surface of the printing paper 9 at the third processing position P3 on the downstream side of the second processing position P2. The fourth recording head 24 ejects Y-color ink droplets onto the upper surface of the printing paper 9 at the fourth processing position P4 on the downstream side of the third processing position P3. In the present embodiment, the first processing position P1, the second processing position P2, the third processing position P3, and the fourth processing position P4 are arranged at equal intervals along the transport direction of the printing paper 9.

The four recording heads 21 to 24 record a single color image on the upper surface of the printing paper 9 by ejecting ink droplets. Then, a multicolor image is formed on the upper surface of the printing paper 9 by superimposing the four monochromatic images. Therefore, if the positions of the ink droplets ejected from the four recording heads 21 to 24 on the printing paper 9 in the transport direction are deviated from each other, the image quality of the printed matter deteriorates. Suppressing such an error in the position of a single-color image on the printing paper 9 within an allowable range is an important factor for improving the print quality of the image recording apparatus 1.

A drying processing unit for drying the ink discharged on the recording surface of the printing paper 9 may be further provided on the downstream side of the recording heads 21 to 24 in the transport direction. The drying processing unit dries the ink by, for example, blowing a heated gas toward the printing paper 9 to vaporize the solvent in the ink adhering to the printing paper 9. However, the drying processing unit may be one that dries the ink by another method such as heating with a heat roller or light irradiation.

Each of the two edge position detection units 30 is a detection unit that detects the position of the edge (edge in the width direction) 91 of the printing paper 9 in the width direction. In the present embodiment, the first detection position Pa on the upstream side of the first processing position P1 on the transfer path and the fourth processing position P4 further downstream from the first detection position Pa on the transfer path are further downstream. The edge position detection unit 30 is arranged at the second detection position Pb on the side. However, the edge position detection unit 30 does not necessarily have to be provided.

FIG. 3 is a diagram schematically showing the structure of the edge position detection unit 30. As shown in FIG. 3, the edge position detecting unit 30 has a floodlight 301 located above the edge 91 of the printing paper 9 and a line sensor 302 located below the edge 91. The floodlight 301 irradiates parallel light downward. The line sensor 302 has a plurality of light receiving elements 321 arranged in the width direction. As shown in FIG. 3, outside the edge 91 of the printing paper 9, the light emitted from the floodlight 301 is incident on the light receiving element 321 and the light receiving element 321 detects the light. On the other hand, inside the edge 91 of the printing paper 9, the light emitted from the floodlight 301 is blocked by the printing paper 9, so that the light receiving element 321 does not detect the light. The edge position detection unit 30 detects the position of the edge 91 of the printing paper 9 in the width direction based on the presence or absence of light detection in the plurality of light receiving elements 321.

As shown in FIGS. 1 and 2, in the following, the edge position detection unit 30 arranged at the first detection position Pa will be referred to as a first edge position detection unit 31. Further, the edge position detection unit 30 arranged at the second detection position Pb is referred to as a second edge position detection unit 32. The first edge position detection unit 31 intermittently detects the position of the edge 91 of the printing paper 9 in the width direction at the first detection position Pa. As a result, the detection result indicating the time-dependent change of the position of the edge 91 in the width direction at the first detection position Pa is acquired. Then, a detection signal (hereinafter, referred to as “first edge signal Ed1”) indicating the obtained detection result is output to the control unit 80. The second edge position detection unit 32 intermittently detects the position of the edge 91 of the printing paper 9 in the width direction at the second detection position Pb. As a result, the detection result indicating the time-dependent change of the position of the edge 91 in the width direction at the second detection position Pb is acquired. Then, a detection signal (hereinafter, referred to as “second edge signal Ed2”) indicating the obtained detection result is output to the control unit 80. However, the first edge position detection unit 31 and the second edge position detection unit 32 may continuously detect the position of the edge 91 of the printing paper 9 in the width direction, respectively.

The encoder 40 is attached to the shaft core of one of the plurality of transfer rollers 12 (transfer roller 122 in FIG. 1). However, the roller to which the encoder 40 is attached is not limited to the transport roller 122. The encoder 40 may be attached to at least one of a plurality of rollers included in the transport mechanism 10. In the present embodiment, the encoder 40 detects the rotation drive amount of the transfer roller 122 and outputs a continuous pulse signal En synchronized with the rotation of the transfer roller 122 to the control unit 80. The continuous pulse signal En is data that reflects a change over time in the transfer speed of the printing paper 9 conveyed by the plurality of transfer rollers 12 including the transfer roller 122. Based on the input continuous pulse signal En, the control unit 80 calculates the measured value Ms of the transfer speed of the printing paper 9 conveyed by the plurality of transfer rollers 12 including the transfer roller 122 in the speed calculation unit 81 described later. do.

The tension detection unit 50 is attached to one of the plurality of transfer rollers 12 (convey roller 123 in FIG. 1). However, the roller to which the tension detection unit 50 is attached is not limited to the transport roller 123. The tension detection unit 50 may be attached to at least one of a plurality of rollers included in the transport mechanism 10. In the present embodiment, the tension detection unit 50 continuously or intermittently measures the force received from the printing paper 9 by the transport roller 123. As a result, the tension detection unit 50 detects the tension applied to the printing paper 9, and outputs the tension signal Te related to the detection result to the control unit 80. The tension signal Te is data that reflects the change over time in the tension applied to the printing paper 9 conveyed by the plurality of transfer rollers 12 including the transfer roller 123 while being in contact with the transfer roller 123. However, the tension detection unit 50 does not necessarily have to be provided.

The information acquisition unit 60 is a device that acquires information related to various set values and conditions in the image recording device 1. The information acquisition unit 60 includes an input interface such as a touch panel, for example. The worker can use the input interface to perform, for example, the type or characteristics of ink ejected from the plurality of recording heads 21 to 24 of the image recording unit 20, environmental conditions including the ambient temperature or humidity of the printing paper 9, and environmental conditions including the ambient temperature or humidity of the printing paper 9. Information related to the type, shape, thickness, etc. of the printing paper 9 (hereinafter referred to as "information Sc") is input. As a result, the information acquisition unit 60 acquires these information Scs. However, the information acquisition unit 60 may have its own sensor such as a thermometer or a hygrometer. Further, the information acquisition unit 60 may directly acquire the information Sc via the sensor. Further, the information acquisition unit 60 may acquire at least one of the information related to the various set values and conditions described above. Further, the information acquisition unit 60 may acquire information other than the information related to the various set values and conditions described above. The signal related to the information Sc acquired by the information acquisition unit 60 is transmitted to the control unit 80.

The control unit 80 is a means for controlling the operation of each unit in the image recording device 1. As conceptually shown in FIG. 1, the control unit 80 includes a processor 801 such as a CPU, a memory 802 such as a RAM, and a storage unit 803 such as a hard disk drive. Further, the program 80P and the data 80D for controlling the operation of the image recording device 1 are stored in the storage unit 803. Further, as shown by a broken line in FIG. 1, the control unit 80 includes a transport mechanism 10, four recording heads 21 to 24, two edge position detection units 30, an encoder 40, a tension detection unit 50, and an information acquisition unit. It is electrically connected to each of the 60s. Further, the control unit 80 may be further electrically connected to the computer 90.

The control unit 80 reads the program 80P and the data 80D stored in the storage unit 803 into the memory 802, and the processor 801 performs arithmetic processing based on the program 80P and the data 80D to perform arithmetic processing on each of the above units in the image recording device 1. To control the operation. As a result, the printing process in the image recording device 1 proceeds.

Further, the storage unit 803 stores the learned model M3 acquired by the pre-learning described later. Based on the control values output from the learned model M3, the control unit 80 rotates each of the three motors 14 to convey the printing paper 9, and ejects ink from the recording heads 21 to 24 to perform printing. .. As a result, when the image recording apparatus 1 performs the printing process, the transport speed of the printing paper 9 is maintained within a predetermined range.

The computer 90 is provided separately from the transfer path of the printing paper 9. The computer 90 has the form of a commercially available personal computer (personal computer). Further, the computer 90 may have a monitor, a keyboard, a pointing device, a touch panel, or the like (not shown). The computer 90 has a processor 901 such as a CPU, a memory 902 such as a RAM, and a storage unit 903 such as a hard disk drive. The program 90P and the data 90D for causing the computer 90 to execute a predetermined process are stored in the storage unit 903. However, the computer 90 may be incorporated in the control unit 80 described above.

<1-2. About pre-learning ＞
Subsequently, the pre-learning that is executed in advance before the printing process is performed in the image recording apparatus 1 will be described. Of the pre-learning, steps S1 and S4, which will be described later, are performed while actually driving the image recording device 1. The functions of the speed calculation unit 81 and the determination unit 800, which will be described later, used when performing step S1 or step S4, temporarily read the program 80P and the data 80D stored in the storage unit 803 into the memory 802, and the program 80P. And, based on the data 80D, it is realized by the processor 801 performing arithmetic processing. Further, steps S2 and S3, which will be described later, are performed on the computer 90. The function of the determination unit 900, which will be described later, used when performing step S2 and step S3 is to temporarily read the program 90P and data 90D stored in the storage unit 903 into the memory 902, and based on the program 90P and data 90D. Therefore, it is realized by the processor 901 performing arithmetic processing.

FIG. 4 is a flowchart showing the flow of pre-learning. As shown in FIG. 4, when performing pre-learning, first, while actually transporting the printing paper 9 in the image recording device 1, ink is ejected to the surface of the printing paper 9, and each part of the image recording device 1 is performed. Acquire a plurality of various measured values in (step S1). Specifically, the printing paper 9 is actually conveyed while changing a plurality of patterns of control values CVd related to the transfer. The control value CVd related to the transfer is, for example, the rotation speed (rotational speed of the drive roller) of the motor 14 connected to the unwind roller 11, the transfer roller 121, and the take-up roller 13, respectively. At the same time, the ink is actually ejected to the surface of the printing paper 9 while changing a plurality of patterns of the control value CVe related to the ejection of the ink. The control value CVe related to ink ejection is, for example, the ink ejection timing from the recording heads 21 to 24 (ink ejection timing from each nozzle 250). The control value CVd related to the transfer is input from the control unit 80 to each motor 14. The control value CVe related to ink ejection is input from the control unit 80 to the recording heads 21 to 24.

In step S1, the rotation speed of the motor 14 may be constant regardless of time, or may change with the passage of time. Further, the ink ejection timing may be regular or irregular. Further, as the control value CVe related to ejection, the amount of ink ejected from the recording heads 21 to 24 (the amount of ink droplets ejected from each nozzle 250) instead of the ink ejection timing or in addition to the ink ejection timing. Ink may be actually ejected onto the surface of the printing paper 9 while changing a plurality of patterns. Further, the rotation speed of the motor 14 may be excessive to cause slip between the surfaces of the plurality of rollers and the printing paper 9, or the ink ejection amount may be excessive to cause the printing paper 9 to be greatly expanded. .. Further, a wide variety of patterns may be performed, including a case where the power of the image recording device 1 is turned off and the motor 14 is not driven, and a case where ink is not ejected. That is, the control value CVd related to transportation can be freely set as long as it includes at least information related to the rotation speed of the motor 14. Further, the control value CVe related to ink ejection can be freely set as long as it includes at least information related to ink ejection timing from the recording heads 21 to 24. Further, one of the control value CVd related to transport and the control value CVe related to discharge may not be treated as a control value, but may be a predetermined value set in advance.

In step S1, a continuous pulse signal En is input from the encoder 40 to the control unit 80 while actually conveying the printing paper 9 and ejecting ink to the surface of the printing paper 9. The control unit 80 has a speed calculation unit 81 (see FIG. 9 described later). The control unit 80 calculates the measured value Ms of the transport speed of the printing paper 9 in the speed calculation unit 81 based on the input continuous pulse signal En.

However, in step S1, a plurality of measured values other than the measured value Ms of the transport speed of the printing paper 9 may be acquired. For example, the measured values of the acceleration and the frequency of the printed paper 9 to be conveyed may be acquired.

Further, in step S1, the first edge position detection unit 31 detects a change with time in the position of the edge 91 in the width direction at the first detection position Pa, and outputs the first edge signal Ed1 indicating the detection result to the control unit 80. Output to. In the present embodiment, the control unit 80 acquires the first edge signal Ed1 as the first reference value CF1. Further, the second edge position detection unit 32 detects a change with time in the position of the edge 91 in the width direction at the second detection position Pb, and outputs a second edge signal Ed2 indicating the detection result to the control unit 80. In the present embodiment, the control unit 80 acquires the second edge signal Ed2 as the second reference value CF2. Further, in step S1, the tension detection unit 50 detects a change in tension applied to the printing paper 9 with time, and outputs a tension signal Te indicating the detection result to the control unit 80. In the present embodiment, the control unit 80 acquires the tension signal Te as the third reference value CF3. However, the control unit 80 does not have to acquire the first reference value CF1, the second reference value CF2, or the third reference value CF3.

Further, in step S1, the information acquisition unit 60 determines the type or characteristic of the ink (ink information), the environmental conditions including the ambient temperature or humidity of the printing paper 9, and the type, shape, or thickness of the printing paper 9 (printing). Information Sc related to (information on Form 9) is acquired. The signal related to the information Sc acquired by the information acquisition unit 60 is transmitted to the control unit 80.

The acquired measured values Ms of the transport speed of the printing paper 9 are the control value CVd related to transport and the control value CVe related to ejection in the storage unit 803 of the control unit 80, and the above-mentioned first reference values CF1 and second reference. It is associated with the value CF2 or the third reference value CF3 and the information Sc, and is accumulated as a data aggregate DA. FIG. 5 is a diagram conceptually showing an example of a data aggregate DA. However, the data aggregate DA may be manually recorded by a worker on a computer 90 or the like.

When a sufficient number of data is accumulated as the data aggregate DA, the worker is based on the accumulated control values CVd, CVe, measured value Ms, reference values CF1, CF2, CF3, and information Sc on the computer 90. Then, a simple simulation model SiMo showing the relationship between the control values CVd, CVe, the reference values CF1, CF2, CF3, and the information Sc and the measured value Ms is created (step S2). Specifically, while using the measured value Ms of the transfer speed of the printing paper 9 as the teacher data (correct answer data), the transfer of the printing paper 9 is performed from the control values CVd, CVe, the reference values CF1, CF2, CF3, and the information Sc. Machine learning is performed on the simple simulation model SiMo for calculating the speed with high accuracy.

The simple simulation model SiMo includes decision trees X (a, b, c, f (CVd, CVe, CF1, CF2, CF3, Sc) ...). FIG. 6 is a diagram conceptually showing an example of the decision tree X (a, b, c, f (CVd, CVe, CF1, CF2, CF3, Sc) ...) Of the present embodiment. In step S2, in machine learning, the calculated value CAs of the transfer speed of the printing paper 9 calculated based on the input control values CVd, CVe, measured value Ms, reference values CF1, CF2, CF3, and information Sc, and data. Included in the decision tree X (a, b, c, f (CVd, CVe, CF1, CF2, CF3, Sc) ...) so as to minimize the difference from the corresponding measured value Ms accumulated in the aggregate DA. The plurality of parameters (a, b, c, f (CVd, CVe, CF1, CF2, CF3, Sc) ...) Are adjusted and updated and saved.

Machine learning is completed when the difference between the calculated value CAs output from the simple simulation model SiMo and the corresponding measured value Ms stored in the data aggregate DA is equal to or less than a predetermined value. As a result, the transfer speed of the printing paper 9 is calculated with high accuracy based on the input control values CVd, CVe, measured values Ms, reference values CF1, CF2, CF3, and information Sc using the simple simulation model SiMo. It becomes possible to do. However, as the values input to the simple simulation model SiMo, the control values CVd, CVe, the measured values Ms, the reference values CF1, CF2, CF3, and a part of the information Sc may be omitted.

However, the simple simulation model SiMo may include a model created by machine learning (for example, GBM (Gradient Boosting Machine)) other than the decision tree. Further, in the simple simulation model SiMo, instead of the model created by machine learning, the control values CVd, CVe, the reference values CF1, CF2, CF3, or the information Sc, and the calculated value CAs of the transfer speed of the printing paper 9 are used. It may include a calculation formula (function) indicating the relationship. Further, the simple simulation model SiMo includes a conditional expression such as "whether the power of the image recording device 1 is on or off" (for example, an expression that outputs 0 (zero) when the power is off). You may.

Subsequently, on the computer 90, the simple simulation model SiMo is used to autonomously learn (machine learning) the control rules for maintaining the transport speed of the printing paper 9 within a predetermined range (step S3). In step S3, the second-stage learning model M2, which will be described later, is created by performing reinforcement learning. FIG. 7 is a block diagram conceptually showing how reinforcement learning is performed in step S3. Further, FIG. 8 is a flowchart showing a detailed flow of step S3.

When performing reinforcement learning on the computer 90, a pre-learning model M0, which is a prototype of the learned model M3 described later, is prepared based on the reinforcement learning program. Then, the prepared pre-learning model M0 is made to learn the relationship between the control values CVd and CVe and the reward reflecting the output from the simple simulation model SiMo. In the present embodiment, the reference values CF1, CF2, CF3, and the information Sc input to the simple simulation model SiMo are set in advance.

Specifically, first, from the pre-learning model M0, certain values are output as the control value CVd related to the transfer and the control value CVe related to the discharge, and are input to the simple simulation model SiMo (step S31). However, as described above, one of the control value CVd related to transport and the control value CVe related to discharge may be a predetermined value set in advance. Further, regarding the control value CVd related to the transfer, the value input to the motor 14 connected to the unwinding roller 11 and the value input to the motor 14 connected to the transport roller 121 are connected to the take-up roller 13. The values input to the motor 14 may be different from each other.

In the initial stage of step S31, the value output from the pre-learning model M0 is a random value. The simple simulation model SiMo outputs the calculated value CAs related to the transport speed of the printing paper 9 based on the values input from the pre-learning model M0, the above reference values CF1, CF2, CF3, and the information Sc ( Step S32). In the following, the pre-learning model M0 after the learning is started in this way will be referred to as the first-stage learning in-learning model M1.

The computer 90 further has a function as a determination unit 900. Based on the reinforcement learning program, the determination unit 900 grants the reward RE1 according to the calculated value CAs related to the transport speed of the printing paper 9, which is the output value from the simple simulation model SiMo (step S33). At this time, the determination unit 900 gives a higher reward RE1 as the calculated value CAs approaches the preset target value. On the other hand, the determination unit 900 gives a lower reward RE1 as the calculated value CAs deviates from the preset target value.

When the reward RE1 is given by the determination unit 900, the first-stage learning model M1 sets the following values as the control value CVd related to transportation and the control value CVe related to discharge based on the given reward RE1. Output. For example, in the first stage learning model M1, when a high reward RE1 is given by the determination unit 900, a value close to the control values CVd and CVe that led to the high reward RE1 being given is output as the next value. do. Further, in the first-stage learning model M1, when a low reward RE1 is given by the determination unit 900, a value away from the control values CVd and CVe that led to the low reward RE1 being given is set as the next value. Output.

In this way, in step S3, while updating the values output from the first-stage learning model M1 (control value CVd related to transport and control value CVe related to discharge), they are input to the simple simulation model SiMo, and further simple simulation is performed. Reinforcement learning is performed to maintain the calculated value CAs from the model SiMo within a predetermined range. Further, in the present embodiment, as reinforcement learning (deep reinforcement learning), for example, machine learning executed by a technique of PPO (Proximal Policy Optimization) is performed. However, as reinforcement learning, machine learning executed by techniques such as DQN (Deep Q Network), Q-learning method, SARSA, Monte Carlo method, epsilon-greedy method, and Boltzmann QPolicy method may be performed.

As described above, in the initial stage of reinforcement learning on the computer 90, the first stage learning model M1 outputs random values as control values CVd and CVe. However, in the process of repeating the processes of steps S31 to S33, the computer 90 tries to increase, maintain, decrease, and the like the control values CVd and CVe according to the reward RE1 given by the determination unit 900. As a result, the relationship between the control values CVd and CVe and the reward reflecting the output from the simple simulation model SiMo is automatically learned. Then, the first-stage learning model M1 can gradually obtain a high reward RE1. That is, the first-stage learning model M1 can gradually output the control values CVd and CVe for maintaining the calculated values CAs from the simple simulation model SiMo within a predetermined range.

After that, the computer 90 determines whether or not to end the reinforcement learning based on the reinforcement learning program (step S34). If the reward RE1 by the determination unit 900 does not reach the desired level, the reinforcement learning on the computer 90 is continued (step S34: no). In that case, the process of steps S31 to S34 described above is executed again.

On the other hand, in step S34, when it is determined that the reward RE1 to be given has reached a desired level, the reinforcement learning on the computer 90 is terminated based on the reinforcement learning program (step S34: yes). Then, the first-stage learning model M1 for which the reinforcement learning on the computer 90 is completed becomes the second-stage learning model M2.

After that, the second-stage learning model M2 is installed in the storage unit 803 of the control unit 80. Then, as shown in FIG. 4, a control rule for maintaining the transport speed of the printing paper 9 within a predetermined range while ejecting ink to the surface of the printing paper 9 while actually transporting the printing paper 9 again. Is autonomously reinforced learning (machine learning) (step S4).

FIG. 9 is a block diagram conceptually showing how reinforcement learning is performed in step S4. Further, FIG. 10 is a flowchart showing a detailed flow of step S4. As shown in FIGS. 9 and 10, when performing reinforcement learning in an actual device, first, a certain value is output as a control value CVd (rotational speed of the motor 14) related to transportation from the model M2 during the second stage learning. Then, based on this value, the motor 14 connected to the unwinding roller 11, the transport roller 121, and the winding roller 13, which are the driving rollers, is rotated. At the same time, a certain value is output from the second-stage learning model M2 as the control value CVe (ejection timing) related to ejection, and ink is ejected from the recording heads 21 to 24 based on this value (step S41). ..

Next, as described above, while transporting the printing paper 9, ink is ejected to the surface of the printing paper 9, and a continuous pulse signal En is further input from the encoder 40 to the control unit 80. The control unit 80 acquires the measured value Ms of the transport speed of the printing paper 9 in the speed calculation unit 81 based on the input continuous pulse signal En (step S42).

The control unit 80 further has a function as a determination unit 800. Based on the reinforcement learning program, the determination unit 800 grants the reward RE2 according to the measured value Ms of the transport speed of the printing paper 9 (step S43). At this time, the determination unit 800 gives a higher reward RE2 as the measured value Ms approaches the preset target value. On the other hand, the determination unit 800 gives a lower reward RE2 as the measured value Ms deviates from the preset target value.

When the reward RE2 is given by the determination unit 800, the second-stage learning model M2 sets the following values as the control value CVd related to transportation and the control value CVe related to discharge based on the given reward RE2. Output. For example, in the second stage learning model M2, when a high reward RE2 is given by the determination unit 800, a value close to the control values CVd and CVe that led to the high reward RE2 being given is output as the next value. do. Further, in the second-stage learning model M2, when a low reward RE2 is given by the determination unit 800, a value away from the control values CVd and CVe that led to the low reward RE2 being given is set as the next value. Output.

In this way, in step S4, the motor 14 and the recording heads 21 to 24 are driven while updating the values output from the second-stage learning model M2 (control value CVd related to transport and control value CVe related to discharge). , Reinforcement learning is performed to maintain the measured value Ms of the transport speed of the printing paper 9 based on the continuous pulse signal En from the encoder 40 within a predetermined range. Further, in the present embodiment, as reinforcement learning (deep reinforcement learning), for example, machine learning executed by a technique of PPO (Proximal Policy Optimization) is performed. However, as reinforcement learning, machine learning executed by techniques such as DQN (Deep Q Network), Q-learning method, SARSA, Monte Carlo method, epsilon-greedy method, and Boltzmann QPolicy method may be performed.

As described above, in step S4, further reinforcement learning is performed in the actual device using the second-stage learning in-flight model M2 for which reinforcement learning on the computer 90 has been completed in advance. Then, in the process of repeating the processes of steps S41 to S43, the control values CVd and CVe are increased, maintained, decreased, and the like in the actual device according to the reward RE2 given by the determination unit 800. As a result, the relationship between the control values CVd and CVe and the reward reflecting the measured value Ms of the transport speed of the printing paper 9 actually transported is automatically learned. Then, the second-stage learning model M2 can gradually obtain a high reward RE2. That is, the second-stage learning model M2 can gradually output the control values CVd and CVe for maintaining the measured value Ms of the transport speed of the printing paper 9 within a predetermined range in the actual device.

After that, on the control unit 80, it is determined whether or not to end the reinforcement learning based on the reinforcement learning program (step S44). If the reward RE2 by the determination unit 800 does not reach the desired level, reinforcement learning using the actual device is continued (step S44: no). In that case, the processes of steps S41 to S44 described above are executed again.

On the other hand, in step S44, when it is determined that the reward RE2 to be given has reached a desired level, the reinforcement learning in the actual device is terminated based on the reinforcement learning program (step S44: yes). Then, the second-stage learning model M2 for which the reinforcement learning on the actual device is completed becomes the trained model M3.

As described above, in the present embodiment, a simple simulation model SiMo is created on the computer 90 (step S2) using the result of collecting data in the actual device in advance (step S1), and the simple simulation model SiMo is used. Reinforcement learning is performed on the computer 90 (step S3). Then, the second-stage learning model M2 obtained by performing reinforcement learning on the computer 90 is transferred to the actual device again, and reinforcement learning is continuously performed (step S4). As a result, reinforcement learning can be started from a state advanced to some extent commensurate with the operation of the actual device. In addition, a considerable amount of reinforcement learning can be performed on the computer 90. As a result, the time required for machine learning can be significantly reduced. Further, even if the control values CVd and CVe output from the model M1 during the first stage learning on the computer 90 become excessive in the initial stage of learning, it is possible to prevent damage to the actual device. Furthermore, by finally performing reinforcement learning using the actual device again, it is possible to create a trained model M3 that outputs more accurate control values CVd and CVe. As a result, for example, in line with the characteristics of each actual device (for example, the characteristics that appear with the aging of the device, the difference in thickness due to minute irregularities on the printing paper 9, the slight difference in viscosity for each ink, etc.). Good operation can be achieved.

After the reinforcement learning in the actual device according to step S4 is completed, the printing process in the image recording device 1 is started. Based on the learned model M3, the control unit 80 rotates each of the three motors 14 to convey the printing paper 9, and ejects ink from the recording heads 21 to 24 to perform printing. As a result, the transport speed of the printing paper 9 is maintained within a predetermined range even when the surface of the roller of the transport mechanism 10 and the printing paper 9 slip or the printing paper 9 is stretched by the ink. As a result, an error in the transport direction of the ink ejection position of each color on the printing paper 9 is suppressed. As a result, the print quality of the image recording device 1 can be improved.

<2. Modification example>
Although one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment. Hereinafter, the differences between the various modifications and the above-described embodiments will be described.

<2-1. First modification>
In the above embodiment, the speed calculation unit 81 of the control unit 80 is conveyed by a plurality of transfer rollers 12 including the transfer roller 122 based on the continuous pulse signal En input from the encoder 40 connected to the transfer roller 122. The measured value Ms of the transport speed of the printing paper 9 was calculated. However, the control unit 80 determines the transfer speed of the printing paper 9 based on the first edge signal Ed1 input from the first edge position detection unit 31 and the second edge signal Ed2 input from the second edge position detection unit 32. The measured value Ms may be calculated. Details will be described below.

FIG. 11 is a graph showing an example of the first edge signal Ed1 and an example of the second edge signal Ed2, respectively. In FIG. 11, the horizontal axis represents the time. The vertical axis of FIG. 11 indicates the position of the edge 91 in the width direction. On the horizontal axis of the graph of FIG. 11, the left end is the current time, and the time becomes older toward the right side. Therefore, the data line in FIG. 11 moves to the right as shown by the white arrow with the passage of time. The edge 91 of the printing paper 9 has fine irregularities. The first edge position detection unit 31 and the second edge position detection unit 32 detect the position of the edge 91 of the printing paper 9 in the width direction every minute time set in advance (for example, every 50 microseconds). As a result, as shown in FIG. 11, data showing the time-dependent change in the position of the edge 91 of the printing paper 9 in the width direction can be obtained. The first edge signal Ed1 is data that reflects the shape of the edge 91 of the printing paper 9 that passes through the first detection position Pa. The second edge signal Ed2 is data that reflects the shape of the edge 91 of the printing paper 9 that passes through the second detection position Pb.

The control unit 80 compares the first edge signal Ed1 and the second edge signal Ed2. Then, the location where the same edge 91 of the printing paper 9 is detected is specified by the first edge signal Ed1 and the second edge signal Ed2. Specifically, for each data section (constant time range) included in the first edge signal Ed1, data with high consistency among a plurality of data sections (constant time range) included in the second edge signal Ed2. Identify the section. Hereinafter, the data section included in the first edge signal Ed1 is referred to as the first data section DS1. Further, the data section included in the second edge signal Ed2 is referred to as a second data section DS2.

For example, a matching method such as cross-correlation or residual sum of squares is used to identify a data interval with high consistency. The control unit 80 selects a plurality of second data sections DS2 as candidates for the corresponding data sections for each first data section DS1. In addition, for each of the plurality of second data sections DS2 that are selected candidates, an evaluation value indicating consistency with the first data section DS1 is calculated. Then, the second data section DS2 having the highest evaluation value is specified as the second data section DS2 corresponding to the first data section DS1 (which has the highest consistency with the first data section DS1).

After that, the control unit 80 determines the first detection position Pa based on the time difference between the detection time TM1 of the first data section DS1 and the detection time TM2 of the second data section DS2 having the highest coincidence with the first data section DS1. The actual transport time ΔT (time difference between the time TM2 and the time TM1) required for transporting the printing paper 9 from the second detection position Pb to the second detection position Pb is calculated. Then, by dividing the distance from the first detection position Pa to the second detection position Pb by the transfer time ΔT, the transfer speed of the printing paper 9 below the image recording unit 20 can be calculated.

<2-2. Second modification>
In addition to the methods described in the above embodiments or modifications, a laser speedometer (not shown) for detecting the transport speed of the printing paper 9 may be used. Then, the laser speed meter irradiates the printing paper 9 with two laser beams, and the printing paper 9 is conveyed based on the result of analyzing the wavelengths of the reflected light reflected by the printing paper 9, respectively. The speed may be calculated.

<2-3. Third variant>
In the above embodiment, reinforcement learning for maintaining the measured value Ms of the transport speed of the printing paper 9 within a predetermined range is performed during the pre-learning that is executed in advance before the printing process is performed in the image recording apparatus 1. Was there. However, in the pre-learning, reinforcement learning for maintaining the measured value Mt of the tension of the printing paper 9 within a predetermined range may be performed.

Specifically, in step S1 described above, the tension detection unit is based on the tension signal Te output from the tension detection unit 50 while actually transporting the printing paper 9 and ejecting ink to the surface of the printing paper 9. Even if a plurality of measured values of tension applied to the printing paper 9 conveyed by a plurality of transfer rollers 12 including the transfer roller 123 while being in contact with the transfer roller 123 to which the 50 is connected are acquired and stored in the data aggregate DA. good.

Further, in step S2 above, the printing paper 9 is obtained from the control values CVd, CVe, the reference values CF1, CF2, CF3, and the information Sc, while using the measured value Mt of the tension of the printing paper 9 as the teacher data (correct answer data). The simple simulation model SiMo for calculating the tension of the above with high accuracy may be machine-learned. Then, the simple simulation model SiMo includes the decision trees X (a, b, c, f (CVd, CVe, CF1, CF2, CF3, Sc) ...), and the input control values CVd, CVe, measured values Ms, To minimize the difference between the calculated tension CAt of the printing paper 9 calculated based on the reference values CF1, CF2, CF3, and the information Sc, and the corresponding measured value Mt stored in the data aggregate DA. A plurality of parameters (a, b, c, f (CVd, CVe, CF1, CF2, CF3) included in the decision tree X (a, b, c, f (CVd, CVe, CF1, CF2, CF3, Sc) ...) , Sc) ...) may be updated and saved. Machine learning is completed when the difference between the calculated value CAt of the tension of the printing paper 9 which is the output value from the simple simulation model SiMo and the corresponding measured value Mt stored in the data aggregate DA becomes equal to or less than a predetermined value.

Further, in step S3 described above, the control rule for maintaining the tension of the printing paper 9 within a predetermined range may be machine-learned on the computer 90 by using the simple simulation model SiMo. Specifically, in step S31, certain values may be output as the control value CVd related to the transfer and the control value CVe related to the discharge, and input to the simple simulation model SiMo. In step S32, the simple simulation model SiMo outputs the calculated value CAt of the tension of the printing paper 9 based on the values input from the pre-learning model M0, the above reference values CF1, CF2, CF3, and the information Sc. You may.

In step S33, the reward RE1 corresponding to the calculated value CAt of the tension of the printing paper 9, which is the output value from the simple simulation model SiMo, may be given. As a result, in step S3, while updating the values output from the first-stage learning model M1 (control value CVd related to transport and control value CVe related to discharge), the values are input to the simple simulation model SiMo, and further, the simple simulation model. Reinforcement learning may be performed to maintain the CAt from SiMo within a predetermined range. Further, in step S34, when it is determined that the reward RE1 has reached a desired level, the reinforcement learning on the computer 90 may be terminated. Then, the learning model created in step S3 may be used as the second-stage learning in-progress model M2.

Further, in step S4 described above, a certain value is output as the control value CVd (rotational speed of the motor 14) related to the transfer from the second stage learning model M2 created in step S3, and the drive is performed based on this value. The motor 14 connected to the unwinding roller 11, the transport roller 121, and the winding roller 13, which are rollers, may be rotated. At the same time, a certain value may be output from the second-stage learning model M2 as a control value CVe (ejection timing) related to ejection, and ink may be ejected from the recording heads 21 to 24 based on this value ( Step S41). Further, in step S42, the measured value Mt of the tension applied to the printing paper 9 at this time may be acquired based on the tension signal Te output from the tension detection unit 50.

In step S43, the reward RE2 corresponding to the measured value Mt of the tension of the printing paper 9 may be given. Then, in step S44, when it is determined that the reward RE2 has reached a desired level, reinforcement learning using the actual device may be terminated.

Further, in the pre-learning, in addition to the above-described embodiment and modification, reinforcement learning for maintaining the measured value of the elongation of the printing paper 9 within a predetermined range may be performed.

<2-4. Other variants>
Further, the above-mentioned image recording device 1 discharges ink as a processing substance while conveying the printing paper 9, and records an image on the surface of the printing paper 9. However, the image recording device 1 may record an image by ejecting a processing substance other than ink onto the surface of the printing paper 9. Further, the transport processing device of the present invention may be a device that records an image on the printing paper 9 by a method other than inkjet (for example, electrophotographic method, exposure, etc.). Further, the above-mentioned image recording apparatus 1 performs a printing process on printing paper 9 as a base material. However, the transport processing apparatus of the present invention may perform a predetermined treatment on a long strip-shaped base material (for example, a resin film, a metal foil, etc.) other than general paper.

Further, the elements appearing in the above-described embodiments and modifications may be appropriately combined as long as there is no contradiction.

1 Image recording device 9 Printing paper 10 Conveying mechanism 11 Unwinding roller 12 Conveying roller 13 Rewinding roller 14 Motor 20 Image recording unit 21 1st recording head 22 2nd recording head 23 3rd recording head 24 4th recording head 30 Edge position Detection unit 31 1st edge position detection unit 32 2nd edge position detection unit 40 Encoder 50 Tension detection unit 60 Information acquisition unit 80 Control unit 81 Speed calculation unit 90 Computer 91 Edge 121 Transport roller 122 Transport roller 123 Transport roller 800 Judgment unit 900 Judgment unit CAs (conveyance speed) calculated value CAt (tension) calculated value CF1, CF2, CF3 Reference value CVd Conveyance control value CVe discharge control value DA data aggregate M0 Pre-learning model M1 First stage learning Model M2 Second stage Learning model M3 Trained model Ms (conveyance speed) measured value Mt (tension) measured value RE1, RE2 Reward Re1, Re2 Reference value Sc Information SiMo Simple simulation model Te Tension signal X (a, b) , C, f (CVd, CVe, CF1, CF2, CF3, Sc) ...)) Determining tree

Claims (8)

A transport processing method in which a processing substance is discharged onto the surface of the substrate while the long strip-shaped substrate is conveyed in the longitudinal direction.
a) A control value related to transportation including the rotation speed of a motor which is a drive source for transporting the base material while actually transporting the base material, or a control value related to discharge including the discharge timing of the processed substance. And the step of acquiring a plurality of measured values related to the tension or the transport speed of the base material, and
b) A step of creating a simple simulation model showing the relationship between the control value and the measured value on a computer based on the acquisition result in the step a).
c) A step of creating a learning model by performing reinforcement learning to maintain the output value from the simple simulation model within a predetermined range while updating the control value on the computer.
d) While updating the control value output from the learning model, the measured value in the case of discharging the processed substance to the surface of the base material while actually transporting the base material is set within a predetermined range. The process of performing reinforcement learning to maintain,
A transport processing method. The transport processing method according to claim 1.
The base material is spread over a plurality of rollers, and is conveyed by rotating a drive roller, which is at least one of the plurality of rollers, driven by the motor.
The control value related to the transfer is a transfer processing method including the rotation speed of the drive roller. The transport processing method according to claim 1 or 2.
The control value related to the discharge is a transport processing method including a discharge amount of the processing substance. The transport processing method according to any one of claims 1 to 3.
In the step a), information on the thickness or type of the base material is further acquired.
In the step b), a transport processing method for creating the simple simulation model showing the relationship between the control value, the information on the base material, and the measured value based on the acquisition result in the step a). The transport processing method according to any one of claims 1 to 4.
In the step a), information on the characteristics of the processed substance is further acquired, and the information is obtained.
In the step b), a transport processing method for creating the simple simulation model showing the relationship between the control value, the information on the processing substance, and the measured value based on the acquisition result in the step a). The transport processing method according to any one of claims 1 to 5.
In the step a), information on environmental conditions including the temperature or humidity around the base material is further acquired.
In the step b), a transport processing method for creating the simple simulation model showing the relationship between the control value, the information related to the environmental condition, and the measured value based on the acquisition result in the step a). The transport processing method according to any one of claims 1 to 6.
In the step b), the simple simulation model includes a decision tree, and the parameters included in the decision tree are adjusted. The transport processing method according to any one of claims 1 to 7.
The reinforcement learning performed in the step c) is a transfer processing method, which is machine learning executed by a technique of PPO or DQN.

Images