JP2023161377A - Machine learning device, data processing system, machine learning method, and data processing method - Google Patents

Abstract

To improve the accuracy of prediction through machine learning.SOLUTION: A machine learning device comprises: a state variable acquisition unit that acquires, as state variables, feature quantity information in an actual printed material obtained by printing, by an image forming apparatus, a white toner image and a color toner image in an overlapped state, the image forming apparatus having a first image forming unit forming the white toner image and second image forming units forming the color toner image, printing failure characteristic information indicating the type of printing failure included in the actual printed material, medium information on a print medium used for the actual printed material, and first control information that is control information when the image forming apparatus prints the actual printed material; a training data acquisition unit that acquires, as training data, second control information in which the feature quantity information is within a predetermined threshold; and a learned model generation unit that generates a learned model by performing machine learning based on the state variables acquired by the state variable acquisition unit and the training data acquired by the training data acquisition unit.SELECTED DRAWING: Figure 2

Description

The present invention relates to a machine learning device, a data processing system, a machine learning method, and a data processing method.

Conventionally, an image obtained by scanning a printed matter printed by an image forming apparatus, medium information of the printing medium used for the printed matter, and control information (secondary transfer voltage) of the image forming apparatus when printing the said printed matter. and toner fixing temperature) into a trained model, and the image forming apparatus There is a system for updating the control information of (for example, see Patent Document 1).

JP2021-33236A

By the way, in an electrophotographic image forming apparatus, for example, when printing a color toner layer on top of a white toner layer as a base, a printing defect occurs in which the white toner layer mixes with the upper color toner layer and stands out. There are things to do. This type of printing defect is called sinking.

In conventional systems, it is not possible to distinguish what type of printing defect is occurring, so when a printing defect such as sinking occurs, the trained model is affected by this printing defect and outputs. There has been a problem in that the prediction accuracy of control information (that is, the prediction accuracy by machine learning) decreases.

The present invention takes the above points into consideration, and aims to propose a machine learning device, a data processing system, a machine learning method, and a data processing method that can improve prediction accuracy by machine learning.

The machine learning device of the present invention prints a white toner image and a color toner image in a superimposed manner using an image forming apparatus having a first image forming section that forms a white toner image and a second image forming section that forms a color toner image. feature information on the actual printed matter, printing defect characteristic information indicating the type of printing defect included in the actual printed matter, medium information of a print medium used in the actual printed matter, a state variable acquisition unit that acquires first control information that is control information when printing as a state variable; and a teacher data acquisition unit that acquires second control information that makes the feature amount information within a predetermined threshold as teacher data. and a trained model generation unit that generates a trained model by performing machine learning based on the state variables acquired by the state variable acquisition unit and the teacher data acquired by the teacher data acquisition unit. Be prepared.

Further, the data processing system of the present invention overlaps a white toner image and a color toner image using an image forming apparatus having a first image forming section that forms a white toner image and a second image forming section that forms a color toner image. The image forming apparatus includes feature information in the printed actual printed matter, printing defect characteristic information indicating the type of printing defect included in the actual printed matter, medium information of the print medium used in the actual printed matter, and an actual printed matter information acquisition unit that acquires first control information that is control information when printing printed matter; and an actual printed matter information acquisition unit that acquires first control information that is control information when printing printed matter, and the learned information generated by the machine learning device described above. The data processing unit includes a data processing unit that inputs the third control information to the model and outputs the third control information, and a third control information storage unit that stores the third control information output from the data processing unit.

Furthermore, the machine learning method of the present invention
A computer is used to print a white toner image and a color toner image superimposed on each other by an image forming apparatus having a first image forming section that forms a white toner image and a second image forming section that forms a color toner image. feature amount information on the printed matter, printing defect characteristic information indicating the type of printing defect included in the actual printed matter, medium information of a printing medium used for the actual printed matter, and information on how the image forming device printed the actual printed matter. a first process of acquiring first control information that is control information at the time as a state variable; a second process of acquiring second control information that makes the feature amount information within a predetermined threshold as teacher data; and a third process of generating a learned model by performing machine learning based on the state variable acquired by the first process and the teacher data acquired by the second process.

Furthermore, the data processing method of the present invention uses a computer to create a white toner image and a color toner image by an image forming apparatus having a first image forming section that forms a white toner image and a second image forming section that forms a color toner image. feature amount information of the actual printed matter printed in a superimposed manner with the actual printed matter, printing defect characteristic information indicating the type of printing defect included in the actual printed matter, medium information of the printing medium used for the actual printed matter, and the image. a fourth process of acquiring first control information that is control information when the forming device prints the actual printed matter; and a fourth process of acquiring first control information that is control information when the forming device prints the actual printed matter, and a process that uses the information acquired by the fourth process to generate information using the machine learning method described above. The method includes a fifth process of inputting the learned model and outputting third control information, and a sixth process of storing the third control information output by the fifth process.

According to the present invention, in addition to the feature information of the actual printed matter, the medium information of the print medium used for the actual printed matter, and the first control information when printing the actual printed matter, the state variables are included in the actual printed matter. Since machine learning is performed using printing defect characteristic information that indicates the type of printing defects that occur, it is possible to generate a desired trained model with high accuracy, and use the trained model to improve the prediction accuracy of control information. can be done.

The present invention can realize a machine learning device, a data processing system, a machine learning method, and a data processing method that can improve prediction accuracy by machine learning.

FIG. 1 is a diagram showing the basic configuration (printing mechanism) of an image forming apparatus. FIG. 1 is a diagram showing the configuration of a machine learning device. It is a figure showing the composition of a diagnostic chart. FIG. 7 is a diagram showing types of printing defects in a predicted fixing temperature region. FIG. 2 is a diagram showing a neural network model for machine learning. 7 is a flowchart showing the procedure of a method for determining the type of printing defect. It is a flowchart showing the procedure of a machine learning method. FIG. 1 is a diagram showing a functional configuration of a data processing system to which a trained model is applied. 1 is a diagram showing an external configuration of an image forming apparatus included in a data processing system. 7 is a flowchart illustrating an operation procedure when predicting control information using a learned model and reflecting the predicted control information on the image forming apparatus. 11 is a flowchart showing a specific procedure of the process of step SP35 in the flowchart shown in FIG. 10. 7 is a graph showing the predicted improvement range achievement rate of toner fixing temperature and the predicted improvement range achievement rate of secondary transfer voltage. 7 is a graph showing a predicted result of toner fixing temperature (before rounding processing).

DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, modes for carrying out the invention (hereinafter referred to as embodiments) will be described in detail using the drawings. In the following, the parts necessary to achieve the object of the present invention will be mainly explained, and the explanation of other known technology parts will be omitted as appropriate.

First, the basic configuration of an electrophotographic image forming apparatus that performs print output as a learning target of a machine learning device and a machine learning method in this embodiment will be described.

[1. Configuration of image forming apparatus]
FIG. 1 shows the basic configuration (printing mechanism) of an image forming apparatus 10 in this embodiment. The image forming apparatus 10 illustrated in FIG. 1 is a so-called intermediate transfer type color printer, and includes a print medium supply section 20, an image forming section 30, a transfer section 40, a fixing section 50, an output section 60, A control unit 70 is provided.

The print medium supply section 20 is for supplying the print medium PM into the image forming apparatus 10, and includes a paper tray 21, a manual feed tray 22, and a plurality of paper feed rollers 23. The paper tray 21 accommodates a plurality of print media PM in a stacked manner. The print medium PM accommodated in this paper tray 21 is generally general paper (A4 plain paper, B4 plain paper, etc.). The manual feed tray 22 is provided in a storable manner on the side of the main body of the image forming apparatus 10, and serves as a tray for feeding a special print medium PM when printing is mainly performed on a special print medium PM different from general paper. It is. Here, the special printing medium PM different from general paper is, for example, label paper. Note that the label paper is an existing one, so a detailed explanation will be omitted, but for example, a label is pasted onto a mount with an adhesive. The plurality of paper feed rollers 23 are rollers arranged at appropriate positions to transport the print medium PM in the paper tray 21 or placed on the manual feed tray 22 to the transport path PL.

The image forming section 30 is for forming images (toner images), and includes a plurality of (five in this embodiment) image forming units 31 (31W, 31K, 31C, 31M, 31Y) arranged in parallel. and a plurality of exposure means (LED heads) 34. The plurality of image forming units 31 (31W, 31K, 31C, 31M, 31Y) have basically the same configuration except for the toner colors, and mainly include a photoreceptor drum 32, a charging roller, It includes a developing roller 33 as a developer carrier, a supply roller, and a toner tank.

The plurality of image forming units 31 (31W, 31K, 31C, 31M, 31Y) form toner images of white, black, cyan, magenta, and yellow, respectively. It is a unit for The image forming apparatus 10 is capable of color printing using the plurality of image forming units 31. Specifically, in each image forming unit 31, an electrostatic latent image formed on the surface of the photoreceptor drum 32 by light irradiated from the exposure means 34 is supplied from a toner tank to the developing roller 33 via a supply roller. A toner image is formed on the surface of the photoreceptor drum 32 by attaching and developing the toner. The toner images formed on the photosensitive drums 32 of each image forming unit 31 are transferred to an intermediate transfer belt 41 of a transfer section 40, which will be described later.

Note that the voltages (charging voltage, developing voltage, supply voltage) applied to each of the charging roller, developing roller 33, and supply roller of each image forming unit 31 are controlled by a control section 70, which will be described later.

The transfer unit 40 is for transferring the toner image formed by the image forming unit 30 onto the printing medium PM, and includes an intermediate transfer belt 41 and a plurality of primary transfer rollers 42 (42W, 42K, 42C, 42M, 42Y). ), a backup roller 43, and a secondary transfer roller 44. The intermediate transfer belt 41 is an annular elastic belt that is supported by a plurality of rollers including a driving roller and is mainly made of a resin material such as rubber. The toner images of each color formed by the image forming unit 31 are transferred (primary transfer) onto the surface of the intermediate transfer belt 41, and the formed toner images are later transferred to the printing medium PM (secondary transfer). ) to be done. The plurality of primary transfer rollers 42 (42W, 42K, 42C, 42M, 42Y) are for transferring the toner images of each color formed by the image forming unit 31 onto the intermediate transfer belt 41, and the plurality of image forming units 31 (31W, 31K, 31C, 31M, 31Y), and is arranged to face each of the photoreceptor drums 32, and is arranged so that the intermediate transfer belt 41 is sandwiched between each of the opposing photoreceptor drums 32. A predetermined primary transfer voltage is applied to the primary transfer roller 42 . The primary transfer voltage is controlled by a control section 70, which will be described later.

The backup roller 43 is one of a plurality of rollers that support the intermediate transfer belt 41, and is disposed at a position facing a secondary transfer roller 44, which will be described later, with the intermediate transfer belt 41 interposed therebetween. The secondary transfer roller 44 is disposed in the middle of the conveyance path PL at a position facing the backup roller 43 with the intermediate transfer belt 41 interposed therebetween. It functions to transfer the toner image formed in advance onto the intermediate transfer belt 41 when the printing medium PM passes through the intermediary transfer belt 41 . A predetermined secondary transfer voltage is applied to this secondary transfer roller 44 . The secondary transfer voltage is controlled by a control section 70, which will be described later.

The fixing unit 50 is for fixing the toner image onto the print medium PM by applying heat and pressure to the print medium PM to which the toner image has been transferred by the transfer unit 40, and is connected to the fixing roller 51 and pressure. A roller 52 is provided. The fixing roller 51 has a built-in heater (not shown), and the fixing temperature is controlled by the current value supplied to the heater. The current value supplied to this heater is controlled by a control section 70, which will be described later. The pressure roller 52 has a biasing force applied to the fixing roller 51 . As a result, a predetermined fixing pressure is applied to the print medium PM passing between the fixing roller 51 and the pressure roller 52.

The output unit 60 outputs the print medium PM on which the toner image is fixed by the fixing unit 50 to the outside of the image forming apparatus 10 as an actual printed matter, and includes an ejection tray 61 and a plurality of conveyance rollers 62 . The ejection tray 61 is formed at the upper part of the image forming apparatus 10, and is used to place the actual printed matter AP outputted via the transport path PL. The plurality of conveyance rollers 62 are provided at various locations on the conveyance path PL, and convey the print medium PM to the discharge tray. Incidentally, a cooling means may be optionally provided at any position of the output section 60 to remove the heat generated when the toner image is fixed.

The control section 70 is for controlling each section of the image forming apparatus 10, and is composed of a well-known CPU, storage means (memory), and the like. As described above, this control section 70 controls the reference light amount of the exposure means 34, the voltages (charging voltage, developing voltage, supply voltage) applied to each of the charging roller, developing roller 33, and supply roller, and the secondary transfer roller It has a function of controlling the secondary transfer voltage applied to the fixing roller 44 and the current supplied to the heater inside the fixing roller 51. The basic configuration of the image forming apparatus 10 has been described above.

In this embodiment, an intermediate transfer type color printer is used as the image forming apparatus 10, but the image forming apparatus 10 is not limited to this, and a tandem type or rotary type that directly transfers data from a photoreceptor drum to a print medium may be used. Alternatively, a printer having a laser head instead of an LED head may be used as the exposure means, or a copying machine, a facsimile machine, or a digital multifunction device combining these functions may be used instead of the printer.

When printing on the print medium PM using the image forming apparatus 10 having the above-described configuration, various control parameters (that is, print setting values) are adjusted in order to obtain an optimal print result. When the present inventors verified these various control parameters, among the various control parameters, the secondary transfer voltage applied to the secondary transfer roller 44 and the toner fixing temperature of the fixing roller 51 (i.e., the toner fixing temperature of the fixing roller 51 It has been found that two types of control information, particularly the current value of the current supplied to the heater in the printer, directly affect the quality of printing. In other words, it has been found that these secondary transfer voltage and toner fixing temperature are control parameters that have a high correlation with printing defects occurring on the printing surface. When we examined the relationship between these two control parameters and printing defects in more detail, we found that high secondary transfer voltage causes dust (printing defects that result in white spots), and vice versa. If the voltage is low, it may cause blurring (printing defects where colors become lighter). In addition, if the toner fixing temperature is high, it may cause raindrops (printing defects that result in raindrop-like patterns), while if the toner fixing temperature is low, it may cause image misalignment (printing defects that occur when the toner image is fixed to the print medium). ) and other causes.

From this, it was found that in the image forming apparatus 10, by adjusting the secondary transfer voltage and toner fixing temperature, it is possible to suppress the occurrence of the above-mentioned printing defects (dust, blur, raindrops, image misalignment, etc.). .

Therefore, in the machine learning device 100 according to the present embodiment, which will be described later, the control information (that is, control parameters) to be adjusted is the secondary transfer voltage and the toner fixing temperature, and learning is performed to predict these two control information. It is now possible to generate a finished model. In other words, the machine learning device 100 uses feature amount information in the actual printed matter actually printed by the image forming device 10 (an image obtained by scanning the actual printed matter, defect characteristic information acquired from the image) and the actual A learned model is generated by supervised learning using medium information of the print medium used for the printed material and control information (secondary transfer voltage and toner fixing temperature) when the image forming apparatus 10 prints the actual printed material. It is supposed to be done.

Incidentally, in the image forming apparatus 10 having the above-described configuration, a white toner layer serving as a base is applied to a high-resistance medium such as label paper or cardboard, which requires a higher secondary transfer voltage for toner transfer than plain paper, for example. When printing a color toner layer on top, a printing defect called sinking (a printing defect in which the white toner layer rises to the top of the color toner layer and mixes colors) may occur.

For this reason, although the details will be described later, in the machine learning device 100 according to the present embodiment, printing defects such as dust, fading, raindrops, image shift, and sinking are treated as printing defects related to the secondary transfer voltage (dust, dirt, etc.). In order to distinguish between printing defects (such as blurring, etc.), printing defects related to toner fixing temperature (raindrops, image deviation, etc.), and printing defects not related to either (such as sinking), feature information (image ), medium information of the print medium used for the actual printed matter, and control information when the image forming apparatus 10 prints the actual printed matter, as well as printing defect characteristic information indicating the characteristics of the printing defect in the actual printed matter. It is now possible to generate a trained model.

[2. Configuration of machine learning device]
FIG. 2 is a schematic block diagram of machine learning device 100 in this embodiment. The machine learning device 100 includes a state variable acquisition section 110, a teacher data acquisition section 120, a learned model generation section 130, and a storage section 140. As can be understood from the above-mentioned components, the machine learning device 100 in this embodiment is a device that generates a learned model by so-called supervised learning.

As described above, the machine learning device 100 is configured to generate a learned model for adjusting the secondary transfer voltage and toner fixing temperature as control information, and more specifically, to adjust the secondary transfer voltage and toner fixing temperature. (i.e., to predict the optimal secondary transfer voltage) and a trained model to adjust the toner fusing temperature (i.e., to predict the optimal toner fusing temperature) separately. It is designed to generate. Here, the learned model for adjusting the secondary transfer voltage is called a transfer voltage prediction model, and the learned model for adjusting the toner fixing temperature is called a fixing temperature prediction model.

Note that in FIG. 2, for the purpose of easy understanding, the machine learning device 100 is shown as an example built in a computer (server device, PC, etc.) separate from the image forming device 10. The apparatus 100 may be built into the image forming apparatus 10.

The state variable acquisition unit 110 acquires parameter information necessary for generating learned models (transfer voltage prediction model and fixing temperature prediction model) as state variables. As described above, the machine learning device 100 generates the transfer voltage prediction model and the fixing temperature prediction model, and the state variable acquisition unit 110 uses the transfer voltage prediction model when generating the transfer voltage prediction model. The state variables necessary for generation are acquired, and when the fixing temperature prediction model is generated, the state variables necessary for generation of the fixing temperature prediction model are acquired. When performing machine learning, state variables are the most important element that determines the accuracy of the generated trained model. If the combination of information acquired as state variables is different, the generated trained model will naturally be different, so it is particularly important to note in the present invention that the combination is also an extremely important element.

In this embodiment, the state variable acquisition unit 110 acquires feature information in the actual printed matter AP, medium information of the printing medium PM (see FIG. 1) used in the actual printed matter AP, and printing defect characteristic information in the actual printed matter AP. and control information when outputting the actual printed matter AP (hereinafter referred to as "first control information") are acquired as state variables. The method for acquiring the state variables can be set arbitrarily depending on the connection status between the machine learning device 100 and the image forming device 10, etc., and can be set using a local or Internet communication means, or by using an arbitrary storage method. obtained through a medium. These four acquired pieces of information are then stored as one data set in the storage unit 140, which will be described later. Here, four pieces of information acquired as state variables will be explained below.

The feature amount information in the actual printed material AP is information regarding printing defects occurring in the actual printed material AP. Here, the feature information actually input to the machine learning device 100 may be data of an image obtained by reading an actual printed matter AP using a well-known scanner.

In this actual embodiment, when generating a trained model, an actual printed matter AP on which a diagnostic chart DC shown in FIG. 3 is printed is used. The diagnostic chart DC is composed of a transfer voltage prediction area AR1 and a fixing temperature prediction area AR2. The transfer voltage prediction area AR1 is, for example, an area in which a blue image with a duty of 100% (that is, a color toner image) is printed over a white image (that is, a white toner image) with a duty of 100%, and is a printing defect related to the secondary transfer voltage. When this occurs, this printing defect appears conspicuously in white on top of blue, making it easy to identify. The image obtained by scanning the portion of the transfer voltage prediction area AR1 printed on the actual print AP (that is, the image of the transfer voltage prediction area AR1, and referred to as the transfer voltage prediction image) is the feature amount information in the actual print AP. used as.

On the other hand, the fixing temperature predicted area AR2 is an area where, for example, a black image with a duty of 100% is printed over a white image with a duty of 30%, and when a printing defect related to the toner fixing temperature occurs, this printing It is easy to identify defects because they appear conspicuously in white on black, and the occurrence of sinking is suppressed as much as possible. The image obtained by scanning the portion of the fixing temperature prediction area AR2 printed on the actual printed matter AP (that is, the image of the fixing temperature predicted area AR2, and referred to as the image for fixing temperature prediction) is the feature amount information in the actual printed matter AP. used as. The image of the transfer voltage prediction area AR1 (transfer voltage prediction image) and the image of the fixing temperature prediction area AR2 (fixing temperature prediction image) are used to predict the secondary transfer voltage or toner fixing temperature. Here, the reason why the image of the fixing temperature prediction area AR2 (image for fixing temperature prediction) is also used when predicting the transfer voltage is that the printing state of the transfer voltage prediction area AR1 is also affected by changes in the fixing temperature. . Similarly, the reason why the image of the transfer voltage prediction area AR1 (transfer voltage prediction image) is also used when predicting the secondary transfer voltage is that the printing condition of the fixing temperature prediction area AR2 also affects changes in the secondary transfer voltage. It is to receive.

Note that, for example, it is of course possible to employ a preprocessing process in which the data of the above image is adjusted in advance to information suitable for inputting into the input layer of the machine learning device 100. As for the specific preprocessing process, those skilled in the art can easily understand that a method commonly used in the technical field of image recognition can be adopted, so the explanation thereof will be omitted here.

The medium information of the print medium PM is various information about the print medium PM, preferably information regarding the presence or absence of coating of the print medium PM, material, thickness, weight, density, paper color, etc. In addition, when the print medium PM is label paper, the paper thickness of the front base material (mounting part) and the back base material (label part), the type of adhesive, etc. are included. In this embodiment, for example, label paper is used as the print medium PM. This kind of media information was identified by the inventor as a parameter that particularly affects the quality of printing, and by performing machine learning based on such media information, it is possible to efficiently obtain highly accurate learned information. It is possible to generate a model. Note that the material as medium information only needs to specify the main material used in the print medium PM, and information on additionally included materials is not necessarily required. In addition, various types of weight can be used as media information as long as they indicate characteristics related to the weight of the print medium. Specifically, for example, the basis weight and ream weight of the image forming apparatus can be used. Those commonly used in the technical field can be used.

The printing defect characteristic information in the actual printed material AP is information indicating the characteristics of the printing defect in the actual printed material AP, and more specifically, the printing defect characteristic information in the transfer voltage prediction area AR1 of the diagnostic chart DC printed on the actual printed material AP. It shows the type and degree of printing defects and the type and degree of printing defects in the fixing temperature prediction area AR2. When generating the transfer voltage prediction model, the type and degree of printing defects in the transfer voltage prediction region AR1 and the degree of printing defects in the fixing temperature prediction region AR2 are used as printing defect characteristic information in the actual printed matter AP. Furthermore, when generating the fixing temperature prediction model, the type and degree of printing defects in the fixing temperature prediction area AR2 and the degree of printing defects in the transfer voltage prediction area AR1 are used as printing defect characteristic information in the actual printed matter AP.

First, the types of printing defects will be explained. There are five types of printing defects: "normal", "printing defects due to rising fixing temperature", "printing defects due to falling fixing temperature", "printing defects due to rising transfer voltage", and "printing defects due to falling transfer voltage". There is.

Among the five types, "printing defect due to increase in fixing temperature" and "printing defect due to decrease in fixing temperature" are types of printing defects in the fixing temperature prediction area AR2. That is, "printing defect due to rise in fixing temperature" indicates that a printing defect such as raindrops has occurred in the fixing temperature prediction area AR2 because the toner fixing temperature is higher than an appropriate value. Further, "printing defect due to fall in fixing temperature" indicates that a printing defect such as image shift occurs in the fixing temperature prediction area AR2 because the toner fixing temperature is lower than an appropriate value.

Further, "printing defect due to increase in transfer voltage" and "printing defect due to decrease in transfer voltage" are types of printing defects in the transfer voltage prediction area AR1. That is, "printing defect due to increase in transfer voltage" indicates that a printing defect such as dust has occurred in the transfer voltage prediction area AR1 because the secondary transfer voltage is higher than an appropriate value. Moreover, "printing defect due to transfer voltage drop" indicates that a printing defect such as blurring has occurred in the transfer voltage prediction area AR1 because the secondary transfer voltage is lower than an appropriate value.

Furthermore, "normal" has different meanings in the fixing temperature prediction area AR2 and the transfer voltage prediction area AR1. This indicates that no printing defects related to the secondary transfer voltage have occurred in the predicted area AR1.

Here, the types of printing defects in the fixing temperature prediction region AR2 will be explained using a specific example. FIGS. 4(A) to 4(D) show the actually printed fixing temperature predicted area AR2. Here, the fixing temperature prediction regions AR2 shown in FIGS. 4(A) to (D) each have different control parameters (toner fixing temperature) during printing, and are 140°C, 150°C, 160°C, and 170°C, respectively. The temperature varies by 10°C. Note that the secondary transfer voltages are all equal to 1000V.

The predicted fixing temperature area AR2 shown in FIG. 4(A) is for the case where the fixing temperature during printing is 140°C, and in this case, no printing defects related to toner fixing temperature have occurred in the predicted fixing temperature area AR2. . Therefore, in this case, the type of printing defect in the fixing temperature prediction area AR2 is "normal".

The predicted fixing temperature area AR2 shown in FIG. 4(B) is for the case where the fixing temperature during printing is 150°C, and in this case, the predicted fixing temperature area AR2 does not have any printing defects related to the toner fixing temperature. . In this case, in the fixing temperature predicted area AR2, a white dot pattern is seen in the area CL1 indicated by an oval in the figure, but this is not a printing defect related to the toner fixing temperature. Therefore, in this case, the type of printing defect in the fixing temperature prediction area AR2 is "normal".

The fixing temperature predicted area AR2 shown in FIG. A fireworks pattern can be seen in the area CL3 indicated by the oval in the figure, and a raindrop pattern can be seen in the area SQ1 indicated by the rectangle in the figure. Among these, the white spot pattern and the fireworks pattern are not printing defects related to toner fixing temperature. On the other hand, a raindrop pattern is a printing defect that occurs because the toner fixing temperature is higher than an appropriate value. Therefore, in this case, the type of printing defect in the fixing temperature prediction area AR2 is "printing defect due to rise in fixing temperature."

The fixing temperature predicted area AR2 shown in FIG. 4(D) is for the case where the fixing temperature during printing is 170°C, and in this case, the fixing temperature predicted area AR2 includes a raindrop pattern in the area SQ2 indicated by a rectangle in the figure. Can be seen. This raindrop pattern is a printing defect that occurs because the toner fixing temperature is higher than an appropriate value. Therefore, in this case as well, the type of printing defect in the fixing temperature prediction area AR2 is "printing defect due to rise in fixing temperature."

The type of printing defect in the fixing temperature prediction area AR2 can be determined using the image of the fixing temperature prediction area AR2 (that is, the fixing temperature prediction image). Various methods can be used for the determination method. For example, a prediction model generated by machine learning of various patterns of fusing temperature prediction images and the types of printing defects at that time can be used to determine the type of printing defects, or A method can be used in which a pattern image is prepared and the type of printing defect is determined by comparing the degree of similarity between the pattern image extracted from the fixing temperature prediction image and the prepared pattern image. The type of printing defect in the transfer voltage prediction area AR1 may also be determined in the same manner.

Here, as an example, a case where a printing defect related to toner fixing temperature does not occur in the fixing temperature prediction area AR2 (FIG. 4 (A), (B)), and a case where a printing defect due to an increase in toner fixing temperature occurs. The types of printing defects in the case where the printing error occurs (FIGS. 4C and 4D) have been explained. Similarly, if a printing defect due to a drop in toner fixing temperature occurs in the fixing temperature prediction area AR2, the type of printing defect is "printing defect due to a drop in fixing temperature." Furthermore, if there is no printing defect related to the secondary transfer voltage in the transfer voltage prediction area AR1, the type of printing defect will be "normal", and if a printing defect due to a drop in the secondary transfer voltage has occurred, the type of printing defect will be "normal". If a printing defect occurs due to an increase in secondary transfer voltage, it becomes a "printing defect due to an increase in transfer voltage."

As described above, the type of printing defect in the fixing temperature prediction area AR2 does not simply indicate whether a printing defect has occurred in the fixing temperature prediction area AR2, but indicates whether a printing defect related to toner fixing temperature has occurred. It is information. As will be described in detail later, by using the type of printing defect in the fixing temperature prediction area AR2 as a state variable when generating the fixing temperature prediction model, the accuracy of predicting the toner fixing temperature using the fixing temperature prediction model can be improved. can be improved.

Similarly, the type of printing defect in the transfer voltage prediction area AR1 is not simply information indicating whether a printing defect has occurred in the transfer voltage prediction area AR1, but information indicating whether a printing defect related to the secondary transfer voltage has occurred. It becomes. As will be described in detail later, by using the type of printing defect in the transfer voltage prediction area AR1 as a state variable when generating the transfer voltage prediction model, the secondary transfer voltage can be predicted using the transfer voltage prediction model. Accuracy can be improved.

Next, the degree of printing defects included in the printing defect characteristic information will be explained. The degree of printing defects is expressed, for example, by a parameter called a PQ value. The calculation method for this PQ value differs between the transfer voltage prediction area AR1 and the fixing temperature prediction area AR2.

In calculating the PQ value in the transfer voltage prediction area AR1, the feature amount of the co-occurrence matrix is extracted from the image of the transfer voltage prediction area AR1 (that is, the image for transfer voltage prediction), and the Maharabinos distance is calculated based on the extracted feature amount. . The calculated Maharabinos distance is then set as the PQ value in the transfer voltage prediction region AR1.

On the other hand, in calculating the PQ value in the fixing temperature prediction area AR2, by performing general-purpose binarization processing on the image in the fixing temperature prediction area AR2 (that is, the image for fixing temperature prediction), the portions that are defective in printing are (white) and other parts (black). Then, the PQ value is determined by dividing the number of pixels in the defective portion by the total number of pixels in the fixing temperature prediction image.

In this embodiment, the PQ value is used as a parameter indicating the degree of printing defects, but this is not limited to this. Any parameter that can indicate the degree of printing defects may be calculated using a calculation method different from the PQ value. Other parameters may also be used. This completes the description of the types and degrees of printing defects included in the printing defect characteristic information in the actual printed matter AP.

The first control information is the control parameters that were actually set when the actual printed matter AP was output, and specifically, the toner fixing temperature of the fixing roller 51 and the secondary transfer roller 44 in the image forming apparatus 10. This is the applied secondary transfer voltage. Note that when generating the transfer voltage prediction model, the secondary transfer voltage is used as the first control information (the toner fixing temperature is also used as the first control information), and when generating the fixing temperature prediction model, the toner fixing temperature is used as the first control information. It is used as control information (secondary transfer voltage is also used as first control information). Further, in the image forming apparatus 10, since the toner image is transferred from the intermediate transfer belt 41 to the printing medium PM, the secondary transfer voltage is adopted as the first control information. In a tandem image forming apparatus that directly transfers a toner image from a photoreceptor drum to a print medium without having a toner image, when transferring a toner image from a photoreceptor drum provided in each of a plurality of image forming units with different toner colors to the print medium PM. The transfer voltage may be employed as the first control information. Four pieces of information acquired as state variables (feature information in the actual printed matter AP, medium information of the printing medium PM used in the actual printed matter AP, printing defect characteristic information in the actual printed matter AP, 1 control information) is described above.

Returning to FIG. 2, the teacher data acquisition unit 120 obtains control information, in short, related to control information, in which feature information, which is information about printing defects, is within a predetermined threshold in an actual printed matter AP that includes printing defects. Improved control information (hereinafter referred to as second control information) that allows printing results free of printing defects to be obtained is acquired as teacher data.

This second control information is, for example, control information (that is, toner fixing temperature and secondary transfer voltage) derived by engineer EN based on the output result of the actual printed matter AP and the control information at that time (that is, toner fixing temperature and secondary transfer voltage). (next transfer voltage). Therefore, the above-mentioned "predetermined threshold value" does not necessarily have to refer to a specific value, and as long as the control information is such that an appropriate output result can be obtained from the perspective of an engineer, etc., the said control information is " It can be said that the feature amount information is within a predetermined threshold. Further, this second control information is acquired by the engineer EN by directly inputting it into the teacher data acquisition unit or by transmitting it via various communication means or arbitrary storage media. Note that when generating the transfer voltage prediction model, the secondary transfer voltage is acquired as the second control information, and when generating the fixing temperature prediction model, the toner fixing temperature is acquired as the second control information. When the secondary transfer voltage is acquired as the second control information, the predetermined threshold value is the secondary transfer voltage or lower (5000 V or lower in the present invention) at which printing defects due to increased secondary transfer voltage do not occur in the actual printed matter. , when the toner fixing temperature is acquired as the second control information, the predetermined threshold value or less is the fixing temperature below (155° C. or less in the present invention) at which printing defects due to an increase in the fixing temperature do not occur in actual printed matter. This second control information is stored in the storage unit 140 in association with the corresponding data set acquired by the state variable acquisition unit 110 and stored in the storage unit 140 as a data set.

The trained model generation unit 130 generates a model based on a data set consisting of four pieces of information as state variables acquired by the state variable acquisition unit 110 and teacher data acquired by the teacher data acquisition unit 120 and associated with the corresponding data set. Then, machine learning is executed and a trained model is generated. The specific method of machine learning carried out here will be described in detail below.

The machine learning device 100 according to the present invention employs supervised learning using a neural network model as its learning method.

FIG. 5 shows an example of a neural network model for supervised learning performed in the trained model generation unit 130. The neural network NN in the neural network model includes k neurons x (neurons x ₁ to x _k ) in the input layer LX, m neurons y1 (neurons y1 ₁ to y1 _m ) in the first hidden layer LY1, and It has n neurons y2 (neurons y2 ₁ to y2 _n ) in two hidden layers LY2 and one neuron z (neuron z ₁ ) in the output layer LZ. In this example, two intermediate layers are provided, but the present invention is not limited to this, and three or more intermediate layers may be provided, or one intermediate layer may be provided.

Nodes for connecting neurons are provided between the input layer LX and the first hidden layer LY1, between the first hidden layer LY1 and the second hidden layer LY2, and between the second hidden layer LY2 and the output layer LZ. Each node is associated with a weight wj (j is a natural number).

The neural network in the neural network model in this embodiment uses a learning data set to learn the correlation between the control information of the image forming apparatus and the printed matter output by the image forming apparatus.

Specifically, when generating the fixing temperature prediction model, the learned model generation unit 130 uses feature information (fixing temperature prediction image, transfer voltage prediction image), medium information, and print defects in the actual printed matter AP as state variables. The characteristic information (the type and degree of printing defects in the fixing temperature prediction area AR2, and the degree of printing defects in the transfer voltage prediction area AR1) and the first control information (toner fixing temperature, secondary transfer voltage) are transmitted to a plurality of input layers LX. By associating it with neuron x, the value of neuron z in the output layer LZ is calculated. That is, the trained model generation unit 130 first calculates the values of the m neurons y1 of the first hidden layer LY1 based on the values of the k neurons x of the input layer LX. Specifically, the learned model generation unit 130 converts the value of each neuron y1 of the first intermediate layer LY1 into each one based on the values of k neurons x of the input layer LX connected to this neuron y1. It is calculated by performing weighted addition using the weight Wi associated with the node. Similarly, the trained model generation unit 130 calculates the values of the n neurons y2 of the second hidden layer LY2 based on the values of the m neurons y1 of the first hidden layer LY1, and calculates the values of the n neurons y2 of the second hidden layer LY2. The value of the neuron z in the output layer LZ is calculated based on the values of the n neurons y2.

Then, the learned model generation unit 130 compares the calculated value of the neuron z ₁ of the output layer LZ with the value of the data t ₁ included in the teacher data DT to find an error. Here, the value of neuron z ₁ includes feature information (fixing temperature prediction image, transfer voltage prediction image) in the actual printed matter AP as a state variable, medium information, printing defect characteristic information (printing in the fixing temperature prediction area AR2). control information (toner fixing temperature, secondary transfer voltage) calculated based on the first control information (toner fixing temperature, secondary transfer voltage) ), and the value of data _t1 is the second control information (toner fixing temperature, secondary transfer voltage) in the teacher data DT. Then, the learned model generating unit 130 repeatedly adjusts the weight wi associated with each node (backpropriation) so that the obtained error becomes smaller.

If the series of steps described above is repeated a predetermined number of times, or until a predetermined condition such as the error described above is smaller than an allowable value is met, then the trained model generation unit 130 completes the learning and stores the neural network model in the storage unit 140 as a fixing temperature prediction model. In this way, the learned model generation unit 130 generates a fixing temperature prediction model that includes information about all the weights wj associated with each node of the neural network model. Here, as an example, generation of a fixing temperature prediction model using a neural network has been described, but a transfer voltage prediction model can be similarly generated using a neural network.

The learned models (fixing temperature prediction model and transfer voltage prediction model) stored in the storage unit 140 are applied to the actual system via a communication means such as the Internet or a storage medium according to a request. A specific example of application of the learned model to an actual system (data processing system) will be described later.

[3. Printing defect type determination method and machine learning method]
Next, a method of determining the type of printing defect and a specific procedure of the machine learning method in this embodiment will be described. Note that the determination of the type of printing defect is performed, for example, by the state variable acquisition unit 110 of the machine learning device 100, and the machine learning is performed, for example, by the trained model generation unit 130 of the machine learning device 100. It is.

First, the specific procedure of the method for determining the type of printing defect will be described using the flowchart shown in FIG. Note that the method for determining the type of printing defect in the image for fusing temperature prediction and the method for determining the type of printing defect in the image for predicting transfer voltage are basically the same, so here, the method for determining the type of printing defect in the image for fusing temperature prediction will be explained. The procedure for determining the type of defect will be explained.

In the first step SP1, defective locations are identified in the fixing temperature prediction image using a preprocessing method such as binarization, and several rectangular regions (referred to as area images) including the defective locations are extracted. In the following step SP2, the pixel value of each pixel is normalized for each of the extracted region images. In the following step SP3, the region image whose pixel values have been normalized is input to a prediction model generated by machine learning. When a region image is input, this prediction model predicts the type of printing defect in the region image (either "normal", "printing defect due to increase in fixing temperature", or "printing defect due to decrease in fixing temperature"). This is what is output as. In the following step SP4, the prediction result (that is, the type of printing defect) output from the prediction model is obtained. Note that the processes of step SP3 and step SP4 are performed in order for all area images extracted from the fixing temperature prediction image.

In the following step SP5, the prediction results obtained from the prediction models (that is, the types of printing defects predicted for each region image) are integrated. Specifically, among the types of printing defects predicted for each region image, the type with the largest number is determined to be the type of printing defects in the fixing temperature prediction image. Here, among the types of printing defects predicted for each region image, the type with the largest number is determined as the type of printing defects in the fixing temperature prediction image. If all of the types of printing defects that have been detected are "normal," the types of printing defects in the image for fusing temperature prediction are determined to be "normal," and the types of printing defects predicted for each region image are determined to be "normal." If at least one of "Printing defects due to rise in temperature" and "Printing defects due to decrease in fixing temperature" is included, either "Printing defects due to increase in fixing temperature" or "Printing defects due to decrease in fixing temperature" are included. The one with a larger number may be determined as the type of printing defect in the fixing temperature prediction image. The specific procedure of the method for determining the type of printing defect in the fixing temperature prediction image has been described above.

Next, the procedure of the machine learning method will be explained using the flowchart shown in FIG. Note that the machine learning method for generating a fusing temperature prediction model and the machine learning method for generating a transfer voltage prediction model have the same basic procedures, so here we will explain the machine learning method for generating a fusing temperature prediction model. Explain the procedure.

In step SP11, first, a pre-learning model with initial value weights is prepared. It is assumed that this pre-learning model is stored in the storage unit 140 in advance, for example. In the subsequent step SP12, the trained model generation unit 130 generates feature information (fixing temperature prediction image and transfer voltage prediction image) in the actual printed matter AP as state variables, medium information, printing defect characteristic information (fixing temperature prediction area The type and degree of printing defects in AR2 and the degree of printing defects in transfer voltage prediction area AR1) and control information during printing (first control information, toner fixing temperature and secondary transfer voltage) are acquired. In the following step SP13, the trained model generation unit 130 generates teacher data TD (second control information and control information that allows normal printing (toner fixing temperature and secondary transfer voltage)) corresponding to the state variables acquired in step SP12. get.

In the following step SP14, the state variables acquired in step SP12 are input to the neural network model NM, and the output (control information) of the neural network model NM is generated. In the following step SP15, the trained model generation unit 130 performs machine learning based on the output (control information) of the neural network model NM and the teacher data TD (that is, control information that allows normal printing). In the following step SP16, the trained model generation unit 130 determines whether the conditions for terminating machine learning are satisfied, and if the conditions are not met, the process returns to step SP12, and if the conditions are met, the process returns to step SP17. Move to. In step SP17, the learned model generation section 130 stores the neural network model NM at this time in the storage section 140 as a fixing temperature prediction model. This completes the description of the specific steps of the machine learning method for generating the fixing temperature prediction model.

[4. Example of application of trained model]
[4-1. Configuration of data processing system applying trained model]
Next, an application example of the learned models (fixing temperature prediction model and transfer voltage prediction model) generated by the machine learning device 100 and the machine learning method according to the present embodiment will be described. FIG. 8 shows a functional configuration of a data processing system 200 to which a machine learning device 100 and a learned model generated by a machine learning method are applied.

This data processing system 200 includes an image forming device 210, a reading device 220, and an image diagnostic device 230. The image forming device 210, the reading device 220, and the image diagnostic device 230 are connected via a predetermined network Nt.

The image forming apparatus 210 is a so-called intermediate transfer type color printer that has a printing mechanism similar to that of the image forming apparatus 10 shown in FIG. Therefore, description of the printing mechanism of the image forming apparatus 210 will be omitted here as appropriate.

The image forming apparatus 210 includes a user interface 211, a control section 212, a storage section 213, and a printing section 214 in addition to the same printing mechanism as the image forming apparatus 10 (see FIG. 1). Further, as shown in FIG. 9, the image forming apparatus 210 has an operation panel 301, an operation key 302, and a start button 303 on the front side of the upper surface of a box-shaped housing 300. It has a manual feed tray 304 on the front side.

The user interface 211 (see FIG. 8) of the image forming apparatus 210 collects print setting values input via the operation panel 301 and sends them to the control unit 212. Further, the user interface 211 receives a screen display command from the control unit 212, and according to the screen display command, acquires various screen data and various message data from the storage unit 213, and causes the operation panel 301 to display various screens and various messages. .

The control unit 212 corresponds to the control unit 70 of the printing mechanism shown in FIG. 1, and controls the operation of the image forming apparatus 210. The storage unit 213 temporarily stores various data such as print data used during the operation of the image forming apparatus 210. Further, the storage unit 213 stores, for example, print data of a diagnostic chart DC shown in FIG. 3. The printing unit 214 receives a print command from the control unit 212, receives print data from the storage unit 213 in accordance with the print command, and executes printing based on the print data. That is, the printing unit 214 prints a print image based on print data on a print medium (for example, label paper) using a print mechanism similar to the print mechanism shown in FIG. 1, and outputs the print medium as an actual printed matter. This printing unit 214 prints, for example, a diagnostic chart DC on a print medium.

The operation panel 301 has a touch panel function and functions as a display section and an operation section. This operation panel 301 displays a menu screen, a user guide screen, instruction messages, and the like. The operation keys 302 are auxiliary keys for operating various screens displayed on the operation panel 301. A start button 303 is a button for starting printing or determining an operation. The manual feed tray 304 corresponds to the manual feed tray 22 of the printing mechanism shown in FIG. 1, and a special print medium (for example, label paper) that is different in size and shape from ordinary paper is set therein.

The reading device 220 is a so-called scanner, and includes a document reading section 221. For example, when a print medium (that is, an actual printed matter) on which a diagnostic chart DC is printed by the image forming apparatus 210 is set on the document table, the original reading unit 221 reads the diagnostic chart DC printed on the actual printed material and converts it into an image. Convert. Note that here, a scanner (reading device 220) is used to read the diagnostic chart DC printed on a print medium and convert it into an image, but the invention is not limited to this. The diagnostic chart DC printed on the medium may be photographed and converted into an image.

The image diagnostic apparatus 230 is a computer such as a so-called PC (personal computer) or a server, and includes an image information extraction section 231, an image feature extraction section 232, a medium information input section 233, a control information input section 234, and an AI diagnosis model 235. The image information extraction unit 231 receives the image of the diagnostic chart DC read by the reading device 220, extracts the image of the transfer voltage prediction area AR1 and the image of the fixing temperature prediction area AR2 from the image, and converts these images into , and sent to the image feature extraction unit 232 and the AI diagnostic model 235 as an image for predicting transfer voltage and an image for predicting fixing temperature.

The image feature extraction unit 232 receives the transfer voltage prediction image and the fixing temperature prediction image from the image information extraction unit 231, and extracts feature amounts from each image. Note that the feature amount extracted by the image feature extraction unit 232 is the characteristic of the printing defect (type and degree of the printing defect). That is, the image feature extraction unit 232 extracts the type and degree of printing defects in the transfer voltage prediction area AR1 from the transfer voltage prediction image, and extracts the type and degree of printing defects in the fixing temperature prediction area AR2 from the fixing temperature prediction image. Extract. Then, the image feature extraction unit 232 extracts the printing defect characteristic information indicating the type and degree of the printing defect in the transfer voltage prediction area AR1 and the printing defect characteristic information indicating the type and degree of the printing defect in the fixing temperature prediction area AR2. to the diagnostic model 235.

The medium information input unit 233 acquires medium information input via the user interface 211 of the image forming apparatus 210 from the image forming apparatus 210 and sends it to the AI diagnostic model 235. Note that this medium information is medium information of the print medium used by the image forming apparatus 210 to print the diagnostic chart DC. Specifically, as described above, the information includes, for example, the presence or absence of a coating on the print medium, its material, thickness, weight, density, paper color, and the like. Further, if the print medium is label paper, the medium information includes the paper thickness of the front base material (mounting part) and the back base material (label part), the type of adhesive, etc. The control information input unit 234 acquires control information (that is, the toner fixing temperature and secondary transfer voltage set when printing the diagnostic chart DC) from the image forming apparatus 210 and sends this to the AI diagnostic model 235.

The AI diagnostic model 235 includes a fixing temperature prediction model 236 generated by the machine learning device 100 described above, a transfer voltage prediction model 237, and a rounding processing section 238. Note that the rounding processing unit 238 may be provided separately from the AI diagnostic model 235. The AI diagnostic model 235 inputs various data received from each part into an appropriate prediction model, predicts appropriate control information in the image forming apparatus 210 using the prediction model, and further rounding this control information. The prediction accuracy has been improved, making it possible to predict more appropriate (that is, optimal) control information.

Specifically, the AI diagnostic model 235 uses the fixing temperature prediction image and transfer voltage prediction image received from the image information extraction unit 231 and the type of printing defect in the fixing temperature prediction area AR2 received from the image feature extraction unit 232. printing defect characteristic information indicating the extent of the printing defect in the transfer voltage prediction area AR1; the medium information received from the medium information input section 233; and the toner fixing temperature and temperature included in the control information received from the control information input section 234. The secondary transfer voltage is input to the fixing temperature prediction model 236. As a result, an appropriate toner fixing temperature for the image forming apparatus 210 is output from the fixing temperature prediction model 236 (the secondary transfer voltage is also output, but not adopted).

Furthermore, the AI diagnostic model 235 rounds the toner fixing temperature output from the fixing temperature prediction model 236 using a rounding processing unit 238 . In this case, the rounding process involves replacing a continuous value such as toner fixing temperature or secondary transfer voltage with one of discrete values at appropriate predetermined intervals. In the case of the toner fixing temperature, discrete values at appropriate intervals are set to values in 5°C increments, such as 130°C, 135°C, 140°C, 145°C, . . . . On the other hand, in the case of the secondary transfer voltage, if the settable range of the secondary transfer voltage is 1000V to 5000V, the discrete values at appropriate intervals are 1000V, 1200V, 1400V, etc. in the range of 1000V to 3500V. In the range from 3500V to 5000V, the values are set in 500V increments such as 3500V, 4000V, 4500V, etc. It should be noted that the above-mentioned intervals of discrete values are just an example, and are set as intervals at which a noticeable change appears in the print result when the toner fixing temperature or the secondary transfer voltage is changed. In other words, for example, in the case of toner fixing temperature, if you change it by 5 degrees Celsius, there will be a noticeable change in the print results before and after the change, so the discrete value will change to a value in 5 degrees Celsius increments. is set to .

Therefore, in the rounding processing unit 238, for example, if the toner fixing temperature output from the fixing temperature prediction model 236 is 138.3°C, the 138.3°C is set as a discrete value by rounding processing. Between 135°C and 140°C, the temperature is replaced with 140°C, which is closer to 138.3°C. In this way, the AI diagnostic model 235 improves the prediction accuracy of the toner fixing temperature and can predict a more appropriate (that is, optimal) toner fixing temperature. Note that the reason why the prediction accuracy is improved by the rounding process will be described in detail later.

Similarly, the AI diagnostic model 235 uses the transfer voltage prediction image and fixing temperature prediction image received from the image information extraction unit 231 and the type of printing defect in the transfer voltage prediction area AR1 received from the image feature extraction unit 232. printing defect characteristic information indicating the degree of printing defect in the predicted fixing temperature region AR2, the medium information received from the medium information input section 233, and the secondary transfer voltage and voltage included in the control information received from the control information input section 234. The toner fixing temperature is input into the transfer voltage prediction model 237. As a result, an appropriate secondary transfer voltage for the image forming apparatus 210 is output from the transfer voltage prediction model 237 (the toner fixing temperature is also output, but not adopted). Furthermore, the AI diagnostic model 235 rounds the secondary transfer voltage output from the transfer voltage prediction model 237 using a rounding processing unit 238 .

The control information (toner fixing temperature and secondary transfer voltage after rounding processing) predicted by the AI diagnostic model 235 in this way is output from the AI diagnostic model 235 and sent to the image forming apparatus 210. After being stored in the storage unit 213 of the image forming apparatus 213, the control parameters (toner fixing temperature and secondary transfer voltage) set in the image forming apparatus 210 are predicted by the AI diagnostic model 235. control information (toner fixing temperature and secondary transfer voltage after rounding). Thereby, the image forming apparatus 210 can thereafter output highly accurate printed images with less printing defects.

Note that although the data processing system 200 in this embodiment includes the image forming device 210, the reading device 220, and the image diagnostic device 230 as separate devices, the present invention is not limited to this, and for example, the image diagnostic device By providing the functions of 230 in the image forming device 210, by providing the reading device 220 in the image forming device 210 to make it a multifunction device, and by providing the function of the image diagnostic device 230 in the multifunction device, it can be read and read by the image forming device 210. The functions of the device 220 and the image diagnostic device 230 may be combined into one device.

[4-2. Operation of data processing system applying trained model]
Next, the operation procedure of the data processing system 200 described above, that is, the operation procedure when predicting appropriate control information in the image forming apparatus 210 and reflecting it on the image forming apparatus 210 will be explained using the flowchart shown in FIG. 10. .

In the first step SP31, when the user selects AI automatic adjustment from the menu screen via the operation panel 301, the control unit 212 of the image forming apparatus 210 obtains a positive result in step SP31 and moves to step SP32. . At this time, the control unit 212 of the image forming apparatus 210 sends an AI automatic adjustment start command to the image diagnostic apparatus 230. On the other hand, if AI automatic adjustment is not selected, the process does not proceed to step SP32. Note that AI automatic adjustment means to predict appropriate control information (toner fixing temperature and secondary transfer voltage) for the image forming apparatus 210 using the AI diagnostic model 235 and automatically adjust the control information (i.e., control parameters) for the image forming apparatus 210. This refers to the adjustment function. Further, in this embodiment, the user operates the operation panel 301 of the image forming apparatus 210 to select AI automatic adjustment, but the invention is not limited to this. A menu screen of the image forming apparatus 210 may be displayed on the communication terminal by accessing the forming apparatus 210, and AI automatic adjustment may be selected from the menu screen.

In step SP32, the control unit 212 of the image forming apparatus 210 displays a medium information input screen on the operation panel 301, and allows the user to input medium information necessary for AI automatic adjustment on the medium information input screen, Thereafter, when the user presses the send button displayed on the medium information input screen, the input medium information is sent to the image diagnostic apparatus 230.

In the following step SP33, the control unit 212 of the image forming apparatus 210 causes the operation panel 301 to display a print screen for printing the diagnostic chart DC, and the print execution button displayed on the print screen is pressed by the user. When operated, the printing unit 214 is controlled to print the diagnostic chart DC. Thereby, the image forming apparatus 210 discharges the actual printed matter on which the diagnostic chart DC is printed. At this time, the control unit 212 of the image forming apparatus 210 sends the control parameters (toner fixing temperature and secondary transfer voltage) used when printing the diagnostic chart DC to the image diagnostic apparatus 230 as control information.

In the following step SP34, when the actual printed matter on which the diagnostic chart DC is printed is set by the user on the document table, the reading device 220 reads this with the document reading unit 221 and converts it into an image (that is, an image of the diagnostic chart DC). do. The reading device 220 then sends this image to the image diagnostic device 230.

In the following step SP35, the image diagnostic device 230, in accordance with the AI automatic adjustment start command sent from the image forming device 210, uses the AI diagnostic model 235 based on the image, control information, and medium information of the sent diagnostic chart DC. The optimum control information (toner fixing temperature and secondary transfer voltage) for the image forming apparatus 210 is predicted, and the control information obtained as the prediction result is sent to the image forming apparatus 210. Note that the process of step SP35 will be described in detail later using the flowchart shown in FIG.

In the following step SP36, the control unit 212 of the image forming device 210 receives the control information (toner fixing temperature and secondary transfer voltage) sent from the image diagnostic device 230, stores it in the storage unit 213, and then prints the control information. The control parameters (toner fixing temperature and secondary transfer voltage) set in section 214 are updated using the received control information. After this, the image forming apparatus 210 executes printing using the updated control parameters, that is, the optimal control information predicted by the AI diagnostic model 235.

Next, using the flowchart shown in FIG. 11, the procedure of the above-mentioned step SP35 (that is, prediction of appropriate control information by the image diagnostic apparatus 230) will be described.

Upon receiving the AI automatic adjustment start command from the image forming apparatus 210, the image diagnostic apparatus 230 sends an operation start command to the image information extraction section 231, medium information input section 233, and control information input section 234, respectively. .

Then, in step SP41, the image information extraction unit 231 extracts a prediction image (that is, a transfer voltage prediction image that is an image of the transfer voltage prediction area AR1 and a fixing temperature prediction image) from the image of the diagnostic chart DC sent from the reading device 220. The fixing temperature prediction image (which is the image of the area AR2) is extracted and sent to the image feature extraction unit 232 and the AI diagnostic model 235.

In step SP42, the image feature extraction unit 232 extracts the type and degree of printing defects (PQ value) as feature quantities from each of the transfer voltage prediction image and fixing temperature prediction image sent from the image information extraction unit 231. and sends it to the AI diagnostic model 235 as respective printing defect characteristic information.

In step SP43, the medium information input unit 233 acquires the medium information sent from the image forming apparatus 210 and sends it to the AI diagnostic model 235. The control information input unit 234 also acquires control information (secondary transfer voltage and toner fixing temperature) sent from the image forming apparatus 210 and sends it to the AI diagnostic model 235 .

In step SP44, the AI diagnostic model 235 uses the transfer voltage prediction image and the fixing temperature prediction image sent from the image information extraction unit 231, and the printing in the transfer voltage prediction area AR1 sent from the image feature extraction unit 232. Printing defect characteristic information indicating the type and degree of the defect and the degree of printing defect in the fixing temperature prediction region AR2, the medium information sent from the medium information input section 233, and the secondary information sent from the control information input section 234. The transfer voltage and toner fixing temperature are input into the transfer voltage prediction model 237. As a result, the transfer voltage prediction model 237 outputs an appropriate secondary transfer voltage for the image forming apparatus 210. Furthermore, the AI diagnostic model 235 predicts the optimal secondary transfer voltage by rounding the secondary transfer voltage output from the transfer voltage prediction model 237 using a rounding processing unit 238 .

On the other hand, the AI diagnostic model 235 uses the fixing temperature prediction image and transfer voltage prediction image sent from the image information extraction unit 231 and the printing in the fixing temperature prediction area AR2 sent from the image feature extraction unit 232. Print defect characteristic information indicating the type and degree of the defect and the degree of printing defect in the transfer voltage prediction area AR1, medium information sent from the medium information input section 233, and toner fixation information sent from the control information input section 234. The temperature and secondary transfer voltage are input into the fixing temperature prediction model 236. As a result, the fixing temperature prediction model 236 outputs an appropriate toner fixing temperature in the image forming apparatus 210. Further, the AI diagnostic model 235 predicts the optimum toner fixing temperature by rounding the toner fixing temperature output from the fixing temperature prediction model 236 using a rounding processing unit 238 .

The operation procedure of the data processing system 200, that is, the operation procedure for predicting appropriate control information in the image forming apparatus 210 and reflecting it on the image forming apparatus 210 has been described above.

[5. Summary and effects]
As described above, in the present embodiment, the machine learning device 100 includes the image forming unit 31W as the first image forming section that forms a white toner image and the second image forming section that forms the color toner image. Feature amount information (image for transfer voltage prediction and fixing temperature prediction image), printing defect characteristic information indicating the type and degree (PQ value) of printing defects included in the actual printed material AP, medium information of the printing medium PM used in the actual printed material AP, and the image forming apparatus 10 A state variable acquisition unit 110 is provided that acquires first control information (secondary transfer voltage and toner fixing temperature), which is control information when printing an AP, as a state variable. Furthermore, the machine learning device 100 includes a teacher data acquisition unit 120 that acquires second control information (secondary transfer voltage and toner fixing temperature) that makes the feature amount information in the actual printed matter AP within a predetermined threshold value as the teacher data, and a state variable A trained model that generates a trained model (transfer voltage prediction model or fixing temperature prediction model) by performing machine learning based on the state variables acquired by the acquisition unit 110 and the teacher data acquired by the teacher data acquisition unit 120. A generation unit 130 is provided.

Further, in the present embodiment, the data processing system 200, which is an application example of the learned model generated by the machine learning device 100, includes an image forming unit 31W as a first image forming section that forms a white toner image and a color toner image forming unit 31W as a first image forming section that forms a white toner image. Feature amount information ( a transfer voltage prediction image and a fixing temperature prediction image), printing defect characteristic information indicating the type and degree (PQ value) of printing defects included in the actual printed matter, and medium information of the printing medium used in the actual printed matter; An image information extraction unit 231 as an actual printed matter information acquisition unit that acquires first control information (secondary transfer voltage and toner fixing temperature) that is control information when the image forming apparatus 210 prints an actual printed matter, and image feature extraction. 232, medium information input section 233, and control information input section 234, and the information acquired by these image information extraction section 231, image feature extraction section 232, medium information input section 233, and control information input section 234. AI diagnostic model as a data processing unit that inputs into the learned model (transfer voltage prediction model or fixing temperature prediction model) generated by the learning device 100 and outputs third control information (secondary transfer voltage and toner fixing temperature) 235, and a storage unit 213 as a third control information storage unit that stores the third control information output from the AI diagnostic model 235.

As described above, in this embodiment, the state variables include the feature amount information (transfer voltage prediction image and fixing temperature prediction image) of the actual printed matter, the medium information of the print medium used for the actual printed matter, and the actual printed matter. In addition to the primary control information (secondary transfer voltage and toner fixing temperature) during printing, machine learning is performed using printing defect characteristic information that indicates the type of printing defects included in the actual printed matter. A highly desired learned model can be generated, and the prediction accuracy of the control information (that is, the third control information) can be improved using the learned model.

In other words, in the electrophotographic image forming apparatus 210, when label paper is used, the margin of the secondary transfer voltage becomes narrow, so for example, a color toner layer is overlaid on a white toner layer as a base. When printing, sinking may occur. Conventionally, it is not possible to distinguish what kind of printing defect is occurring, so when a printing defect that is not related to control information, such as sinking, occurs, the learned model is affected by this printing defect and The prediction accuracy of the output control information will decrease. In contrast, in this embodiment, by adding the types of printing defects included in actual printed matter to the state variables, a learned model is generated in which the influence of printing defects that are not related to control information is reduced. Therefore, it is possible to generate a trained model with higher accuracy compared to the conventional method, and the prediction accuracy of control information (that is, third control information) can be improved using the trained model. .

That is, according to the present embodiment, the state variable for generating the fixing temperature prediction model includes the type of printing defect ("normal", "fixing temperature rise" included in the fixing temperature prediction area AR2 printed on the actual printed material). By adding "Printing defects due to fusing temperature" or "Printing defects due to decrease in fixing temperature"), machine learning can be performed to distinguish whether printing defects related to fixing temperature have occurred in the fixing temperature prediction area AR2. can. By doing this, in this embodiment, it is possible to generate a fixing temperature prediction model in which the influence of printing defects unrelated to toner fixing temperature is reduced, so that a desired prediction model with higher precision can be generated compared to the conventional method. A fixing temperature prediction model can be generated, and the accuracy of predicting toner fixing temperature can be improved using the fixing temperature prediction model.

Similarly, according to the present embodiment, it is possible to generate a desired transfer voltage prediction model with higher accuracy than in the past, and to use the transfer voltage prediction model to improve the prediction accuracy of the secondary transfer voltage. can be improved.

Furthermore, in this embodiment, by adding the degree of printing defects (PQ value) in addition to the type of printing defects to the state variable, the accuracy is higher than when only the type of printing defects is added. A desired learned model can be generated, and the prediction accuracy of control information can be further improved using the learned model.

Furthermore, in the present embodiment, the AI diagnostic model 235 is provided with a rounding unit 238 that rounds the control information predicted by the learned model (transfer voltage prediction model or fixing temperature prediction model), It is now output as the third control information. By doing so, according to the present embodiment, the prediction accuracy of the third control information can be improved compared to the case where rounding processing is not performed.

The graph in FIG. 12 shows how much the prediction accuracy of control information actually improves. In the graph of FIG. 12, the left side shows the predicted result of the toner fixing temperature, and the right side shows the predicted result of the secondary transfer voltage. In addition, the horizontal axis of this graph shows the methods to be compared, and the vertical axis shows the improvement range achievement rate of the prediction results. Here, the improvement range achievement rate represents the proportion of verification data whose predicted results fall within the improvement range among all verification data.

In other words, for example, if the appropriate improvement range of toner fixing temperature that does not cause printing defects is 120°C to 140°C, the achievement rate of the improvement range of toner fixing temperature is that the predicted result is 120°C out of all the verification data. Represents the percentage of verification data that falls within the range of ~140°C. Therefore, the toner fixing temperature improvement range achievement rate is determined by predicting the toner fixing temperature a predetermined number of times, and how many times out of the predetermined times the predicted toner fixing temperature falls within the improvement range (120°C to 140°C). It can be calculated by checking. The improvement range achievement rate of the secondary transfer voltage can also be calculated in the same manner.

The left side of the graph in Figure 12 shows the prediction results when toner fixing temperature is predicted using three different methods, and the right side shows the prediction results when the secondary transfer voltage is predicted using three different methods. Showing results. Of the three different methods, the first method (on the left) is the same as the conventional method, which predicts control information by inputting the image of the actual printed material, media information, and control information into a trained model. It is a method. The second (center) method is a method in which rounding processing is added to the first method. In other words, the second method is a method of inputting an image of an actual printed material, medium information, and control information into a learned model and rounding the obtained control information. The third method is the method of the present embodiment, which is obtained by inputting the type and degree of printing defects included in the actual printed material in addition to the image of the actual printed material, medium information, and control information into a trained model. This is a method of rounding the received control information.

As is clear from this graph, the achievement rate of the improved range of toner fixing temperature by the first method is 72.8%, and the achievement rate of the improved range of toner fixing temperature by the second method is 87.8%. %, and the achievement rate of the improvement range of the toner fixing temperature by the third method, which is the method of this embodiment, is 91.9%. The verification results show that the prediction accuracy when predicting toner fixing temperature using the method of this embodiment is higher than when using other methods.

Similarly, the achievement rate of the improvement range of the secondary transfer voltage by the first method was 59.9%, and the achievement rate of the improvement range of the secondary transfer voltage by the second method was 91.5%. The achievement rate of the improvement range of the secondary transfer voltage by the third method, which is a method of the present invention, is 95.5%. This verification result shows that the prediction accuracy when predicting the secondary transfer voltage using the method of this embodiment is higher than when using other methods.

Also, here, we will show the prediction results when control information is predicted using the first method, which is a conventional method, and the prediction result when control information is predicted using the third method, which is the method of this embodiment. The prediction results are shown in the graph of FIG. In this graph, the horizontal axis represents toner fixing temperature, and the vertical axis represents secondary transfer voltage. Further, here, it is assumed that the improvement range of the toner fixing temperature is 140° C. to 155° C., and the improvement range of the secondary transfer voltage is 3600V to 5000V. The circles shown in the graph of FIG. 13 indicate the prediction results when the toner fixing temperature is predicted using the conventional method, and the triangle marks indicate the prediction results when the toner fixing temperature is predicted using the method of this embodiment. The prediction results are shown. Note that the prediction result shown in FIG. 13 when control information is predicted using the method of this embodiment is the prediction result before rounding processing is performed.

From this graph, it can be seen that even before rounding (that is, without rounding), the prediction result of predicting control information using the method of this embodiment is different from the prediction result of predicting control information using the conventional method. It can be seen that the improvement range achievement rate is higher and the prediction accuracy is higher than the prediction result when predicting.

Here, the reason why the prediction accuracy of control information is improved by rounding processing will be briefly explained. As shown in the graph of FIG. 13, the prediction result (that is, the prediction result indicated by the triangle mark) when the toner fixing temperature is predicted using the method of this embodiment before the rounding process is performed is within the improvement range. It appears concentrated around 140°C, which is the minimum value of . Therefore, when the prediction results at this time are rounded, the prediction results that were slightly lower than 140°C (the area surrounded by the dotted line in the figure) will be raised to 140°C and fall into the improved range. , the prediction accuracy is further improved. In other words, the rounding process allows prediction results that are slightly outside the improvement range to be included in the improvement range, thereby increasing the improvement range achievement rate and improving prediction accuracy. The same applies to the secondary transfer voltage.

[6. Other embodiments]
[6-1. Other embodiment 1]
In the above-described embodiment, the state variables include feature information on the actual printed matter (transfer voltage prediction image and fixing temperature prediction image), medium information of the print medium used for the actual printed matter, and information on the actual printed matter. Machine learning was performed using the control information obtained when printing was performed, and printing defect characteristic information indicating the type and degree (PQ value) of printing defects contained in actual printed matter.

Of these, for the prediction images (transfer voltage prediction image and fixing temperature prediction image), the type of printing defect, and the degree of printing defect (PQ value), it is not necessary to use all three pieces of information as state variables. It's okay. For example, among these, the prediction images (transfer voltage prediction image and fixing temperature prediction image) may be omitted, or the degree of printing failure (PQ value) may be omitted. Note that if the prediction images (transfer voltage prediction image and fixing temperature prediction image) are omitted, for example, the degree of printing defects may be treated as feature amount information in the actual printed matter instead of the prediction image. . In this way, even if you omit the prediction images (transfer voltage prediction image and fixing temperature prediction image) or omit the degree of printing defects (PQ value), especially when printing on label paper, Therefore, prediction accuracy can be expected to improve compared to conventional methods.

Furthermore, the present invention is not limited to this, and environmental information around the image forming apparatus (for example, information on the surrounding temperature and humidity) may be added to the state variables.

[6-2. Other Embodiment 2]
Furthermore, in the embodiment described above, the diagnostic chart DC includes the transfer voltage prediction area AR1 and the fixing temperature prediction area AR2, each of which is an area in which a color toner image is printed over a white toner image. Among these, the fixing temperature predicted area AR2 is an area where a black image with a duty of 100% is printed over a white image with a duty of 30%. Here, for a black image with a duty of 100%, it is desirable to perform solid printing using black toner, but if the image forming apparatus does not have black toner or the remaining amount of black toner is small, black toner Instead, a black image with a duty of 100% may be printed using other color toners.

[6-3. Other Embodiment 3]
Furthermore, in the embodiment described above, the data processing system 200 uses the learned model generated by the machine learning device 100 to determine optimal control information (secondary transfer voltage and toner fixing temperature) for the image forming device 210. I tried to predict it. Here, the image forming apparatus 210 updates the learned model (that is, performs AI automatic adjustment) at a predetermined timing (for example, every certain period or every time a new print medium PM is set). The control unit 212 may prompt the user by displaying the information on the operation panel 301.

[6-4. Other embodiment 4]
Furthermore, the present invention is not limited to the embodiments described above. That is, the scope of the present invention extends to embodiments in which a part or all of the above-described embodiments are arbitrarily combined, and embodiments in which a part is extracted.

The present invention can be widely used in, for example, electrophotographic image forming apparatuses.

10... Image forming device, 31W, 31K, 31C, 31M, 31Y... Image forming unit, 70... Control section, 100... Machine learning device, 110... State variable acquisition section, 120... Teacher data acquisition section , 130... Learned model generation unit, 140... Storage unit, 200... Data processing system, 210... Image forming device, 211... User interface, 212... Control unit, 213... Storage unit, 214... ...Printing section, 220...Reading device, 221...Document reading section, 230...Image diagnostic device, 231...Image information extraction section, 232...Image feature extraction section, 233...Medium information input section, 234... ... Control information input section, 235 ... AI diagnostic model, 236 ... Fixing temperature prediction model, 237 ... Transfer voltage prediction model, 238 ... Rounding processing section, 301 ... Operation panel, PM ... Print medium, AP ... ...Actual printed matter, DC...Diagnostic chart, AR1...Train voltage prediction area, AR2...Fusing temperature prediction area, NM...Neural network model, DT...Teacher data.

Claims (10)

Feature amount information on an actual printed matter printed by overlapping a white toner image and a color toner image by an image forming apparatus having a first image forming section that forms a white toner image and a second image forming section that forms a color toner image , printing defect characteristic information indicating the type of printing defect included in the actual printed matter, medium information of a printing medium used for the actual printed matter, and control information when the image forming apparatus printed the actual printed matter. a state variable acquisition unit that acquires certain first control information as a state variable;
a teacher data acquisition unit that acquires second control information that makes the feature amount information within a predetermined threshold value as teacher data;
a trained model generation unit that generates a trained model by performing machine learning based on the state variables acquired by the state variable acquisition unit and the teacher data acquired by the teacher data acquisition unit; Device. The feature amount information on the actual printed matter is an image printed on the actual printed matter,
The printing defect characteristic information indicates the type of printing defect in the image,
The first control information indicates a toner fixing temperature when the image forming apparatus prints the actual printed matter,
the second control information indicates a toner fixing temperature in the image forming apparatus;
The trained model generation unit includes:
a learned model for predicting toner fixing temperature in the image forming apparatus by performing machine learning based on the state variables acquired by the state variable acquisition unit and the teacher data acquired by the teacher data acquisition unit; The machine learning device according to claim 1. The feature amount information on the actual printed matter is an image printed on the actual printed matter,
The printing defect characteristic information indicates the type of printing defect in the image,
The first control information indicates a transfer voltage when the image forming apparatus prints the actual printed matter,
The second control information indicates a transfer voltage in the image forming apparatus,
The trained model generation unit includes:
A learned model for predicting the transfer voltage in the image forming apparatus is created by performing machine learning based on the state variable acquired by the state variable acquisition unit and the teacher data acquired by the teacher data acquisition unit. The machine learning device according to claim 1. The defect characteristic information indicates whether or not a printing defect related to the first control information has occurred in the actual printed material as a type of printing defect included in the actual printed material. machine learning device. The machine learning device according to claim 1, wherein the defect characteristic information indicates the type of printing defect included in the actual printed matter and the degree of the printing defect. The machine learning device according to claim 1, wherein the actual printed matter is label paper. Feature amount information on an actual printed matter printed by overlapping a white toner image and a color toner image by an image forming apparatus having a first image forming section that forms a white toner image and a second image forming section that forms a color toner image , printing defect characteristic information indicating the type of printing defect included in the actual printed matter, medium information of a printing medium used for the actual printed matter, and control information when the image forming apparatus printed the actual printed matter. an actual printed matter information acquisition unit that acquires certain first control information;
a data processing unit that inputs the information acquired by the actual printed matter information acquisition unit into the learned model generated by the machine learning device according to claim 1 and outputs third control information;
and a third control information storage unit that stores the third control information output from the data processing unit. The data processing unit includes:
performing a rounding process to replace the control information output from the learned model as a result of inputting the information acquired by the actual printed matter information acquisition unit into the learned model with one of discrete values at a predetermined interval; The data processing system according to claim 7, wherein the subsequent control information is output as third control information. using a computer,
Feature amount information on an actual printed matter printed by overlapping a white toner image and a color toner image by an image forming apparatus having a first image forming section that forms a white toner image and a second image forming section that forms a color toner image , printing defect characteristic information indicating the type of printing defect included in the actual printed matter, medium information of a printing medium used for the actual printed matter, and control information when the image forming apparatus printed the actual printed matter. a first process of acquiring certain first control information as a state variable;
a second process of acquiring second control information that makes the feature amount information within a predetermined threshold value as teacher data;
and a third process of generating a learned model by performing machine learning based on the state variable obtained by the first process and the teacher data obtained by the second process. How to learn. using a computer,
Feature amount information on an actual printed matter printed by overlapping a white toner image and a color toner image by an image forming apparatus having a first image forming section that forms a white toner image and a second image forming section that forms a color toner image , printing defect characteristic information indicating the type of printing defect included in the actual printed matter, medium information of a printing medium used for the actual printed matter, and control information when the image forming apparatus printed the actual printed matter. a fourth process of acquiring certain first control information;
a fifth process of inputting the information acquired by the fourth process into the learned model generated by the machine learning method according to claim 9 and outputting third control information;
and a sixth process for storing the third control information output by the fifth process.

Images