EP4659192A1 - Information processing apparatus, information processing method and program - Google Patents

Information processing apparatus, information processing method and program

Info

Publication number
EP4659192A1
EP4659192A1 EP24712603.0A EP24712603A EP4659192A1 EP 4659192 A1 EP4659192 A1 EP 4659192A1 EP 24712603 A EP24712603 A EP 24712603A EP 4659192 A1 EP4659192 A1 EP 4659192A1
Authority
EP
European Patent Office
Prior art keywords
defect
information processing
image
processing apparatus
industrial process
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP24712603.0A
Other languages
German (de)
French (fr)
Inventor
Yasuhiro Toyooka
Kazuhiro Kobashi
Naoki Ikegawa
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Kurita Water Industries Ltd
Original Assignee
Kurita Water Industries Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kurita Water Industries Ltd filed Critical Kurita Water Industries Ltd
Publication of EP4659192A1 publication Critical patent/EP4659192A1/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B23/00—Testing or monitoring of control systems or parts thereof
    • G05B23/02—Electric testing or monitoring
    • G05B23/0205—Electric testing or monitoring by means of a monitoring system capable of detecting and responding to faults
    • G05B23/0259—Electric testing or monitoring by means of a monitoring system capable of detecting and responding to faults characterized by the response to fault detection
    • G05B23/0275—Fault isolation and identification, e.g. classify fault; estimate cause or root of failure
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10056—Microscopic image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • the present invention relates to an information processing apparatus, an information processing method and a program.
  • Patent Document 1 discloses a technology for performing slime control in a water system containing reducing substance through predetermined control.
  • the present invention provides an information processing apparatus capable of easily specifying a cause of a defect, etc.
  • an information processing apparatus comprises an image acquisition unit, a relationship model acquisition unit, and an output unit.
  • the image acquisition unit is configured to acquire a predetermined image from within an industrial process.
  • the predetermined image includes an image with respect to a defect occurred within the industrial process.
  • the relationship model acquisition unit is configured to acquire a relationship model.
  • the relationship model is a model created by associating analysis data obtained by analyzing a predetermined point in the industrial process with image data with respect to an image of a corresponding defect.
  • the output unit is configured to output a cause of the defect and/or a countermeasure to the defect based on the predetermined image and the relationship model.
  • An information processing apparatus comprising: an image acquisition unit configured to acquire a predetermined image from within an industrial process, the predetermined image including an image with respect to a defect occurred within the industrial process; a relationship model acquisition unit configured to acquire a relationship model, the relationship model being a model created by associating analysis data obtained by analyzing a predetermined point in the industrial process with image data with respect to an image of a corresponding defect; and an output unit configured to output a cause of the defect and/or a countermeasure to the defect based on the predetermined image and the relationship model.
  • the output unit is configured to output a cause of the defect and a countermeasure to the defect.
  • the analysis data is data obtained by analyzing the predetermined point using a chemical or biochemical method.
  • the analysis data includes data with respect to on one or more analyses selected from a group comprising optical microscopic observation, electron microscopic observation, FT-IR analysis, pyrolysis gas chromatography analysis, ninhydrin reaction test, X-ray Fluorescence analysis, X-ray diffraction analysis, bacteria count analysis, fungi count analysis, element mapping analysis, iodine starch reaction test, extraction amount by organic solvent, extraction amount by acid, and extraction amount by alkali.
  • the analysis data is data obtained by analyzing defect in the industrial process as a predetermined point in the industrial process.
  • the industrial process is a process comprising a step of a water system.
  • the industrial process is a paper manufacturing process.
  • the image data associated with the relationship model includes information on any of color, size, and shape of the defect
  • the output unit is configured to output a cause of the defect and/or a countermeasure to the defect based on information on any of color, size, and shape of the defect in the predetermined image.
  • the predetermined image is an image captured by an RGB camera.
  • the output unit is configured to output an operating condition of the industrial process as a countermeasure to the defect.
  • the information processing apparatus according to any one of (1) to (10), wherein: the relationship model is a model configured to be updated prior to the output unit outputting the cause of the defect and/or the countermeasure to the defect.
  • An information processing method executed by an information processing apparatus comprising: an image acquisition step of acquiring a predetermined image from within an industrial process, the predetermined image including an image with respect to a defect occurred within the industrial process; a relationship model acquisition step of acquiring a relationship model, the relationship model being a model created by associating analysis data obtained by analyzing a predetermined point in the industrial process with image data with respect to an image of a corresponding defect; and an output step of outputting a cause of the defect and/or a countermeasure to the defect based on the predetermined image and the relationship model.
  • a program allowing a computer to function as: an image acquisition unit configured to acquire a predetermined image from within an industrial process, the predetermined image including an image with respect to a defect occurred within the industrial process; a relationship model acquisition unit configured to acquire a relationship model, the relationship model being a model created by associating analysis data obtained by analyzing a predetermined point in the industrial process with image data with respect to an image of a corresponding defect; and an output unit configured to output a cause of the defect and/or a countermeasure to the defect based on the predetermined image and the relationship model.
  • an information processing apparatus or the like capable of easily specifying a cause of a defect, etc. is provided.
  • FIG. 1 shows an overall configuration of an information processing system 100.
  • FIG. 2 shows a hardware configuration of an information processing apparatus 1.
  • FIG. 3 is a functional block diagram showing a function of an information processing apparatus 1.
  • FIG. 4 is an activity diagram showing a flow of information processing using an information processing apparatus 1, etc.
  • FIG. 5 is an example of an output content displayed by an output unit 113.
  • a program for realizing a software in the present embodiment may be provided as a non-transitory computer readable medium that can be read by a computer, may be provided for download from an external server, or may be provided in such a manner that the program can be activated on an external computer to realize functions thereof on a client terminal (so-called cloud computing).
  • the "unit" may include, for instance, a combination of hardware resources implemented by a circuit in a broad sense and information processing of software that can be concretely realized by these hardware resources. Further, various information is performed in the present embodiment, and the information can be represented by, for instance, physical values of signal values representing voltage and current, high and low signal values as a set of binary bits consisting of 0 or 1, or quantum superposition (so-called qubits), and communication/calculation can be executed on a circuit in a broad sense.
  • qubits quantum superposition
  • the circuit in a broad sense is a circuit realized by combining at least an appropriate number of a circuit, a circuitry, a processor, a memory, or the like.
  • a circuit includes application specific integrated circuit (ASIC), programmable logic device (e.g., simple programmable logic device (SPLD), complex programmable logic device (CPLD), field programmable gate array (FPGA)), or the like.
  • ASIC application specific integrated circuit
  • SPLD simple programmable logic device
  • CPLD complex programmable logic device
  • FPGA field programmable gate array
  • FIG. 1 shows an overall configuration of the information processing system 100.
  • the information processing system 100 is a system used to output a cause of a defect or a countermeasure to the defect occurring in an industrial process P.
  • the information processing system 100 comprises an information processing apparatus 1 and an imaging apparatus 2, which are connected via a communication line.
  • the communication line here includes the Internet, wireless, etc., and serves to mediate exchange of data between apparatuses connected to own line.
  • a system exemplified by the information processing system 100 comprises one or more apparatuses or components. Therefore, the information processing apparatus 1 alone may be considered as an example of a system, and an example including the imaging apparatus 2 or the industrial process P to be applied may be referred to as a system as well.
  • illustration will continue regarding each apparatus etc. that may configure the information processing system 100.
  • FIG. 2 shows a hardware configuration of the information processing apparatus 1.
  • the information processing apparatus 1 comprises a controller 11, a storage unit 12, an input unit 13, a display unit 14, and a communication unit 15, each of these components is configured to be electrically connected by a communication bus 10.
  • a communication bus 10 a communication bus 10.
  • the controller 11 is, for example, an unshown central processing unit (CPU).
  • the controller 11 is configured to realize various functions related to the information processing apparatus 1 by reading a predetermined program stored in the storage unit 12.
  • information processing by software stored in the storage unit 12 is specifically realized by the controller 11, an example of hardware, thereby may be executed as each functional unit included in the controller 11. Further details on these will be described in the next section.
  • the controller 11 is not limited to being singular and may be implemented with two or more controllers 11 for each function. Additionally, a combination thereof may be applied.
  • the storage unit 12 is configured to store various information as defined by the above description. This may be implemented as a storage device such as a solid state drive (SSD) storing various programs related to the information processing apparatus 1 that are executed by the controller 11, or as a memory such as a random access memory (RAM) that stores temporarily necessary information (argument, sequence, etc.) for program operation.
  • the storage unit 12 stores various programs, variables, etc. related to the information processing apparatus 1 that are executed by the controller 11.
  • the input unit 13 may be included in a housing of the information processing apparatus 1 or may be externally attached.
  • the input unit 13 may be implemented as a touch panel integrated with the display unit 14. With the touch panel, a user may input through tapping, swiping, or other operation.
  • a switch button, a mouse, a QWERTY keyboard, etc. may be employed instead of the touch panel.
  • the input unit 13 receives operation input performed by the user. This input, treated as a command signal, is transferred to the controller 11 via the communication bus 10, and the controller 11 may execute predetermined control or calculation as necessary.
  • the display unit 14 may be included in a housing of the information processing apparatus 1 or may be externally attached, for example.
  • the display unit 14 is configured to display a screen of graphical user interface (GUI) that is operable by a user. For instance, this is preferable to be implemented by using different display devices such as a CRT display, a liquid crystal display, an organic EL display, and a plasma display, depending on type of the information processing apparatus 1.
  • GUI graphical user interface
  • the communication unit 15 is configured to transmit various electrical signals from the information processing apparatus 1 to an external component. Further, the communication unit 15 is configured to receive various electrical signals from an external component to the information processing apparatus 1.
  • the communication unit 15 may be implemented with a network communication function to enable communication of various information between the information processing apparatus 1 and the external component via a communication line.
  • the imaging apparatus 2 is an apparatus that captures at least part of the industrial process P as an image.
  • the imaging apparatus 2 may be appropriately selected from known apparatuses capable of imaging an object, etc. From a certain perspective, the imaging apparatus 2 may be a digital camera, a camera attached to a smartphone, a camera attached to a computer, or the like.
  • the imaging apparatus 2 may be an RGB camera, which captures at least part of the industrial process P as an image. In this case, data to be acquired may be so-called RGB data.
  • the industrial process P to which the information processing system 100 in the present embodiment applies may be appropriately set from known industrial processes.
  • the industrial process P is typically assumed to be a process implemented in various known plants, such as a paper manufacturing process, a steel manufacturing process, a power generation process, a petroleum refining process, a chemical process, a painting process, a semiconductor processing process, etc.
  • the industrial process P comprises a water system process, since predetermined analysis data is used in the information processing system 100 according to the present embodiment.
  • the industrial process P is a paper manufacturing process.
  • a flow of information processing will be illustrated assuming that the industrial process P is a paper manufacturing process using a water system.
  • FIG. 3 is a functional block diagram showing a function of the information processing apparatus 1.
  • information processing by software stored in the storage unit 12
  • hardware the controller 11
  • the information processing apparatus 1 may comprise an image acquisition unit 111, a relationship model acquisition unit 112, an output unit 113, a relationship model creation unit 114, and a storage management unit 115 as each functional unit. It should be noted that each functional unit may be increased or omitted as appropriate depending on application to which the information processing apparatus 1 is applied.
  • the image acquisition unit 111 is configured to execute an image acquisition step.
  • the image acquisition unit 111 acquires a predetermined image from within the industrial process P.
  • the predetermined image includes an image with respect to a defect occurring in the industrial process P.
  • the image acquisition unit 111 is configured, for instance, to acquire various information via the communication unit 15 from the imaging apparatus 2, which is capable of imaging at least part of the industrial process P.
  • the relationship model acquisition unit 112 is configured to execute a relationship model acquisition step.
  • the relationship model acquisition unit 112 acquires a relationship model.
  • the relationship model is created by associating analysis data obtained by analyzing a predetermined point within the industrial process P with image data with respect to an image of the corresponding defect. Further details regarding the model will be described in a later section.
  • the output unit 113 is configured to execute an output step. In the output step, the output unit 113 generates various output objects. In a typical manner, the output object is configured to be recognizable to a user or the like. In this case, the output unit 113 is configured to create display information and control the display information to be visible to the user, etc.
  • the display information may be visual information itself, such as screen, image, icon, text, etc., generated in a recognizable manner for the user, or may be, for example, rendering information for displaying visual information such as screen, image, icon, text, etc. on various apparatus or terminals.
  • the output unit 113 may be configured to output a signal to allow a printing apparatus to drive or to allow a predetermined apparatus to drive. In the information processing system 100 according to the present embodiment, the output unit 113 is configured to output a cause of the defect and/or a countermeasure to the defect based on the predetermined image and the relationship model. Specific manners of output content will be described later.
  • the relationship model creation unit 114 is configured to execute a relationship model creation step. In the relationship model creation step, the relationship model creation unit 114 creates or updates a relationship model to be used in the above-described output step or the like.
  • the storage management unit 115 is configured to execute a storage management step. In the storage management step, the storage management unit 115 is configured to manage various information to be stored that is associated with the information processing system 100 according to the present embodiment. Typically, the storage management unit 115 is configured to allow information handled by the information processing apparatus 1, etc. to be stored in a storage area. Examples of the storage area include the storage unit 12 of the information processing apparatus 1 or storage units of various apparatuses or terminals, but the storage area does not necessarily have to be within the information processing system 100, and the storage management unit 115 may manage various information to be stored in an external storage unit or the like as well.
  • FIG. 4 is an activity diagram showing a flow of information processing using the information processing apparatus 1, etc.
  • the image acquisition unit 111 acquires a predetermined image from within the industrial process P (step S1).
  • the step S1 may typically be accomplished by the imaging apparatus 2 capturing a predetermined point of the industrial process P and the image acquisition unit 111 acquiring image data via a communication line.
  • an object to be captured by the imaging apparatus 2 may be any object that is capturable, and may be either organic or inorganic.
  • the object may be a predetermined equipment (including various equipment such as reaction vessel, tank, transport apparatus, etc.) present in the industrial process P, may be a raw material used in the industrial process P, or may be a product or an intermediate manufactured in the industrial process P.
  • the image acquired in the step S1 includes an image with respect to the defect that occurred in the industrial process P.
  • defect refers to a state in which the industrial process P is not operating as designed or a resultant product thereof (in other words, a state that is not normal (abnormal state)).
  • the information processing method according to the present embodiment may be applied to a case where there is an abnormality in the industrial process P or a case where the resultant product is a different image from that when the industrial process P is operating normally. Typical examples include when an equipment has a defect or fault, when a raw material or a product (intermediate product) has a different color or pattern than usual, or when a raw material or a product (intermediate product) has a different dimension than usual.
  • an image including such a defect is acquired by the imaging apparatus 2.
  • the relationship model acquisition unit 112 acquires a relationship model (step S2).
  • the order of performing step S1 and step S2 is arbitrary, step S1 may be performed prior to step S2, step S2 may be performed prior to step S1, or both steps may be performed simultaneously.
  • the relationship model is a model created by associating analysis data obtained by analyzing a predetermined point in the industrial process P with image data related to an image of a corresponding defect.
  • the relationship model is a model of the relationship between the predetermined analysis data and the image data related to the image of the corresponding defect.
  • the model may be, for example, a function or a look-up table showing the relationship between the predetermined analysis data and the image data related to the image of the corresponding defect, or may be a learned model in which the relationship between the predetermined analysis data and the image data related to the image of the corresponding defect has been learned.
  • the relationship between the analysis data and the image data related to the image of the corresponding defect may be analyzed based on a known analysis method.
  • regression analysis method linear model, generalized linear model, generalized linear mixed model, ridge regression, Lasso regression, elastic net, support vector regression, projection tracking regression, etc.
  • time series analysis VAR model, SVAR model, ARIMAX model, SARIMAX model, state space model, etc.
  • decision tree decision tree, regression tree, random forest, XGBoost (Extreme Gradient Boosting), etc.
  • neural network simple perceptron, multilayer perceptron, Transformer, DNN (Deep Neural Network), CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), LSTM (Long Short Term Memory), etc.
  • Bayes naive Bayes, etc.
  • clustering k-means, k-means++, etc.
  • ensemble learning Boosting, Adaboost, etc.
  • a neural network is preferably to be employed among the above-mentioned analysis methods.
  • a software or a program for creating and operating the relationship model according to the present embodiment may be selected from among those that can use the neural network.
  • framework such as Keras, TensorFlow, or PyTorch
  • software such as Teachable Machine (Google LLC) or DataRobot (DataRobot, Inc.) may be used as well.
  • relationship model creation unit 114 creation of the relationship model is exemplified as being performed by the relationship model creation unit 114, it is not necessarily limited thereto, and the information processing method according to the present embodiment may be performed by creating such a relationship model outside the information processing system 100 and installing the created relationship model in the information processing apparatus 1, etc.
  • the relationship model may be a model configured to be updated prior to the output unit 113 outputting the cause of the defect and/or the countermeasure to the defect (before the output step is executed).
  • the relationship model creation unit 114 may be configured to update the relationship model sequentially in accordance with accumulation of information with respect to the countermeasure to the defect, etc.
  • the analysis data related to the relationship model does not necessarily have to be data obtained by analyzing the defect itself.
  • the analysis data related to the relationship model need not be analysis data regarding the product itself but may be data analyzing manufacturing equipment or raw material used.
  • the analysis data is preferably data obtained by analyzing a defect in the industrial process P as a predetermined point in the industrial process P. This latter manner makes it easier to evaluate a defect with higher accuracy.
  • the analysis data according to the present embodiment may be based on any known analysis method.
  • the analysis data according to the present embodiment may be data obtained by analyzing a predetermined point with a chemical or biochemical method.
  • the analysis data according to the present embodiment may include data with respect to on one or more analyses selected from a group comprising optical microscopic observation, electron microscopic observation, FT-IR analysis, pyrolysis gas chromatography analysis, ninhydrin reaction test, X-ray Fluorescence analysis, X-ray diffraction analysis, bacteria count analysis, fungi count analysis, element mapping analysis, iodine starch reaction test, extraction amount by organic solvent, extraction amount by acid, and extraction amount by alkali.
  • compositional characteristics related to a defect may be estimated from such analysis data. Therefore, the cause or the countermeasure for the defect may be output with high accuracy from the image information acquired by the image acquisition unit 111. Note that in the case of paper manufacturing process, etc. (including water system) as illustrated above, the use of such analysis data tends to contribute to improving accuracy of an output result.
  • the image data associated with the relationship model may include information on any of color, size, and shape of the defect.
  • the output unit 113 is preferably configured to output a cause of the defect and/or a countermeasure to the defect based on information on any of color, size, and shape of the defect in the predetermined image.
  • an RGB camera may be applied as the imaging apparatus 2, and color, size, and shape of the defect may be associated with RGB data of image information acquired by the RGB camera. That is, a combination of RGB data for each pixel may be used as a basis for outputting a cause, etc. in the output step described later.
  • the information processing method outputs the cause of the defect and/or the countermeasure to the defect based on the image acquired by the image acquisition unit 111 and the relationship model acquired by the relationship model acquisition unit 112 (step S3).
  • This step may be accomplished by a function of the output unit 113.
  • the output unit 113 may output only one of the cause of the defect and the countermeasure to the defect, it is preferable that the output unit 113 outputs the cause of the defect and the countermeasure to the defect.
  • FIG. 5 is an example of output content displayed by the output unit 113.
  • a display screen D in FIG. 5 is visual information output by the output unit 113 of the information processing apparatus 1, typically displayed as a screen on the display unit 14 of the information processing apparatus 1.
  • the display screen D shows the image of the defect acquired by the image acquisition unit 111, along with a type of defect (type of fault), a cause, and a countermeasure of the defect, each of which is associated with the image of the defect.
  • the "type of defect” refers to a broad classification of defects (faults), and more detailed cause is indicated in an item “cause”.
  • an operating condition of the industrial process P is output as a countermeasure to the defect. (see an item “countermeasure” in FIG. 5).
  • Countermeasure A Application of water treatment chemicals, change of type, adjustment of addition amount, change of addition location, suspension of addition
  • water treatment chemicals here include slime control agents, preservative, pitch control agent, defoamer, scale inhibitor, process cleaning agent (acidic agent, alkaline agent, chelating agent, etc.), yield filtration agent (include organic and inorganic), flocculation agent (include organic and inorganic), coagulant (include organic and inorganic), charge adjuster, degassing agent, cleaner aid, felt conditioner, roll stain inhibitor for paper machine, canvass stain inhibitor for paper machines, creping agent, evaporation aid, deinking agent or bleaching agent (surfactant, sodium hydroxide, sodium hypochlorite, chlorine dioxide, ozone, hydrogen peroxide, chelating agents, sodium silicate, etc.) beating aid, release aid, etc.
  • Countermeasure B Application of functional chemicals affecting water treatment, change of type, adjustment of addition amount, change of addition location, suspension of addition
  • functional chemicals include paper strength agent (include dry and wet), sizing agent, dye, sticky agent, filler (such as calcium carbonate and kaolin), etc.
  • Countermeasure C Application of water treatment equipment, change of type, change of operation method, adjustment of operation load, suspension of operation
  • water treatment equipment here include pressurized flotation apparatus, coagulation sedimentation apparatus, colander, washing equipment such as filter and dehydrator, foreign matter removal apparatus such as screen, cleaner, and floater, etc.
  • change of operation method includes changing supplementary water amount or supplementary water kind to a system, changing amount of process water discharged from the system, etc.
  • Countermeasure D Application of production equipment affecting water treatment, change of type, change of operation method, adjustment of operation load, suspension of operation
  • Examples of production equipment here include release machine, disperser, beating machine, paper machine, steaming equipment, bleaching equipment, etc.
  • change of operation method includes changing type or blending ratio of raw material, production order of production item, timing of periodic repair, timing of cleaning inside system, etc.
  • the contents of the countermeasures are not limited to the above, but may be set as appropriate depending on type and scale of the industrial process P.
  • the content output by the output unit 113 is not limited to such a display screen, and the cause or the countermeasure is outputted as printed matter may be exemplified as an example.
  • the output unit 113 may output the operating condition of the industrial process P as a signal.
  • a part of the equipment provided in the industrial process P may be configured to operate automatically according to the content of the signal.
  • the content output by the output unit 113 may be stored in a predetermined storage area by a function of the storage management unit 115. Further, the relationship model may be updated according to the output content stored in such a manner. Such updating of the relationship model may be realized by a function of the relationship model creation unit 114.
  • a cause of a defect or the like may be easily specified.
  • the embodiment described above shows an information processing method using a relationship model, but information associated when creating the relationship model is not limited thereto. That is, the relationship model used in the present embodiment may be associated with various other conditions such as weather condition, condition related to region, and condition related to equipment age.
  • the information processing apparatus 1 performs various storage and controls
  • two or more external apparatuses may be applied instead of the information processing apparatus 1.
  • image of defect, information on cause, countermeasure, etc. may be distributed and stored in two or more external apparatuses using blockchain technology or the like.
  • Various analyses are conducted on 23 defective samples from a paperboard production equipment to specify a main component of a defect and to estimate a cause and a water treatment method for a countermeasure.
  • a database with a sample size of 23 is created by adding image data of the defect.
  • a relationship model is created using supervised learning to estimate the cause suggested by the analyzed data from the image data of the defect.
  • a procedure for creating the relationship model is to use Teachable Machine (Google LLC) to create a model with a deep neural network.
  • Information processing apparatus 2 Imaging apparatus 3: Type of countermeasure 4: Type of countermeasure 10: Communication bus 11: Controller 12: Storage unit 13: Input unit 14: Display unit 15: Communication unit 100: Information processing system 111: Image acquisition unit 112: Relationship model acquisition unit 113: Output unit 114: Relationship model creation unit 115: Storage management unit D: Display screen P: Industrial process

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Chemical & Material Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Signal Processing (AREA)
  • Automation & Control Theory (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • General Factory Administration (AREA)

Abstract

The present invention provides an information processing apparatus capable of easily specifying a cause of a defect, etc. According to an aspect of the present invention, an information processing apparatus is provided. The information processing apparatus comprises an image acquisition unit, a relationship model acquisition unit, and an output unit. The image acquisition unit is configured to acquire a predetermined image from within an industrial process. The predetermined image includes an image with respect to a defect that occurred within the industrial process. The relationship model acquisition unit is configured to acquire a relationship model. The relationship model is a model created by associating analysis data obtained by analyzing a predetermined point in the industrial process with image data with respect to an image of corresponding defect. The output unit is configured to output a cause of the defect and/or a countermeasure to the defect based on the predetermined image and the relationship model.

Description

    INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING METHOD AND PROGRAM
  • The present invention relates to an information processing apparatus, an information processing method and a program.
  • Various technologies have been developed to prevent occurrence of problem in an industrial process. Patent Document 1 discloses a technology for performing slime control in a water system containing reducing substance through predetermined control.
  • Japanese Patent Application, Publication No. 2009-241018
  • While there are technologies to prevent the occurrence of detects such as those described above, an unexpected defect may occur in an actual industrial process. Conventionally, various analyses have been conducted to specify a cause of such a defect, but the reality is that these analyses take a considerable number of days. Moreover, since the analysis to specify such a defect requires skill of an analyst, a technology capable of easily specifying the cause of defect has been desired.
  • In view of the above circumstances, the present invention provides an information processing apparatus capable of easily specifying a cause of a defect, etc.
  • According to an aspect of the present invention, an information processing apparatus is provided. The information processing apparatus comprises an image acquisition unit, a relationship model acquisition unit, and an output unit. The image acquisition unit is configured to acquire a predetermined image from within an industrial process. The predetermined image includes an image with respect to a defect occurred within the industrial process. The relationship model acquisition unit is configured to acquire a relationship model. The relationship model is a model created by associating analysis data obtained by analyzing a predetermined point in the industrial process with image data with respect to an image of a corresponding defect. The output unit is configured to output a cause of the defect and/or a countermeasure to the defect based on the predetermined image and the relationship model.
  • Specifically, the present invention may be provided in each of following aspects.
    (1) An information processing apparatus, comprising:
    an image acquisition unit configured to acquire a predetermined image from within an industrial process, the predetermined image including an image with respect to a defect occurred within the industrial process;
    a relationship model acquisition unit configured to acquire a relationship model, the relationship model being a model created by associating analysis data obtained by analyzing a predetermined point in the industrial process with image data with respect to an image of a corresponding defect; and
    an output unit configured to output a cause of the defect and/or a countermeasure to the defect based on the predetermined image and the relationship model.
    (2) The information processing apparatus according to (1), wherein:
    the output unit is configured to output a cause of the defect and a countermeasure to the defect.
    (3) The information processing apparatus according to (1) or (2), wherein:
    the analysis data is data obtained by analyzing the predetermined point using a chemical or biochemical method.
    (4) The information processing apparatus according to (1) or (2), wherein:
    the analysis data includes data with respect to on one or more analyses selected from a group comprising optical microscopic observation, electron microscopic observation, FT-IR analysis, pyrolysis gas chromatography analysis, ninhydrin reaction test, X-ray Fluorescence analysis, X-ray diffraction analysis, bacteria count analysis, fungi count analysis, element mapping analysis, iodine starch reaction test, extraction amount by organic solvent, extraction amount by acid, and extraction amount by alkali.
    (5) The information processing apparatus according to any one of (1) to (4), wherein:
    the analysis data is data obtained by analyzing defect in the industrial process as a predetermined point in the industrial process.
    (6) The information processing apparatus according to any one of (1) to (5), wherein:
    the industrial process is a process comprising a step of a water system.
    (7) The information processing apparatus according to any one of (1) to (6), wherein:
    the industrial process is a paper manufacturing process.
    (8) The information processing apparatus according to any one of (1) to (7), wherein:
    the image data associated with the relationship model includes information on any of color, size, and shape of the defect, and
    the output unit is configured to output a cause of the defect and/or a countermeasure to the defect based on information on any of color, size, and shape of the defect in the predetermined image.
    (9) The information processing apparatus according to any one of (1) to (8), wherein:
    the predetermined image is an image captured by an RGB camera.
    (10) The information processing apparatus according to any one of (1) to (9), wherein:
    the output unit is configured to output an operating condition of the industrial process as a countermeasure to the defect.
    (11) The information processing apparatus according to any one of (1) to (10), wherein:
    the relationship model is a model configured to be updated prior to the output unit outputting the cause of the defect and/or the countermeasure to the defect.
    (12) An information processing method executed by an information processing apparatus, comprising:
    an image acquisition step of acquiring a predetermined image from within an industrial process, the predetermined image including an image with respect to a defect occurred within the industrial process;
    a relationship model acquisition step of acquiring a relationship model, the relationship model being a model created by associating analysis data obtained by analyzing a predetermined point in the industrial process with image data with respect to an image of a corresponding defect; and
    an output step of outputting a cause of the defect and/or a countermeasure to the defect based on the predetermined image and the relationship model.
    (13) A program allowing a computer to function as:
    an image acquisition unit configured to acquire a predetermined image from within an industrial process, the predetermined image including an image with respect to a defect occurred within the industrial process;
    a relationship model acquisition unit configured to acquire a relationship model, the relationship model being a model created by associating analysis data obtained by analyzing a predetermined point in the industrial process with image data with respect to an image of a corresponding defect; and
    an output unit configured to output a cause of the defect and/or a countermeasure to the defect based on the predetermined image and the relationship model.
  • According to the aspects, an information processing apparatus or the like capable of easily specifying a cause of a defect, etc. is provided.
  • FIG. 1 shows an overall configuration of an information processing system 100. FIG. 2 shows a hardware configuration of an information processing apparatus 1. FIG. 3 is a functional block diagram showing a function of an information processing apparatus 1. FIG. 4 is an activity diagram showing a flow of information processing using an information processing apparatus 1, etc. FIG. 5 is an example of an output content displayed by an output unit 113.
  • Hereinafter, embodiments of the present invention will be described. It should be noted that various features described in the embodiment below can be combined with each other.
  • A program for realizing a software in the present embodiment may be provided as a non-transitory computer readable medium that can be read by a computer, may be provided for download from an external server, or may be provided in such a manner that the program can be activated on an external computer to realize functions thereof on a client terminal (so-called cloud computing).
  • In the present embodiment, the "unit" may include, for instance, a combination of hardware resources implemented by a circuit in a broad sense and information processing of software that can be concretely realized by these hardware resources. Further, various information is performed in the present embodiment, and the information can be represented by, for instance, physical values of signal values representing voltage and current, high and low signal values as a set of binary bits consisting of 0 or 1, or quantum superposition (so-called qubits), and communication/calculation can be executed on a circuit in a broad sense.
  • Further, the circuit in a broad sense is a circuit realized by combining at least an appropriate number of a circuit, a circuitry, a processor, a memory, or the like. In other words, it is a circuit includes application specific integrated circuit (ASIC), programmable logic device (e.g., simple programmable logic device (SPLD), complex programmable logic device (CPLD), field programmable gate array (FPGA)), or the like.
  • 1. Hardware configuration
    In this section, a hardware configuration of an information processing system 100 according to the present embodiment will be described. FIG. 1 shows an overall configuration of the information processing system 100.
  • The information processing system 100 according to the present embodiment is a system used to output a cause of a defect or a countermeasure to the defect occurring in an industrial process P.
    The information processing system 100 according to the present embodiment comprises an information processing apparatus 1 and an imaging apparatus 2, which are connected via a communication line. The communication line here includes the Internet, wireless, etc., and serves to mediate exchange of data between apparatuses connected to own line. On the other hand, a system exemplified by the information processing system 100 comprises one or more apparatuses or components. Therefore, the information processing apparatus 1 alone may be considered as an example of a system, and an example including the imaging apparatus 2 or the industrial process P to be applied may be referred to as a system as well. Hereinafter, illustration will continue regarding each apparatus etc. that may configure the information processing system 100.
  • (Information processing apparatus 1)
    FIG. 2 shows a hardware configuration of the information processing apparatus 1. The information processing apparatus 1 comprises a controller 11, a storage unit 12, an input unit 13, a display unit 14, and a communication unit 15, each of these components is configured to be electrically connected by a communication bus 10. Hereinafter, each unit of the information processing system 100 will be illustrated.
  • (Controller 11)
    The controller 11 is, for example, an unshown central processing unit (CPU). The controller 11 is configured to realize various functions related to the information processing apparatus 1 by reading a predetermined program stored in the storage unit 12. In other words, information processing by software stored in the storage unit 12 is specifically realized by the controller 11, an example of hardware, thereby may be executed as each functional unit included in the controller 11. Further details on these will be described in the next section. It should be noted that the controller 11 is not limited to being singular and may be implemented with two or more controllers 11 for each function. Additionally, a combination thereof may be applied.
  • (Storage unit 12)
    The storage unit 12 is configured to store various information as defined by the above description. This may be implemented as a storage device such as a solid state drive (SSD) storing various programs related to the information processing apparatus 1 that are executed by the controller 11, or as a memory such as a random access memory (RAM) that stores temporarily necessary information (argument, sequence, etc.) for program operation. The storage unit 12 stores various programs, variables, etc. related to the information processing apparatus 1 that are executed by the controller 11.
  • (Input unit 13)
    The input unit 13 may be included in a housing of the information processing apparatus 1 or may be externally attached. For example, the input unit 13 may be implemented as a touch panel integrated with the display unit 14. With the touch panel, a user may input through tapping, swiping, or other operation. Of course, a switch button, a mouse, a QWERTY keyboard, etc. may be employed instead of the touch panel. In other words, the input unit 13 receives operation input performed by the user. This input, treated as a command signal, is transferred to the controller 11 via the communication bus 10, and the controller 11 may execute predetermined control or calculation as necessary.
  • (Display unit 14)
    The display unit 14 may be included in a housing of the information processing apparatus 1 or may be externally attached, for example. The display unit 14 is configured to display a screen of graphical user interface (GUI) that is operable by a user. For instance, this is preferable to be implemented by using different display devices such as a CRT display, a liquid crystal display, an organic EL display, and a plasma display, depending on type of the information processing apparatus 1.
  • (Communication unit 15)
    The communication unit 15 is configured to transmit various electrical signals from the information processing apparatus 1 to an external component. Further, the communication unit 15 is configured to receive various electrical signals from an external component to the information processing apparatus 1. The communication unit 15 may be implemented with a network communication function to enable communication of various information between the information processing apparatus 1 and the external component via a communication line.
  • (Imaging apparatus 2)
    The imaging apparatus 2 is an apparatus that captures at least part of the industrial process P as an image. The imaging apparatus 2 may be appropriately selected from known apparatuses capable of imaging an object, etc. From a certain perspective, the imaging apparatus 2 may be a digital camera, a camera attached to a smartphone, a camera attached to a computer, or the like. As an example, the imaging apparatus 2 may be an RGB camera, which captures at least part of the industrial process P as an image. In this case, data to be acquired may be so-called RGB data.
  • (Industrial process P)
    The industrial process P to which the information processing system 100 in the present embodiment applies may be appropriately set from known industrial processes. The industrial process P is typically assumed to be a process implemented in various known plants, such as a paper manufacturing process, a steel manufacturing process, a power generation process, a petroleum refining process, a chemical process, a painting process, a semiconductor processing process, etc.
    As will be described later, it is preferable that the industrial process P comprises a water system process, since predetermined analysis data is used in the information processing system 100 according to the present embodiment. In particular, it is preferable that the industrial process P is a paper manufacturing process. In the following description, a flow of information processing will be illustrated assuming that the industrial process P is a paper manufacturing process using a water system.
  • 2. Functional configuration
    In this section, a functional configuration according to the present embodiment will be described. FIG. 3 is a functional block diagram showing a function of the information processing apparatus 1. As mentioned above, information processing by software (stored in the storage unit 12) is concretely realized by hardware (the controller 11), thereby may be executed as each functional unit included in the controller 11.
  • Specifically, the information processing apparatus 1 (the controller 11) may comprise an image acquisition unit 111, a relationship model acquisition unit 112, an output unit 113, a relationship model creation unit 114, and a storage management unit 115 as each functional unit. It should be noted that each functional unit may be increased or omitted as appropriate depending on application to which the information processing apparatus 1 is applied.
  • (Image acquisition unit 111)
    The image acquisition unit 111 is configured to execute an image acquisition step. In the image acquisition step, the image acquisition unit 111 acquires a predetermined image from within the industrial process P. Here, the predetermined image includes an image with respect to a defect occurring in the industrial process P. During this acquisition, the image acquisition unit 111 is configured, for instance, to acquire various information via the communication unit 15 from the imaging apparatus 2, which is capable of imaging at least part of the industrial process P.
  • (Relationship model acquisition unit 112)
    The relationship model acquisition unit 112 is configured to execute a relationship model acquisition step. In the relationship model acquisition step, the relationship model acquisition unit 112 acquires a relationship model. Here, the relationship model is created by associating analysis data obtained by analyzing a predetermined point within the industrial process P with image data with respect to an image of the corresponding defect. Further details regarding the model will be described in a later section.
  • (Output unit 113)
    The output unit 113 is configured to execute an output step. In the output step, the output unit 113 generates various output objects. In a typical manner, the output object is configured to be recognizable to a user or the like. In this case, the output unit 113 is configured to create display information and control the display information to be visible to the user, etc. The display information may be visual information itself, such as screen, image, icon, text, etc., generated in a recognizable manner for the user, or may be, for example, rendering information for displaying visual information such as screen, image, icon, text, etc. on various apparatus or terminals. Additionally, the output unit 113 may be configured to output a signal to allow a printing apparatus to drive or to allow a predetermined apparatus to drive. In the information processing system 100 according to the present embodiment, the output unit 113 is configured to output a cause of the defect and/or a countermeasure to the defect based on the predetermined image and the relationship model. Specific manners of output content will be described later.
  • (Relationship model creation unit 114)
    The relationship model creation unit 114 is configured to execute a relationship model creation step. In the relationship model creation step, the relationship model creation unit 114 creates or updates a relationship model to be used in the above-described output step or the like.
  • (Storage management unit 115)
    The storage management unit 115 is configured to execute a storage management step. In the storage management step, the storage management unit 115 is configured to manage various information to be stored that is associated with the information processing system 100 according to the present embodiment. Typically, the storage management unit 115 is configured to allow information handled by the information processing apparatus 1, etc. to be stored in a storage area. Examples of the storage area include the storage unit 12 of the information processing apparatus 1 or storage units of various apparatuses or terminals, but the storage area does not necessarily have to be within the information processing system 100, and the storage management unit 115 may manage various information to be stored in an external storage unit or the like as well.
  • 3. Detail of information processing
    In a third section, an information processing method executed by the information processing apparatus 1, etc., will be described with reference to an activity diagram, etc. FIG. 4 is an activity diagram showing a flow of information processing using the information processing apparatus 1, etc.
  • First, in the information processing method according to the present embodiment, the image acquisition unit 111 acquires a predetermined image from within the industrial process P (step S1).
  • The step S1 may typically be accomplished by the imaging apparatus 2 capturing a predetermined point of the industrial process P and the image acquisition unit 111 acquiring image data via a communication line. It should be noted that an object to be captured by the imaging apparatus 2 may be any object that is capturable, and may be either organic or inorganic. For instance, the object may be a predetermined equipment (including various equipment such as reaction vessel, tank, transport apparatus, etc.) present in the industrial process P, may be a raw material used in the industrial process P, or may be a product or an intermediate manufactured in the industrial process P.
  • In the information processing method according to the present embodiment, the image acquired in the step S1 includes an image with respect to the defect that occurred in the industrial process P.
  • The term "defect" refers to a state in which the industrial process P is not operating as designed or a resultant product thereof (in other words, a state that is not normal (abnormal state)). The information processing method according to the present embodiment may be applied to a case where there is an abnormality in the industrial process P or a case where the resultant product is a different image from that when the industrial process P is operating normally. Typical examples include when an equipment has a defect or fault, when a raw material or a product (intermediate product) has a different color or pattern than usual, or when a raw material or a product (intermediate product) has a different dimension than usual. In the information processing method according to the present embodiment, an image including such a defect is acquired by the imaging apparatus 2.
  • On the other hand, in the information processing method according to the present embodiment, the relationship model acquisition unit 112 acquires a relationship model (step S2). The order of performing step S1 and step S2 is arbitrary, step S1 may be performed prior to step S2, step S2 may be performed prior to step S1, or both steps may be performed simultaneously.
  • In step S2, the relationship model is a model created by associating analysis data obtained by analyzing a predetermined point in the industrial process P with image data related to an image of a corresponding defect. Here, the relationship model is a model of the relationship between the predetermined analysis data and the image data related to the image of the corresponding defect. The model may be, for example, a function or a look-up table showing the relationship between the predetermined analysis data and the image data related to the image of the corresponding defect, or may be a learned model in which the relationship between the predetermined analysis data and the image data related to the image of the corresponding defect has been learned.
  • Regarding such a relationship model, the relationship between the analysis data and the image data related to the image of the corresponding defect may be analyzed based on a known analysis method. Typically, regression analysis method (linear model, generalized linear model, generalized linear mixed model, ridge regression, Lasso regression, elastic net, support vector regression, projection tracking regression, etc.), time series analysis (VAR model, SVAR model, ARIMAX model, SARIMAX model, state space model, etc.), decision tree (decision tree, regression tree, random forest, XGBoost (Extreme Gradient Boosting), etc.), neural network (simple perceptron, multilayer perceptron, Transformer, DNN (Deep Neural Network), CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), LSTM (Long Short Term Memory), etc.), Bayes (naive Bayes, etc.), clustering (k-means, k-means++, etc.), ensemble learning (Boosting, Adaboost, etc.), or the like may be employed to analyze and obtain a desired prediction model.
  • In the present embodiment, a neural network is preferably to be employed among the above-mentioned analysis methods. Moreover, a software or a program for creating and operating the relationship model according to the present embodiment may be selected from among those that can use the neural network. For instance, framework such as Keras, TensorFlow, or PyTorch, or software such as Teachable Machine (Google LLC) or DataRobot (DataRobot, Inc.) may be used as well.
  • Although creation of the relationship model is exemplified as being performed by the relationship model creation unit 114, it is not necessarily limited thereto, and the information processing method according to the present embodiment may be performed by creating such a relationship model outside the information processing system 100 and installing the created relationship model in the information processing apparatus 1, etc. Further, the relationship model may be a model configured to be updated prior to the output unit 113 outputting the cause of the defect and/or the countermeasure to the defect (before the output step is executed). In this case, the relationship model creation unit 114 may be configured to update the relationship model sequentially in accordance with accumulation of information with respect to the countermeasure to the defect, etc.
  • The analysis data related to the relationship model does not necessarily have to be data obtained by analyzing the defect itself. To take an example of evaluating a defect in a product, the analysis data related to the relationship model need not be analysis data regarding the product itself but may be data analyzing manufacturing equipment or raw material used. On the other hand, the analysis data is preferably data obtained by analyzing a defect in the industrial process P as a predetermined point in the industrial process P. This latter manner makes it easier to evaluate a defect with higher accuracy.
  • The analysis data according to the present embodiment may be based on any known analysis method. As an example, the analysis data according to the present embodiment may be data obtained by analyzing a predetermined point with a chemical or biochemical method. In another viewpoint, the analysis data according to the present embodiment may include data with respect to on one or more analyses selected from a group comprising optical microscopic observation, electron microscopic observation, FT-IR analysis, pyrolysis gas chromatography analysis, ninhydrin reaction test, X-ray Fluorescence analysis, X-ray diffraction analysis, bacteria count analysis, fungi count analysis, element mapping analysis, iodine starch reaction test, extraction amount by organic solvent, extraction amount by acid, and extraction amount by alkali.
  • That is, in the present embodiment, compositional characteristics related to a defect may be estimated from such analysis data. Therefore, the cause or the countermeasure for the defect may be output with high accuracy from the image information acquired by the image acquisition unit 111. Note that in the case of paper manufacturing process, etc. (including water system) as illustrated above, the use of such analysis data tends to contribute to improving accuracy of an output result.
  • On the other hand, the image data associated with the relationship model may include information on any of color, size, and shape of the defect. The output unit 113 is preferably configured to output a cause of the defect and/or a countermeasure to the defect based on information on any of color, size, and shape of the defect in the predetermined image.
  • As described above, in the information processing system 100 according to the present embodiment, an RGB camera may be applied as the imaging apparatus 2, and color, size, and shape of the defect may be associated with RGB data of image information acquired by the RGB camera. That is, a combination of RGB data for each pixel may be used as a basis for outputting a cause, etc. in the output step described later.
  • Subsequently, the information processing method according to the present embodiment outputs the cause of the defect and/or the countermeasure to the defect based on the image acquired by the image acquisition unit 111 and the relationship model acquired by the relationship model acquisition unit 112 (step S3).
  • This step may be accomplished by a function of the output unit 113. Although the output unit 113 may output only one of the cause of the defect and the countermeasure to the defect, it is preferable that the output unit 113 outputs the cause of the defect and the countermeasure to the defect.
  • An example of content output by the output unit 113 will be described with reference to FIG. 5. FIG. 5 is an example of output content displayed by the output unit 113.
  • A display screen D in FIG. 5 is visual information output by the output unit 113 of the information processing apparatus 1, typically displayed as a screen on the display unit 14 of the information processing apparatus 1. The display screen D shows the image of the defect acquired by the image acquisition unit 111, along with a type of defect (type of fault), a cause, and a countermeasure of the defect, each of which is associated with the image of the defect.
  • The "type of defect" refers to a broad classification of defects (faults), and more detailed cause is indicated in an item "cause". In addition, in the output content shown in FIG. 5, an operating condition of the industrial process P is output as a countermeasure to the defect. (see an item "countermeasure" in FIG. 5).
  • Note that if the industrial process P is a paper manufacturing process, following are enumerated as operating conditions that serve as countermeasures.
  • (Countermeasure A) Application of water treatment chemicals, change of type, adjustment of addition amount, change of addition location, suspension of addition
    Examples of water treatment chemicals here include slime control agents, preservative, pitch control agent, defoamer, scale inhibitor, process cleaning agent (acidic agent, alkaline agent, chelating agent, etc.), yield filtration agent (include organic and inorganic), flocculation agent (include organic and inorganic), coagulant (include organic and inorganic), charge adjuster, degassing agent, cleaner aid, felt conditioner, roll stain inhibitor for paper machine, canvass stain inhibitor for paper machines, creping agent, evaporation aid, deinking agent or bleaching agent (surfactant, sodium hydroxide, sodium hypochlorite, chlorine dioxide, ozone, hydrogen peroxide, chelating agents, sodium silicate, etc.) beating aid, release aid, etc.
  • (Countermeasure B) Application of functional chemicals affecting water treatment, change of type, adjustment of addition amount, change of addition location, suspension of addition
    Examples of functional chemicals here include paper strength agent (include dry and wet), sizing agent, dye, sticky agent, filler (such as calcium carbonate and kaolin), etc.
  • (Countermeasure C) Application of water treatment equipment, change of type, change of operation method, adjustment of operation load, suspension of operation
    Examples of water treatment equipment here include pressurized flotation apparatus, coagulation sedimentation apparatus, colander, washing equipment such as filter and dehydrator, foreign matter removal apparatus such as screen, cleaner, and floater, etc. Further, change of operation method includes changing supplementary water amount or supplementary water kind to a system, changing amount of process water discharged from the system, etc.
  • (Countermeasure D) Application of production equipment affecting water treatment, change of type, change of operation method, adjustment of operation load, suspension of operation
    Examples of production equipment here include release machine, disperser, beating machine, paper machine, steaming equipment, bleaching equipment, etc. In addition, change of operation method includes changing type or blending ratio of raw material, production order of production item, timing of periodic repair, timing of cleaning inside system, etc.
  • Of course, the contents of the countermeasures are not limited to the above, but may be set as appropriate depending on type and scale of the industrial process P.
  • Furthermore, the content output by the output unit 113 is not limited to such a display screen, and the cause or the countermeasure is outputted as printed matter may be exemplified as an example. Moreover, as part of the countermeasure, the output unit 113 may output the operating condition of the industrial process P as a signal. In this case, a part of the equipment provided in the industrial process P may be configured to operate automatically according to the content of the signal.
  • The content output by the output unit 113 may be stored in a predetermined storage area by a function of the storage management unit 115. Further, the relationship model may be updated according to the output content stored in such a manner. Such updating of the relationship model may be realized by a function of the relationship model creation unit 114.
  • As described above, with the information processing method executed by the information processing apparatus 1 according to the present embodiment, a cause of a defect or the like may be easily specified.
  • 4. Variation
    In a fourth section, a variation of the information processing method executed by the information processing apparatus 1 or the like described above will be illustrated.
  • Although the above-described embodiment is described as a configuration of the information processing apparatus 1, a program that causes a computer to function as each unit of the information processing apparatus may be applied.
  • The embodiment described above shows an information processing method using a relationship model, but information associated when creating the relationship model is not limited thereto. That is, the relationship model used in the present embodiment may be associated with various other conditions such as weather condition, condition related to region, and condition related to equipment age.
  • In the above embodiment, although the information processing apparatus 1 performs various storage and controls, two or more external apparatuses may be applied instead of the information processing apparatus 1. In other words, image of defect, information on cause, countermeasure, etc. may be distributed and stored in two or more external apparatuses using blockchain technology or the like.
    (Example)
  • Hereinafter, the present invention will be described in further detail with reference to an example and a comparative example. Note that the present invention is not limited to the following examples.
  • (Creation and validation of relationship model)
    Various analyses (chemical analysis, etc.) are conducted on 23 defective samples from a paperboard production equipment to specify a main component of a defect and to estimate a cause and a water treatment method for a countermeasure. In addition, a database with a sample size of 23 is created by adding image data of the defect. Employing the database, a relationship model is created using supervised learning to estimate the cause suggested by the analyzed data from the image data of the defect. A procedure for creating the relationship model is to use Teachable Machine (Google LLC) to create a model with a deep neural network.
  • Next, to verify accuracy of the relationship model, 15 defective samples are prepared in addition to the 23 samples mentioned above and analyzed in the same manner. As a result of applying image data of the 15 samples to the relationship model and estimating a component and a cause of the defect, an estimated result and an actual analysis result are matched with high accuracy.
    This indicates that the information processing apparatus or the like according to the present invention may easily specify a cause of a defect.
  • Finally, although various embodiments of the present invention have been described, these are presented as examples and are not intended to limit the scope of the present invention. Other novel embodiments may be implemented in various forms, and various omissions, substitutions, or modification may be made within the scope of invention without departing from the spirit of the present invention. The embodiments or variations thereof are included in the scope and spirit of the present invention, as well as within the scope of the invention and equivalents thereof recited in the claims.
  • 1: Information processing apparatus
    2: Imaging apparatus
    3: Type of countermeasure
    4: Type of countermeasure
    10: Communication bus
    11: Controller
    12: Storage unit
    13: Input unit
    14: Display unit
    15: Communication unit
    100: Information processing system
    111: Image acquisition unit
    112: Relationship model acquisition unit
    113: Output unit
    114: Relationship model creation unit
    115: Storage management unit
    D: Display screen
    P: Industrial process

Claims (13)

  1. An information processing apparatus, comprising:
    an image acquisition unit configured to acquire a predetermined image from within an industrial process, the predetermined image including an image with respect to a defect occurred within the industrial process;
    a relationship model acquisition unit configured to acquire a relationship model, the relationship model being a model created by associating analysis data obtained by analyzing a predetermined point in the industrial process with image data with respect to an image of a corresponding defect; and
    an output unit configured to output a cause of the defect and/or a countermeasure to the defect based on the predetermined image and the relationship model.
  2. The information processing apparatus according to claim 1, wherein:
    the output unit is configured to output a cause of the defect and a countermeasure to the defect.
  3. The information processing apparatus according to claim 1 or 2, wherein:
    the analysis data is data obtained by analyzing the predetermined point using a chemical or biochemical method.
  4. The information processing apparatus according to claim 1 or 2, wherein:
    the analysis data includes data with respect to on one or more analyses selected from a group comprising optical microscopic observation, electron microscopic observation, FT-IR analysis, pyrolysis gas chromatography analysis, ninhydrin reaction test, X-ray Fluorescence analysis, X-ray diffraction analysis, bacteria count analysis, fungi count analysis, element mapping analysis, iodine starch reaction test, extraction amount by organic solvent, extraction amount by acid, and extraction amount by alkali.
  5. The information processing apparatus according to any one of claims 1 to 4, wherein:
    the analysis data is data obtained by analyzing the defect in the industrial process as a predetermined point in the industrial process.
  6. The information processing apparatus according to any one of claims 1 to 5, wherein:
    the industrial process is a process comprising a step of a water system.
  7. The information processing apparatus according to any one of claims 1 to 6, wherein:
    the industrial process is a paper manufacturing process.
  8. The information processing apparatus according to any one of claims 1 to 7, wherein:
    the image data associated with the relationship model includes information on any of color, size, and shape of the defect, and
    the output unit is configured to output a cause of the defect and/or a countermeasure to the defect based on information on any of color, size, and shape of the defect in the predetermined image.
  9. The information processing apparatus according to any one of claims 1 to 8, wherein:
    the predetermined image is an image captured by an RGB camera.
  10. The information processing apparatus according to any one of claims 1 to 9, wherein:
    the output unit is configured to output an operating condition of the industrial process as a countermeasure to the defect.
  11. The information processing apparatus according to any one of claims 1 to 10, wherein:
    the relationship model is a model configured to be updated prior to the output unit outputting the cause of the defect and/or the countermeasure to the defect.
  12. An information processing method executed by an information processing apparatus, comprising:
    an image acquisition step of acquiring a predetermined image from within an industrial process, the predetermined image including an image with respect to a defect occurred within the industrial process;
    a relationship model acquisition step of acquiring a relationship model, the relationship model being a model created by associating analysis data obtained by analyzing a predetermined point in the industrial process with image data with respect to an image of a corresponding defect; and
    an output step of outputting a cause of the defect and/or a countermeasure to the defect based on the predetermined image and the relationship model.
  13. A program allowing a computer to function as:
    an image acquisition unit configured to acquire a predetermined image from within an industrial process, the predetermined image including an image with respect to a defect occurred within the industrial process;
    a relationship model acquisition unit configured to acquire a relationship model, the relationship model being a model created by associating analysis data obtained by analyzing a predetermined point in the industrial process with image data with respect to an image of a corresponding defect; and
    an output unit configured to output a cause of the defect and/or a countermeasure to the defect based on the predetermined image and the relationship model.
EP24712603.0A 2023-03-24 2024-02-27 Information processing apparatus, information processing method and program Pending EP4659192A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2023048188A JP2024136890A (en) 2023-03-24 2023-03-24 Information processing device, information processing method, and program
PCT/JP2024/007174 WO2024202839A1 (en) 2023-03-24 2024-02-27 Information processing apparatus, information processing method and program

Publications (1)

Publication Number Publication Date
EP4659192A1 true EP4659192A1 (en) 2025-12-10

Family

ID=90366553

Family Applications (1)

Application Number Title Priority Date Filing Date
EP24712603.0A Pending EP4659192A1 (en) 2023-03-24 2024-02-27 Information processing apparatus, information processing method and program

Country Status (6)

Country Link
EP (1) EP4659192A1 (en)
JP (1) JP2024136890A (en)
KR (1) KR20250133371A (en)
CN (1) CN120770041A (en)
TW (1) TW202445388A (en)
WO (1) WO2024202839A1 (en)

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4876094B2 (en) 2008-03-31 2012-02-15 ソマール株式会社 Slime control method and apparatus
JP2020123284A (en) * 2019-01-31 2020-08-13 三菱重工機械システム株式会社 Inspection support device and method
JP2022139155A (en) * 2021-03-11 2022-09-26 オムロン株式会社 Information processing device, method, and program
JP7390085B2 (en) * 2021-09-07 2023-12-01 株式会社メンテック Defect classification system

Also Published As

Publication number Publication date
TW202445388A (en) 2024-11-16
WO2024202839A1 (en) 2024-10-03
JP2024136890A (en) 2024-10-04
CN120770041A (en) 2025-10-10
KR20250133371A (en) 2025-09-05

Similar Documents

Publication Publication Date Title
KR102321765B1 (en) Display device quality inspection method, device, electronic device and storage medium
Zhou et al. Differential feature based hierarchical PCA fault detection method for dynamic fault
Zhou et al. Gui-g1: Understanding r1-zero-like training for visual grounding in gui agents
KR20230127931A (en) Defect detection system, method and storage medium for display device
Jiang et al. Canonical variate analysis-based monitoring of process correlation structure using causal feature representation
JP2017058838A (en) Information processing apparatus, test system, information processing method, and program
US20230005120A1 (en) Computer and Visual Inspection Method
WO2024202839A1 (en) Information processing apparatus, information processing method and program
JP7118365B2 (en) Image inspection device
Van Nguyen et al. Semiconductor probe card proactive maintenance using graph self-supervised learning and an empirical study
JP7437610B2 (en) Image inspection device, image inspection method, and trained model generation device
Shetty Vision-based inspection system employing computer vision & neural networks for detection of fractures in manufactured components
Azeroual Beyond Black Boxes: Adaptive XAI for Dynamic Data Pipelines
CN118827970B (en) A method and system for detecting and analyzing flat-panel televisions based on intelligent feedback
Suwattananuruk et al. Denoising variational autoencoders for smart inspection of wafer probe card PCB channels for advancing quality control for semiconductor manufacturing
KR102844783B1 (en) A proxy interpreter to upgrade automated legacy systems
Kavitha et al. Deep Learning Based Product Defect Detection for Sustainable Smart Manufacturing
US20230274409A1 (en) Method for automatic quality inspection of an aeronautical part
Schiele et al. Low-shot visual anomaly detection with multimodal large language models
CN119677693A (en) Method for operating waste liquid treatment equipment, method for predicting properties of treated waste liquid, operating system and prediction system
Prioli et al. Self-adaptive production performance monitoring framework under different operating regimes
WO2026023253A1 (en) Information processing system, information processing method, and program
WO2026023228A1 (en) Information processing system, information processing method, and program
KR101303866B1 (en) Method of monitoring substrate processing apparatus
JP7439157B2 (en) Inspection equipment

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20250901

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR