KR102543064B1 - System for providing manufacturing environment monitoring service based on robotic process automation - Google Patents

Abstract

Provided is a system for providing a manufacturing environment monitoring service based on robotic process automation (RPA), which comprises: at least one monitoring terminal which collects at least one manufacturing environment data from facilities and processes in a manufacturing environment; a user terminal which outputs a normal state or an abnormal state of facilities or processes with a result of collecting and analyzing at least one manufacturing environment data; and a monitoring service providing server which comprises a collection unit using the RPA to collect at least one manufacturing environment data collected from facilities and processes in the manufacturing environment from at least one monitoring terminal, a state analysis unit allowing the RPA to analyze at least one manufacturing environment data and inform the user terminal of whether the state is normal or abnormal, and a visualization unit visualizing the normal state or the abnormal state to output on the user terminal. According to the present invention, users can intuitively configure artificial intelligence flows.

Description

RPA-based manufacturing environment monitoring service provision system {SYSTEM FOR PROVIDING MANUFACTURING ENVIRONMENT MONITORING SERVICE BASED ON ROBOTIC PROCESS AUTOMATION}

The present invention relates to an RPA-based manufacturing environment monitoring service providing system, which monitors the manufacturing environment to inform normal and abnormal conditions, performs AI predictive maintenance that detects abnormalities or predicts fixed defects, and uses RPA without program coding. It provides a solution that can create, change, and delete artificial intelligence algorithms using

As the digital transformation of companies accelerates along with the 4th industrial revolution, interest in RPA (Robotic Process Automation) as a tool for process automation is hot. RPA is a technology that automates simple, rule-based tasks that humans repeatedly perform using information systems through software robots. Since the 2010s, many companies at home and abroad have adopted RPA, and the spread of solid development platforms is promoting the spread of RPA. The RPA project analyzes the business processes to be automated through qualitative methods such as workshops, interviews, and observations in the process analysis and design stage, and writes them in Process Design Documents. In addition, when RPA performs simple, repetitive, standardized tasks performed by humans instead, it is further enhancing the scope and effect by combining various technologies such as AI (Artificial Intelligence) and OCR (Optical Character Recognition). RPA, an automation technology using software robots, and hyperautomation, a combination of RPA and AI, are expected to automate simple and repetitive business tasks.

At this time, a method of analyzing the RPA performance process or learning the user's process by machine learning to drive the RPA according to the learned process has been researched and developed. Published on January 18, 2022) and Korean Patent Publication No. 2022-0115202 (published on August 17, 2022), the user terminal analyzes the code of the RPA application or outputs the result of visualizing the rules, and the RPA process Analyze and display, and if the RPA violates the control and rules as a result of the analysis, the configuration to stop the control and process of the RPA, and the operation of the user terminal in the RPA, and the monitored result using the machine learning of the RPA A configuration for performing learning and verification to be implemented in, and proceeding with the operation of the RPA according to the learned result is disclosed.

However, in the former case, if RPA proceeds with a process that violates rules or policies, it only initiates a configuration that stops it. However, the object recognition recording module cannot recognize all of the keyboard log or mouse log output on the actual user screen, and a delay for recognition occurs, interfering with the user's work. In addition, even if the process definition is prepared through a qualitative method, it cannot faithfully reflect various exceptions and errors that may occur in actual work, so the bot code must be continuously modified in the development stage, and the process definition in the test stage You have to spend a lot of time dealing with exceptions and errors that are not reflected in the definition. In addition, exceptions and errors that occur during the maintenance phase can cause work disruption, which can lead to reluctance to use RPA. Therefore, AI predictive maintenance is used to predict and notify equipment failures that occur in manufacturing facilities based on RPA in advance, and even if users are not RPA experts, they can intuitively create AI flows and learn, verify, and relearn AI. Research and development of the platform is required.

In one embodiment of the present invention, RPA (Robotic Process Automation) receives at least one manufacturing environment data collected from facilities and processes in the manufacturing environment, guides the user terminal whether it is in a normal state or abnormal state, and artificial intelligence model It uses an artificial intelligence predictive maintenance method that informs equipment or process failures in advance by predicting abnormal signs or defects occurring from equipment or processes using Therefore, users can create, modify, change, and delete artificial intelligence flows just by dragging and dropping, and artificial intelligence models are re-learned at a preset cycle, so that even non-majors can develop, maintain and delete artificial intelligence models without coding. It is possible to provide an RPA-based manufacturing environment monitoring service providing system that can even be repaired. However, the technical problem to be achieved by the present embodiment is not limited to the technical problem as described above, and other technical problems may exist.

As a technical means for achieving the above technical problem, an embodiment of the present invention, at least one monitoring terminal for collecting at least one manufacturing environment data from facilities and processes in the manufacturing environment, at least one collection of manufacturing environment data And at least one monitoring terminal for at least one manufacturing environment data collected from equipment and processes in the manufacturing environment using a user terminal and RPA (Robotic Process Automation) that outputs the normal or abnormal state of the equipment or process as a result of the analysis. A collection unit that collects at least one manufacturing environment data, a state analysis unit that analyzes at least one manufacturing environment data and informs the user terminal of whether it is a normal state or an abnormal state, and a visualization that visualizes and outputs the normal state or abnormal state from the user terminal and a monitoring service providing server including a unit.

According to any one of the above-described problem solving means of the present invention, RPA (Robotic Process Automation) receives at least one manufacturing environment data collected from facilities and processes in the manufacturing environment and determines whether it is in a normal state or an abnormal state to a user terminal It uses an artificial intelligence predictive maintenance method that notifies failures in facilities or processes in advance by predicting abnormal symptoms or defects occurring from facilities or processes using artificial intelligence models, and users can intuitively configure artificial intelligence flows. By visualizing each step so that users can create, modify, change, and delete artificial intelligence flows only by drag & drop, and artificial intelligence models are retrained at preset cycles, even non-majors can use no-coding (No-Coding). ) method to develop, maintain, and even repair artificial intelligence models.

1 is a diagram for explaining an RPA-based manufacturing environment monitoring service providing system according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating a monitoring service providing server included in the system of FIG. 1 .
3 and 4 are diagrams for explaining an embodiment in which an RPA-based manufacturing environment monitoring service is implemented according to an embodiment of the present invention.
5 is an operational flowchart for explaining a method for providing an RPA-based manufacturing environment monitoring service according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail so that those skilled in the art can easily practice the present invention with reference to the accompanying drawings. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected" but also the case where it is "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, this means that it may further include other components, not excluding other components, unless otherwise stated, and one or more other characteristics. However, it should be understood that it does not preclude the possibility of existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

As used throughout the specification, the terms "about", "substantially", etc., are used at or approximating that value when manufacturing and material tolerances inherent in the stated meaning are given, and do not convey an understanding of the present invention. Accurate or absolute figures are used to help prevent exploitation by unscrupulous infringers of the disclosed disclosure. The term "step of (doing)" or "step of" as used throughout the specification of the present invention does not mean "step for".

In this specification, a "unit" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Further, one unit may be realized using two or more hardware, and two or more units may be realized by one hardware. On the other hand, '~ unit' is not limited to software or hardware, and '~ unit' may be configured to be in an addressable storage medium or configured to reproduce one or more processors. Thus, as an example, '~unit' refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, and procedures. , subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. Functions provided within components and '~units' may be combined into smaller numbers of components and '~units' or further separated into additional components and '~units'. In addition, components and '~units' may be implemented to play one or more CPUs in a device or a secure multimedia card.

In this specification, some of the operations or functions described as being performed by a terminal, device, or device may be performed instead by a server connected to the terminal, device, or device. Likewise, some of the operations or functions described as being performed by the server may also be performed by a terminal, apparatus, or device connected to the server.

In this specification, some of the operations or functions described as mapping or matching with the terminal mean mapping or matching the terminal's unique number or personal identification information, which is the terminal's identifying data. can be interpreted as

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram for explaining an RPA-based manufacturing environment monitoring service providing system according to an embodiment of the present invention. Referring to FIG. 1 , the RPA-based manufacturing environment monitoring service providing system 1 may include at least one user terminal 100, a monitoring service providing server 300, and at least one monitoring terminal 400. However, since the RPA-based manufacturing environment monitoring service providing system 1 of FIG. 1 is only an embodiment of the present invention, the present invention is not limitedly interpreted through FIG. 1 .

At this time, each component of FIG. 1 is generally connected through a network (Network, 200). For example, as shown in FIG. 1 , at least one user terminal 100 may be connected to the monitoring service providing server 300 through the network 200 . In addition, the monitoring service providing server 300 may be connected to at least one user terminal 100 and at least one monitoring terminal 400 through the network 200 . In addition, at least one monitoring terminal 400 may be connected to the monitoring service providing server 300 through the network 200 .

Here, the network refers to a connection structure capable of exchanging information between nodes such as a plurality of terminals and servers, and examples of such networks include a local area network (LAN) and a wide area network (WAN: Wide Area Network), the Internet (WWW: World Wide Web), wired and wireless data communications networks, telephone networks, and wired and wireless television communications networks. Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), 5th Generation Partnership Project (5GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi , Internet (Internet), LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), RF (Radio Frequency), Bluetooth (Bluetooth) network, NFC ( A Near-Field Communication (Near-Field Communication) network, a satellite broadcasting network, an analog broadcasting network, a Digital Multimedia Broadcasting (DMB) network, etc. are included, but not limited thereto.

In the following, the term at least one is defined as a term including singular and plural, and even if at least one term does not exist, each component may exist in singular or plural, and may mean singular or plural. It will be self-evident. In addition, the singular or plural number of each component may be changed according to embodiments.

At least one user terminal 100 may be a terminal that outputs an abnormal state or a normal state of a facility or process in a manufacturing environment using a web page, app page, program, or application related to an RPA-based manufacturing environment monitoring service. In addition, the user terminal 100 may be a terminal that performs creation, editing, modification, deletion, etc. of an artificial intelligence algorithm without coding using RPA. To this end, the user terminal 100 may be a terminal that creates, modifies, and deletes a flow including at least one step performed in the RPA. At this time, the user terminal 100 is provided with at least one activity corresponding to at least one execution code, and drags and drops the at least one activity to add, change, or delete steps of the flow. It may be a terminal that changes or creates a flow without coding.

Here, at least one user terminal 100 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, a laptop, a desktop, a laptop, and the like equipped with a navigation system and a web browser. In this case, at least one user terminal 100 may be implemented as a terminal capable of accessing a remote server or terminal through a network. At least one user terminal 100 is, for example, a wireless communication device that ensures portability and mobility, navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet ) may include all types of handheld-based wireless communication devices such as terminals, smartphones, smart pads, tablet PCs, and the like.

The monitoring service providing server 300 may be a server providing an RPA-based manufacturing environment monitoring service web page, app page, program or application. In addition, the monitoring service providing server 300 allows the RPA to collect the manufacturing environment data collected from the process or facility, and the RPA inputs the manufacturing environment data to the artificial intelligence algorithm as a query and returns the result to the user terminal 100 It may be a server that allows transmission. In addition, the monitoring service providing server 300, even if the user of the user terminal 100 is not a developer or programmer who developed RPA, can perform tasks such as changing, deleting, or adding steps in a flow with an intuitive GUI It may be a terminal that allows In addition, the monitoring service providing server 300 may be a server that allows the RPA to proceed with the process of pre-processing and refining the manufacturing environment data input to the artificial intelligence algorithm. Accordingly, when trying to build a new artificial intelligence algorithm in the user terminal 100, the monitoring service providing server 300 automatically creates a dataset without hiring a separate labeler or manually preprocessing and labeling the user. It can be a server that builds and sets up modeling in such a way that artificial intelligence algorithms learn and verify to operate according to the sequence set by the user. In addition, the monitoring service providing server 300 may be a server that performs AI predictive maintenance to detect anomalies or predict defects by analyzing manufacturing environment data with an artificial intelligence algorithm.

Here, the monitoring service providing server 300 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, a laptop, a desktop, a laptop, and the like equipped with a navigation system and a web browser.

At least one monitoring terminal 400 detects and monitors various manufacturing environment data generated from processes and facilities using a web page, app page, program, or application related to the RPA-based manufacturing environment monitoring service, It may be a terminal that transmits manufacturing environment data at a point in time.

Here, at least one monitoring terminal 400 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, a laptop, a desktop, a laptop, and the like equipped with a navigation system and a web browser. In this case, at least one monitoring terminal 400 may be implemented as a terminal capable of accessing a remote server or terminal through a network. At least one monitoring terminal 400 is, for example, a wireless communication device that ensures portability and mobility, navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet ) may include all types of handheld-based wireless communication devices such as terminals, smartphones, smart pads, tablet PCs, and the like.

2 is a block diagram illustrating a monitoring service providing server included in the system of FIG. 1, and FIGS. 3 and 4 show an embodiment in which an RPA-based manufacturing environment monitoring service is implemented according to an embodiment of the present invention. It is a drawing for explanation.

Referring to FIG. 2, the monitoring service providing server 300 includes a collection unit 310, a state analysis unit 320, a visualization unit 330, an AI predictive maintenance unit 340, a relearning unit 350, a furnace It may include a coding unit 360, an AI correction unit 370, an automatic AI generation unit 380, a classification unit 390, and a hybrid unit 391.

The monitoring service providing server 300 according to an embodiment of the present invention or another server (not shown) operating in conjunction with at least one user terminal 100 and at least one monitoring terminal 400 monitors the RPA-based manufacturing environment In the case of transmitting a service application, program, app page, web page, etc., at least one user terminal 100 and at least one monitoring terminal 400, the RPA-based manufacturing environment monitoring service application, program, app page, web page etc. can be installed or opened. In addition, a service program may be driven in at least one user terminal 100 and at least one monitoring terminal 400 by using a script executed in a web browser. Here, the web browser is a program that allows users to use the web (WWW: World Wide Web) service, and means a program that receives and displays hypertext described in HTML (Hyper Text Mark-up Language). For example, Netscape , Explorer, Chrome, and the like. In addition, an application means an application on a terminal, and includes, for example, an app running on a mobile terminal (smart phone).

Referring to FIG. 2 , the collection unit 310 collects, from at least one monitoring terminal 400, at least one manufacturing environment data collected from facilities and processes in the manufacturing environment using RPA (Robotic Process Automation). can do. Manufacturing environment data may include equipment data including vibration, current, vision, temperature, and noise collected from equipment, and process data including production conditions, product information, equipment status, and control conditions collected from processes. It is not limited to those listed and is not excluded for reasons not listed.

At this time, the at least one monitoring terminal 400 may be a device or terminal that collects at least one manufacturing environment data from facilities and processes in the manufacturing environment, for example, a sensor for measuring temperature or humidity, each facility or Cameras filming the process, MEMS (Micro Electro Mechanical Systems) integrated with SoC (System on Chip) to collect data of each equipment or facility, infrared image sensor, acceleration sensor, pressure sensor, 9-axis motion sensor, microphone, magnetic It may be a sensor, a proximity sensor, a capacitive sensor, an inclination sensor, an ultrasonic sensor, a photo sensor, a vision sensor, a safety sensor, or an optical sensor. At this time, the optical sensor is an active sensor that detects an object by calculating the time difference, angle difference, etc. returned after shooting light, or a passive sensor that detects an object by trigonometry using two or more sensors. can include

A stereo camera, a Time of Flight (ToF) camera, and a structured light camera can be used as the 3D image sensor. The stereo camera creates a 3D image with two cameras, and the ToF camera calculates the time it takes for the shot light to return, calculates the distance data, and converts it into a 3D image. The structured light camera uses a camera and a projector, and searches for a coincidence point of structured light codes in the camera image and the projector image, and obtains the 3D coordinates of the point. In addition, it may further include LiDAR, which detects distance, direction, speed, temperature, material distribution and concentration characteristics by emitting a pulse laser to a target and measuring the time and intensity it takes for the light to return. However, it is not limited to those listed and is not excluded for reasons not listed.

For example, if the system monitors CNC (Computer Numerical Control) facilities, it may be configured with CNC (Computer Numerical Control) facilities, the monitoring terminal 400, a data collection cloud server, a user program, and the like. It is assumed that a monitoring terminal 400 capable of collecting data by installing power, temperature, and vibration sensors is installed in the CNC equipment. The monitoring terminal 400 transmits the power value, temperature, and vibration detection measurement values per second detected through the sensor to the data collection cloud server through WIFI whenever a product is produced. Since the RPA must collect the manufacturing environment data received from the monitoring terminal 400 in the database without loss for analysis of the collected data, the device MAC address, board channel, sensor NoSQL database system, which is related source data, by receiving data from multiple devices that receive sensor type, data value, data type, and status code data in a 1-second period through POST communication method can be stored in

RPA, data reception and storage function for data collection uses raw data to store device MAC address, device port (channel) number, sensor type, sensor measurement value, and reception date and time in RDB, To facilitate distributed storage and expansion, it can be stored in the database in a structure that specifies data as an independent RDB table for each sensor of each device in 1-minute increments. The cloud server checks the received data and stores the device name, serial number, date and time, measurement value, etc. in the database. It is displayed in the form of a dashboard, and the overload value of the facility set as an analysis result according to the product is detected in real time and transmitted to the user terminal 100 so that the facility operation can be maintained normally.

RPA plays the role of data gathering, which collects a large amount of data at the same time, the role of data processing, which periodically imports manufacturing environment data into data gathering, processes it, and stores it in a database. role can be fulfilled.

data gathering data processing data selection Manufacturing environment data collection and storage Processing and analysis of manufacturing environment data Manufacturing data visualization

The state analysis unit 320 may allow the RPA to analyze at least one piece of manufacturing environment data and inform the user terminal 100 whether or not the manufacturing environment is in a normal state or an abnormal state. The user terminal 100 may output a normal state or an abnormal state of a facility or process as a result of collecting and analyzing at least one piece of manufacturing environment data. For example, if the normal range of the current data is up to [X-Y] and the error range is within 10%, the current data exceeding this range can be determined as the current data in the abnormal range. Exceeding alone does not immediately inform an abnormal state, but an abnormal state can be announced when an abnormal current is combined with other conditions that may affect each facility or process. Of course, it is self-evident that an abnormal state is not immediately excluded even when the numerical value is exceeded. The visualization unit 330 may visualize and output a normal state or an abnormal state in the user terminal 100 . Each data can be visualized in the form of an infographic on the DashBoard. The dashboard displays all facilities of the plant and monitors the data collected in real time, such as PoP and SPC, etc., such as the sensor update cycle, the characteristics and display of data values analyzed according to the conditions to be processed and displayed. It is possible to visualize by distinguishing a kind of pattern, such as period to be performed, positioning position in the monitor, and data classification for always displaying or spot displaying. In addition, after classifying information according to importance and priority, it is possible to compose various templates that match the characteristics of the industry. Facility information shows facility name, facility status, real-time collection data, production information, and detection information, and can output real-time power values of all facilities and vibration detection information. At this time, the facility ID, facility title, facility creator, creation date, sensor interlocking key, modification date, facility state, facility description, etc. may be mapped and stored.

The AI predictive maintenance unit 340 inputs the manufacturing environment data collected by the RPA as a query to at least one artificial intelligence algorithm, and detects anomalies in the equipment or process in the manufacturing environment data or outputs a defect prediction as a result and can be transmitted to the user terminal 100. At this time, Predictive Maintenance is a technology that predicts the possibility of failure and remaining lifespan of factory facilities and equipment based on artificial intelligence algorithms. Data-based analysis of vibrations and currents generated while equipment and facilities are operating is used to check abnormal signs in real time and notify the manager, so that measures can be taken before facilities fail, resulting in downtime Damage caused by a drop in utilization rate can be reduced. At this time, the step of predictive maintenance is standardized, and may be shown in Table 2, for example. To this end, after performing modeling by processing each manufacturing environment data in real time, predictive analysis is performed using artificial intelligence algorithms, and diagnosis and action are taken by predicting repair and replacement times after real-time monitoring.

step detail Step 1 data acquisition - Detection of abnormal signs Step 2 Diagnosis -Diagnosing whether there is a defect and the cause of the defect
- Severity Classification Step 3 prognosis -Prediction of future defects
-Prediction of remaining useful life Step 4 present and act - Suggestions and measures to solve problems Step 5 post hoc analysis -Check problem solving
-Post condition diagnosis

For example, process data such as pressure, temperature, and pH level represent the security of the physical manufacturing process. Analyzing control process data is to detect anomalies by analyzing these process data. Processes such as detecting anomalies by making predictions after analyzing process data or predicting the results of control commands through analysis of control commands may be performed. The re-learning unit 350 may generate a learning dataset based on the collected manufacturing environment data and the response of the user terminal 100 to automatically re-learn at least one artificial intelligence model at a preset cycle. For example, by using the manufacturing environment data accumulated from Monday to Friday on a weekly cycle, it is possible to make it work during the non-running time of the factory, for example, Saturday and Sunday.

The no-coding unit 360 generates, modifies, and deletes a flow including at least one step that proceeds in the RPA, and at least one corresponding to at least one execution code. An activity may be provided, and when at least one activity is inserted into a step in a flow using a drag and drop method, the RPA may be driven according to the changed flow. For example, let's assume that A, an employee of a fish cake manufacturer that mass-produces fish cake, wants to identify abnormal signs of fryer equipment that fry fish cake among manufacturing facilities. Assume that employee A is an ordinary person, not a coding expert. At this time, employee A has to select an artificial intelligence algorithm to identify anomalies, but this process is already difficult. You don't know which artificial intelligence algorithm to choose and what data to input. At this time, RPA holds artificial intelligence algorithms and dataset types that can identify anomalies in each manufacturing facility in advance, and even if the user enters only the data of [Fryer] and [Abnormal signs], the type of dataset to be input and It selects an artificial intelligence algorithm that can learn and verify it. In this way, if the user inputs only [Fryer] and [Abnormal symptom] and presses the Create button, the user's task is finished, and the flow for the artificial intelligence algorithm is output as shown in FIG. 3D.

At this time, the type of data set to identify the fryer abnormality is the camera photographing the fryer, the current output from the fryer, and the vibration of the fryer. Even if the user does not collect all of them, each manufacturing environment data includes the type of facility, , Since the type of detected data and the detected value are all tagged, it is in a labeled state, so after collecting this data from the database, an artificial intelligence algorithm for image analysis (algorithm #1), to determine whether the vibration data is abnormal loading an artificial intelligence algorithm (algorithm #2) that can detect anomalies in current data and an artificial intelligence algorithm (algorithm #3) that can determine whether current data is abnormal, and data suitable for each artificial intelligence algorithm, that is, algorithm #1 Image data, vibration data in Algorithm #2, and current data in Algorithm #3 are input, respectively, so that the artificial intelligence algorithm (Algorithm #1, #2, #3) performs learning and verification to be modeled. Then, when the learning and verification are completed, the flow created by the user, that is, information about whether the fryer is out of order is output to the user terminal 100 as shown in FIG. 3G. The artificial intelligence algorithm created in this way continuously informs the user of the results of monitoring the fish cake fryer and predicts whether or not it will fail. It continues until the user deletes this artificial intelligence algorithm. Each cell is a step. If you want to get rid of the AI algorithm itself, you can delete all cells after dragging. If you want to delete only one step, select a cell with the mouse and press the DEL button to delete it. If you want to add a step, you can add it from the list of activities.

The no-coding unit 360 selects a cell corresponding to at least one step so that the order of at least one step is changed and then drops it between different steps, so that the RPA operates according to the changed order You can change the order of executable code. For example, suppose there is a flow of [A→B→C→D→E]. Flow consists of each step. Assuming that the user wants to change step B to B', select an activity in the activity list corresponding to B' and drag and drop it to the cell where step B exists, and the user's work is done. Even a person who does not even know the C language can complete the maintenance work of modifying the flow of [A→B→C→D→E] to the flow of [A→B'→C→D→E]. At this time, the activity list refers to the area indicated by ② shown on the left side of FIG. 3D. Although it is described in text format in ②, it is possible to intuitively select what kind of work to add by overlaying a guide or explanation or graphic GUI. Alternatively, you can select an activity in the form of a keyword search. And, the flow means the region ③ shown on the right side of FIG. 3D. That is, the steps in the flow can be easily changed by overwriting (changing) the activity in the list in the area ② on the left of FIG. 3D or inserting (adding) the cell corresponding to the activity in the area ③ on the right. At this time, the cell means a box in which each activity is written in the region ③ of FIG. 3D.

The AI modification unit 370 can change, add, and delete the driving order of artificial intelligence algorithms when the user terminal 100 uses RPA to change, add, or delete the order of artificial intelligence algorithms. . At this time, since the user is not an expert, it is possible to guide whether this change, addition, or deletion is possible. For example, if you want to delete the [AI drive] step in the flow of [Data input-AI drive-Data output], you should inform them not to delete it. This is because if the artificial intelligence algorithm is missing, it is like the device to process the input data is missing, and it does not work. For example, if the noodle maker is not turned in the process of [injecting flour - driving noodle maker - outputting noodles], flour cannot be made into noodles. By providing such guidance, information on what the user wants to delete intuitively can be presented as an icon, infographic, or text.

The automatic AI generator 380, when arranging activities corresponding to at least one executable code in order using RPA in the user terminal 100, generates at least one artificial intelligence algorithm to be driven according to the activity At least one manufacturing environment data corresponding to the established flow is built into a dataset (DataSet), and after performing modeling that goes through learning and verification for the built dataset, the user selects the most accurate artificial intelligence algorithm and creates it. It can be adopted and set as an artificial intelligence algorithm that performs a flow.

Continuing the example of the fish cake manufacturing plant described above, if employee A thinks that the failure of the fish cake fryer is affected not only by vibration and current but also by temperature and humidity, [temperature], [humidity], [fryer ] and [anomaly], RPA extracts an artificial intelligence algorithm that can predict anomalies by interpreting temperature and humidity. Of course, for this, when RPA is built in a fish cake manufacturing plant, the artificial intelligence algorithm and data set type for each keyword must be defined, set, and built in advance in the definition. For each type of data, for example, it is known which artificial intelligence algorithm should be applied depending on whether it is structured data, semi-structured data, or unstructured data, and whether it is text data, numeric data, video data, or data If an artificial intelligence algorithm is specified in advance depending on the type, RPA works hard to supply a dataset so that the artificial intelligence algorithm can become smart, informs the user of the result, and the artificial intelligence algorithm learns more to improve accuracy. Newly collected data can be fed to artificial intelligence algorithms so that they can go higher and become smarter.

The process of attaching a label to each data, for example, a video of a fish cake fryer, or inserting temperature data and humidity data into an artificial intelligence algorithm, is a simple, repetitive task that requires manpower. RPA takes care of this process. As shown in Figure 3b, the red line is the task that RPA does. It stores manufacturing site data in a database, inputs it to an artificial intelligence algorithm to help the artificial intelligence algorithm do its job, and stores the results again in the database, RPA is responsible for notifying the result to the human user and delivering the data to the dashboard so that the result can be visualized. Accordingly, the temperature and humidity data are also automatically imported from the database by RPA and labeled, or if they are previously labeled, they are imported and used as they are. After the artificial intelligence algorithm is modeled to be learned and verified, it predicts abnormalities using the input temperature and humidity, and as a result, RPA collects information on whether or not it really malfunctions, and then automatically re-learn

The classification unit 390 may store text data among manufacturing environment data in a database, and store image data corresponding to an image or video among manufacturing environment data in a preset folder. Text data can also be classified according to whether it is numbers or text used by ordinary people. In the case of the former, artificial intelligence algorithms that process numbers, and in the case of texts used by humans, through natural language processing (NLP) and natural language understanding (NLU) Since converting into understandable machine language is required, these two can be stored separately. In the case of images or videos, vision-based deep learning must be used separately, so they can be stored separately from text. Accordingly, the hybrid unit 391 may establish hybrid artificial intelligence by setting at least one artificial intelligence algorithm for processing text data and at least one artificial intelligence algorithm for processing image data, respectively.

<automatic generation of RPA definition>

One embodiment of the present invention can create RPA to follow the tasks automatically performed by the user. For example, when user A occurs event B, it goes through [C-D-E] and outputs F. If this task is a standardized task, it follows user A’s action as it is [occurs event B-performs C-D-E-outputs F] RPA definition can be created. To do this, it is necessary to follow up all of the user's logs. The user may shop or chat with co-workers during business hours, so it is necessary to distinguish between them. In the case of using the automatic module disclosed in the prior art, all logs are not collected, so detailed Since problems occur that must be specified by a person, it is necessary to collect and classify logs.

RPA definition automatic generation module is installed in the user terminal 100, and the task of collecting the keyboard/mouse event data generated by the user terminal 100 and the screen capture image at the time the corresponding event occurs is defined as ① log collection. Log collection collects detailed logs of which keyboard buttons were pressed and which coordinates were clicked along with screen capture images. Then, ② data processing and refinement are performed. The similarity between the collected images is measured, the images are grouped, and the grouping results are reflected in the event data. Next, create a process event log that processes data processed through log conversion into event data, which is an input data format for process mining. At this time, the process event log creation task is responsible for creating the properties of case, activity, and timestamp. ③ The result of the work is that a log suitable for process mining analysis is created as an output. This result can be used as an input for professional process mining analysis tools such as ProM and Disco, and as a result, it can be used to implement definitions in the analysis and design stages, which are the first and second stages of the RPA project life cycle. It collects all behavioral logs generated by actual users when performing their tasks, and creates a process log in a form that can be visualized and analyzed, enabling more detailed and clear analysis and design than qualitative methods such as general interviews and observations. do.

<Non-work log detection>

Since all keyboard and mouse events generated from the user terminal 100 and task performance screen images are collected, there is a high possibility that the collected logs include not only business process-related events but also non-business-related events and screen logs. . Therefore, it is necessary to detect non-business logs that do not correspond to actual business from the collected logs. For example, non-business logs can include all tasks that do not correspond to business processes, such as Internet search, shopping, and SNS, and are not subject to RPA. It can also be classified as a non-business log. These non-business logs should be identified and removed because they cause deterioration in the quality of collected logs and high business process variability.

To this end, logo recognition can be used, which detects non-business images by checking whether a particular logo is present in the image. In addition, a similarity measurement algorithm for image grouping may be used as a similarity measurement method. In order to classify similar images, an image may be converted into a hash value by using an average hash function. After reducing the image and converting the color to grayscale, the average of each pixel is calculated, and if the brightness of the pixel is greater than the average, input 1, otherwise output as an array. In addition, similarity between images converted into comparable formats may be measured using a normalized distance algorithm. If the normalized distance between the two images is equal to or less than a predetermined distance, they are classified as the same task. A template matching algorithm can also be used to determine whether a non-business logo is included in a business image. This template matching algorithm uses a correlation coefficient.

<Create process event log>

First of all, cases need to be separated. In the business process log that does not divide cases, a meaningless business process map that is continuously performed is obtained because individual process instances are not separated. At this time, it is necessary to separate cases based on the start screen. When the screen corresponding to the start of work is recorded, it is determined that the previous work has ended and a new work has started. The case separation function can be implemented with a very simple loop. The task performance image file is called and the similarity between the image given as the task start screen and the pre-stored task start screen is measured, and if the similarity does not satisfy the standard value, a new case is set and if the standard value is satisfied, it is included in the existing case. By executing these loop statements, log data separated by individual tasks can be obtained. Data processed through the case separation function accurately reflects actual work performance, enabling process mining analysis. Since tools for process mining analysis can use tools of known technologies, only the following simple basic concepts are introduced and detailed descriptions will be omitted.

<Process Mining>

A number of commercial tools have been developed to support process mining. Process mining provides a set of comprehensive tools that provide fact-based insights and support process improvement using event data recorded in such various information systems. Process mining is largely classified into three types depending on the process to be analyzed, the viewpoint of analysis, and how the event log and process model are related to each other. First, process discovery uses event data as input data to automatically discover process models. At this time, there is no need for a predefined process model. Second, conformance checking is to compare and analyze the predefined process model and the actually performed process. Third, enhancement is to extend the process model discovered through process mining or the existing process model from various perspectives.

An embodiment of the present invention collects event data for process mining analysis, and the event data is composed of an identifier attribute, a class attribute, a timestamp attribute, a numeric attribute, and other attributes. Identifier properties refer to specific entities such as customer name, purchase order number, and machine number. Class attributes correspond to behaviors such as activity names and status codes. The timestamp attribute represents the time the class was executed, and the numeric attribute represents a kind of measurement value. For example, when a purchase order number is issued, an attribute called purchase quantity may correspond to a numerical attribute. Additional properties are data not included in the above categories and can have a structure such as a list or a table. In this way, the process mining event data, which is the final product, can support work process visualization, which is the biggest difficulty in the analysis and design stages of the RPA robot development project, by using process mining specialized tools (known technologies). In addition, since it is data that records all events of the person in charge, it can be used for RPA development by discovering various hidden work patterns. In addition, event logs of RPA robots can be collected and used to discover insights for evaluation and advancement.

Hereinafter, an operation process according to the configuration of the above-described monitoring service providing server of FIG. 2 will be described in detail with reference to FIGS. 3 and 4 as examples. However, it will be apparent that the embodiment is only any one of various embodiments of the present invention, and is not limited thereto.

The applicant's company (DNS Solutions) is a company that specializes in the field of RPA. It takes several hours to organize finance, accounting, purchasing, and customer management once, and there is a limit to reducing work hours even if the proficiency level increases. RPA automates these tasks. The applicant also demonstrated RPA technology directly, without touching the keyboard, but accessing the National Tax Service Hometax, downloading sales and purchase slips and making them into excel files, viewing them and refining them into offset slips according to the form, and inputting them into a specific accounting program The job is done in less than 5 minutes. Unlike macro programs based on coordinates, work can be performed regardless of changes in the external environment, and it can be applied anywhere other than standard office work. These RPAs can collect manufacturing environment data as shown in FIG. 3a, input queries to artificial intelligence algorithms, and automatically collect, input, output, and visualize data necessary for each step, and as shown in FIG. 3b, RPAs do You can define roles (red line). As shown in FIG. 3c, in an environment where there is no company developing RPA in Korea, domestic and foreign companies encroach on Korean patents and most of the foreign companies use Ui-Path, the applicant of the present invention implements most of the functions of Ui-Path, while maintaining 80% of companies using RPA leave within 5 years and only 20% maintain RPA because of the difficulty, so even ordinary people who do not know the C language can perform maintenance by using the activity according to an embodiment of the present invention. implemented to make it work.

The execution screen of the solution (NeoRPA) of the present invention provides a basic menu (development-new, open, save, run, debug, stop, record) as shown in ① as shown in FIG. 3D, and provides an activity menu as shown in ②. , A collection of GUI-styled program codes, more than 300 of which have been developed, and since inserting these activities into ③ is the end, no-coding programming is possible, and then modify, transform, change, and edit as desired by the user. And it can be deleted or created, so the maintenance rate and utilization rate can be very high. In the main screen for running the AI model as shown in FIG. 3e, RPA automatically collects each vibration data, vision data is automatically collected, and the execution code is automatically completed, so RPA proceeds all the way to visualization. The result may be output as shown in FIG. 3G.

<RPA Execution Screen>

4a to 4g are diagrams capturing screen recordings of the process of automatically collecting, editing, and inputting data by RPA using BANDICAM. At this time, since each process could not be captured, it is clear that the steps of RPA do not stay only at the captured steps. At this time, referring to FIG. 4A, each step of the artificial intelligence algorithm exists. In order to execute one step of FIG. 4B, it is necessary to check whether a folder in the user terminal 100 is designated and whether a file exists in the folder. At this time, the RPA informs that the operation as shown in FIG. 4c is necessary, and the RPA, not the user, automatically accesses the web page, finds a menu for creating folders and files in the web page, and designates a filter or identifier in the menu to After searching, when the RPA creates and downloads a file in a folder in the user terminal 100 as shown in FIG. 4d, checks and corrects whether all necessary information in the file is included and whether the format is properly specified as shown in FIG. 4e and download it. Now, returning to the screen of FIG. 4A, two files are sequentially loaded from a pre-designated folder (a folder automatically designated and created by RPA) as shown in FIGS. 4F and 4G so that necessary data for each step is input. These data collection, storage, and loading processes are simple and repetitive tasks that increase work fatigue when performed by humans, but do not improve efficiency. Even if people do not care about this process, RPA automatically loads, designates and uploads files, so it is possible to distribute tasks that simple repetitive tasks are done by RPA and other important tasks are done by people.

<RPA & AI activity execution screen>

4h to 4g are processes in which RPA creates and sets monitoring algorithms for FRB lines. It is set in advance that vibration and vision are required to monitor the FRB line, so a step of [vibration data-vision data-output] is preset and output as shown in FIG. 4H. At this time, as shown in FIG. 4i, the RPA checks the vibration vision monitoring (output step), connects to the database to obtain vibration data as shown in FIG. Loading. Then, as shown in FIGS. 4L and 4M, it is checked whether vibration data and vision data are displayed in the [Output] step, and debugging is performed by pressing the [Execute] menu at the top. Then, by accessing the page shown in FIG. 4n and outputting the screen shown in FIG. 4o on the dashboard, monitoring can be performed in the user terminal 100 .

Matters not described in the RPA-based manufacturing environment monitoring service provision method of FIGS. 2 to 4 are the same as those described in the RPA-based manufacturing environment monitoring service provision method through FIG. Since it can be inferred, the following description will be omitted.

5 is a diagram illustrating a process of transmitting and receiving data between components included in the RPA-based manufacturing environment monitoring service providing system of FIG. 1 according to an embodiment of the present invention. Hereinafter, an example of a process of transmitting and receiving data between each component will be described through FIG. 5, but the present application is not limited to such an embodiment, and according to various embodiments described above, It is obvious to those skilled in the art that a process of transmitting and receiving data may be changed.

Referring to FIG. 5, the monitoring service providing server collects at least one manufacturing environment data collected from facilities and processes in the manufacturing environment from at least one monitoring terminal using Robotic Process Automation (RPA) (S5100). .

In addition, the monitoring service providing server causes the RPA to analyze at least one manufacturing environment data and informs the user terminal whether it is in a normal state or an abnormal state (S5200), and visualizes and outputs the normal state or abnormal state in the user terminal Do (S5300).

The order between the above-described steps (S5100 to S5300) is only an example, and is not limited thereto. That is, the order of the above-described steps (S5100 to S5300) may be mutually changed, and some of the steps may be simultaneously executed or deleted.

Matters not described in the RPA-based manufacturing environment monitoring service provision method of FIG. 5 are the same as those described for the RPA-based manufacturing environment monitoring service provision method through FIGS. Since it can be inferred, the following description will be omitted.

The RPA-based manufacturing environment monitoring service providing method according to an embodiment described with reference to FIG. 5 may be implemented in the form of a recording medium including instructions executable by a computer such as an application or program module executed by a computer. there is. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer readable media may include all computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The above-described RPA-based manufacturing environment monitoring service providing method according to an embodiment of the present invention is executed by an application basically installed in the terminal (which may include a program included in a platform or operating system, etc. It may be executed by an application (ie, a program) directly installed in the master terminal by a user through an application providing server such as an application store server, a web server related to an application or a corresponding service. In this sense, the RPA-based manufacturing environment monitoring service providing method according to an embodiment of the present invention described above is implemented as an application (i.e., a program) that is basically installed in a terminal or directly installed by a user, and is readable by a computer such as a terminal. can be recorded on a recordable medium.

The above description of the present invention is for illustrative purposes, and those skilled in the art can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts should be construed as being included in the scope of the present invention. do.

Claims (10)

at least one monitoring terminal for collecting at least one manufacturing environment data from facilities and processes in the manufacturing environment;
a user terminal outputting a normal state or an abnormal state of the facility or process as a result of collecting and analyzing the at least one manufacturing environment data; and
A collection unit that collects at least one manufacturing environment data collected from facilities and processes in the manufacturing environment from the at least one monitoring terminal using Robotic Process Automation (RPA), and the RPA analyzes the at least one manufacturing environment data At least one step (Step At least one activity corresponding to at least one execution code is provided to create, modify, and delete a flow including a ), and the at least one activity is a drag and drop method A monitoring service providing server including a no-coding unit for driving the RPA according to the changed flow when inserted into a step in the flow as;
RPA-based manufacturing environment monitoring service providing system that includes.
According to claim 1,
The monitoring service providing server,
Inputting the manufacturing environment data collected by the RPA into at least one artificial intelligence algorithm as a query, and outputting an abnormal symptom detection or defect prediction of the equipment or process in the manufacturing environment data as a result and transmitting it to the user terminal AI predictive maintenance department;
RPA-based manufacturing environment monitoring service providing system further comprising a.
According to claim 1,
The monitoring service providing server,
a re-learning unit for automatically re-learning the at least one artificial intelligence model at a predetermined cycle by generating a learning dataset based on the collected manufacturing environment data and the response of the user terminal;
RPA-based manufacturing environment monitoring service providing system further comprising a.
delete According to claim 1,
The no-coding unit,
When a cell corresponding to the at least one step is selected and then dropped between different steps so that the order of the at least one step is changed, the order of the execution code so that the RPA operates according to the changed order RPA-based manufacturing environment monitoring service providing system, characterized in that for changing.
According to claim 2,
The monitoring service providing server,
An AI correction unit for changing, adding, and deleting the driving order of the artificial intelligence algorithm when the user terminal uses the RPA to change, add, or delete the order of the artificial intelligence algorithm;
RPA-based manufacturing environment monitoring service providing system further comprising a.
According to claim 1,
The monitoring service providing server,
When the user terminal arranges the activities corresponding to the at least one execution code in order using the RPA, at least one artificial intelligence algorithm corresponding to the generated flow is driven according to the activity Artificial intelligence that builds manufacturing environment data into a dataset, performs modeling through learning and verification on the built dataset, selects the most accurate artificial intelligence algorithm, and performs the flow created by the user Automatic AI generation unit that adopts and sets algorithms;
RPA-based manufacturing environment monitoring service providing system further comprising a.
According to claim 1,
The manufacturing environment data,
Facility data including vibration, current, vision, temperature and noise collected from the facility;
Process data including production conditions, product information, equipment status and control conditions collected from the process;
RPA-based manufacturing environment monitoring service providing system comprising a.
According to claim 1,
The monitoring service providing server,
a classification unit for storing text data among the manufacturing environment data in a database and storing image data corresponding to an image or video among the manufacturing environment data in a preset folder;
RPA-based manufacturing environment monitoring service providing system comprising a.
According to claim 9,
The monitoring service providing server,
a hybrid unit configured to establish hybrid artificial intelligence by setting at least one artificial intelligence algorithm for processing the text data and at least one artificial intelligence algorithm for processing the image data, respectively;
RPA-based manufacturing environment monitoring service providing system comprising a.

Images