KR20210063610A - Microservice-based artificial inteligence device for providing artificial inteligence service and method thereof - Google Patents

Abstract

An AI device of the present invention comprises: an input processing module that handles pre-processing of input data; an AI model calculation module that processes an AI model calculation process for the input data pre-processed by the input processing module; and an output processing module that processes post-processing process of the result data calculated by the AI model calculation module, wherein the pre-processing process, the AI model calculation process, and the post-processing process are characterized in that they are processes defined according to microservices. Therefore, the present invention is capable of configuring an existing application logic to respond to various service environments.

Description

MICROSERVICE-BASED ARTIFICIAL INTELIGENCE DEVICE FOR PROVIDING ARTIFICIAL INTELIGENCE SERVICE AND METHOD THEREOF

The present invention relates to a technology for providing an artificial intelligence service, and more particularly, to a technology for dividing an application into components of a microservice unit.

- Current machine learning and neural network technologies are competing to increase accuracy, and for this purpose, rapid and accurate analysis algorithms and frameworks are being developed through technologies such as cloud or GPU.

- However, in order to use the developed models in real life, the appropriate distribution and update of algorithms (models) that provide accurate reasoning with each framework must be able to proceed.

- Current AI applications provide input data reception, input data processing, model operation, and data post-processing in a single application, rather than a service type configuration that provides the service to the outside with a single application configuration.

Such a configuration is a factor that greatly degrades the reusability of services and application logic because other applications using the same model require modification of existing applications.

An object of the present invention is to provide an apparatus and method for providing 1) a microservice-based service structure and 2) input/output control technology based on the service structure for updating software reusability and management performance in existing artificial intelligence software. have.

The above and other objects, advantages and features of the present invention, and a method for achieving them will become clear when referring to the embodiments described below in detail in conjunction with the accompanying drawings.

An AI device according to an aspect of the present invention includes an input processing module for processing a pre-processing of input data; an AI model calculation module for processing an AI model calculation process with respect to the input data preprocessed by the input processing module; and an output processing module for processing a post-processing process of the result data calculated by the AI model computation module, wherein the pre-processing process, the AI model computation process, and the post-processing process are processes defined according to microservices. .

A method of distributing a containerized application according to another aspect of the present invention is a method of distributing a containerized application in a microservice platform device, the method comprising: selecting a target application from among existing applications; analyzing the selected target application and extracting an input pre-processing function defined as a microservice, an AI model calculation function, and a post-processing function; configuring the extracted input pre-processing function, AI model calculation function, and post-processing function into containerized modules, respectively; setting an operation policy for each containerized module; and distributing each containerized module.

A microservices platform device according to another aspect of the present invention includes: an application selection unit for selecting a target application having an input pre-processing function, an AI model calculation function, and a post-processing function; a container configuration unit that configures the input pre-processing function, the AI model calculation function, and the post-processing function of the selected target application into containerized modules, respectively; and an operation policy setting unit for setting an operation policy for each containerized module.

AI device according to another aspect of the present invention, an input processing module for processing a preprocessing process of changing the image file or video file loaded from a specific url to the resizing (resize) and array (array) form; an AI model calculation module for processing an AI model calculation process for the data generated according to the preprocessing process; and an output processing module for processing a post-processing process of the result data calculated by the AI model computation module, wherein the pre-position process, the AI model computation process, and the post-processing process are defined according to microservices to be.

According to the present invention, the existing application performs pre-processing of input data and post-processing of output in a single application in the form of a single application, making it difficult to service and reducing the reusability of the developed code, whereas microservice-based AI service By proposing the architecture and providing input/output management technology based on it, it is possible to configure the existing application logic to respond to various service environments.

In addition, in order to improve the performance of the analysis application, it is possible to enhance the code reusability for service configuration by replacing only the input/output models with the corresponding microservice, rather than updating the entire application, and also for managing the input/output model. By proposing a systematic plan, it is possible to reduce the difficulty of developing AI-based services and promote service.

1 is an overall configuration diagram of a microservice-based AI device according to an embodiment of the present invention.
2 is an overall configuration diagram of a microservice platform device for distributing a microservice-based application according to an embodiment of the present invention.

In the present invention, various modifications may be made and various embodiments may be provided, and specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to a specific embodiment, it should be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present invention.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present application, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof does not preclude in advance.

Unless otherwise defined, all terms including technical or scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms as defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in the present application. Does not.

[Definition of Terms]

microservices ( microservice )

Microservices refers to a software development technique that is a type of service-oriented architecture style that structures an application as a collection of loosely coupled services. Microservices can be used as a term to define the smallest unit of a process. Some more terms related to these microservices are as follows. Services in a microservice architecture (MSA) are often processes that communicate over a network to achieve a goal using technology agnostic protocols such as HTTP. However, services may use other kinds of interprocess communication mechanisms, such as shared memory. Services can run in the same process, for example in OSGI bundles. Services in a microservices architecture can be independently deployed, and service replacement is easy. Microservices are categorized by function. Services can be implemented using different programming languages, databases, hardware, and software environments according to optimal conditions. Microservices are small, message-passable, context-bound, autonomously developed, independently deployable, distributed, and released as automated processes.

container

Container is used as a term to describe a standardized way of packaging an application's code, configuration, and dependencies into a single object. In other words, containers package code along with configuration files and dependencies needed to run consistently in any environment. Containers provide process isolation that allows you to set CPU and memory usage for optimized use of computing resources. In other words, containers run as separate processes that share the resources of the operating system. This enables rapid start and stop of containers. Containers share the operating system installed on the server and run as resource-isolated processes, ensuring fast, reliable and consistent deployment regardless of environment. In the context of microservices, containers are used as a term to provide process isolation that allows applications to be easily separated and run as independent components called microservices. Containers standardize how code is deployed, making it easy to build workflows for applications running across cloud environments. Using containers, you can quickly scale machine learning or AI models to perform training and inference.

Recently, analysis technologies such as neural networks and machine learning technologies are rapidly evolving due to high-performance computing environments such as GPU and cloud. However, these neural networks and machine learning technologies are focused on improving accuracy, so they generally require very high-spec resources.

These applications are being developed through a platform (eg, Tensorflow, etc.) to support the neural network operation on hardware that can support the neural network operation. In this case, the application is configured as a single application type that supports all of the pre-processing of received or collected input data, the model computation process for the pre-processed input data, and the post-processing of the result data generated according to the model computation process.

In the configuration of such a single application type, there are 1) difficulties in converting the application into a service, 2) the need to manage the entire code when improving application performance, and 3) difficulties in configuring and managing web-based services.

In order to improve this, the present invention provides a microservice-based AI service method.

Before describing the microservice-based AI service method according to the present invention, first, an existing AI or artificial intelligence application performs the following basic processes.

1) Load the source data to be analyzed.

2) Preprocessing is performed to provide the source data to the input of the neural network AI model.

3) Apply the preprocessed source data to the model and proceed with the analysis. At this time, the analyzed result is generated at the end (output end) of the neural network model.

4) Post-process the analysis result generated in the arrangement of the neural network model.

5) Post-processed results are presented to the user.

The above process corresponds to an actual image processing-based AI model, for example, an image classification application, as follows.

1) Load an image file from a specific url.

2) In order to provide the image file to the input terminal of the neural network model (or AI model), it is resized to a specific size and/or the image is changed to an array form.

3) The image of the array obtained in 2) is provided to the input of the neural network model.

4) Sort the data generated at the output of the neural network model (AI model). For example, one or five indexes with the highest value are extracted, and the values of the model parameter file are compared again to derive top-1 and top-5 values.

5) The text generated as a result of 4) is displayed to the user.

Another example is below. Here, an example of receiving a video stream, extracting an object of the corresponding video stream, and providing the result as a video is presented.

1) Load a video file from a specific URL.

2) Extract one frame from the input video. The extracted frame is resized as necessary or converted into an array form.

3) The data generated as a result of 2) is provided to the input of the neural network model (or AI model).

4) Using the data generated at the output of the neural network model, the object is displayed on the image collected in 2). For example, a bounding box and an object name are additionally displayed.

5) The image created as a result of 4) is provided through HTTP streaming (MJPEG, etc.).

As such, the above processes performed in the existing AI application are currently developed and provided in the form of a single application.

However, calculations related to the actual neural network (AI model) are provided only in step 3), steps 1) and 2) are preprocessed, and steps 4) and 5) can be separated (classified) into post-processing. . Here, if each process can be provided independently in the form of microservices, it will be possible to configure the expansion of various services for existing AI applications.

The microservice-based AI service method according to the present invention is characterized in that the pre-processing process performed in the existing AI application, the computation process of the AI model, and the post-processing process are independently operated in units of microservices.

1 is an overall configuration diagram of an AI device based on microservices according to an embodiment of the present invention.

A microservice-based artificial intelligence device (hereinafter, referred to as an 'AI device') 100 according to an embodiment of the present invention executes an application developed (programmed or coded) based on a microservice architecture. It may be a device.

Such an AI device is a computing device including at least one processor, a memory, a communication module, and a system bus connecting them to execute an application developed based on the microservice architecture, or a user terminal including such a computing device; It may be embodied as a desktop or server.

The application developed based on the microservice architecture executed by the AI device or at least one processor included in the AI device includes a pre-processing of input data performed by an existing application, and an AI model operation on the pre-processed input data. It is implemented as a plurality of containerized software modules to independently change the post-processing process of the output data generated according to the process and the computation process of the AI model, and to combine these processes.

In an embodiment of the present invention, as shown in FIG. 1, containerized software modules are input processing module 110, AI model operation module 120, output processing module 130 and service control module ( 140) is referred to.

input processing module 110

The input processing module 110 is a software module that processes the pre-processing of input data in units of microservices, and may be a containerized module based on a container technology such as Docker.

The containerized input processing module 110 receives input data, and performs a process of pre-processing the received input data according to the characteristics of the input terminal of the AI model included in the AI model operation module 120 .

The preprocessing process performed in units of microservices may be, for example, resizing and changing an image file or a video file loaded from a specific url into an array form.

AI Model Calculation Module (120)

The AI model calculation module 120 is a module containerized to include the AI model, and processes the process of applying the preprocessed input data input from the input processing module 110 to the AI model in units of microservices. Here, the result data output from the AI model calculation module 120 is data generated by the output terminal of the AI model.

output processing module 130

The output processing module 130 is a software module that post-processes the result value input from the above-described AI model calculation module 120 and processes the process of providing the post-processed result data to the user in units of microservices. Like the components 110 and 120 , it may be a containerized module based on a container technology such as Docker.

The post-processing process performed in units of microservices may, for example, perform a process of aligning result data generated at the output end of the AI model. For example, it may be a process of providing the highest value among the sorted data or a plurality of data sorted in descending order based on the highest value to the user.

As another example of a post-processing process performed in units of microservices, an object is displayed in an image collected using the result data generated at the output end of the AI model, and the image in which the object is displayed is provided to the user through HTTP streaming (MJPEG, etc.) It may be a process to provide. Here, the process of displaying the object on the collected image may be, for example, displaying a bounding box and/or an object name on the collected image.

service control module 140

The service control module 140 controls the connection and operation between the modules 110 , 120 and 130 divided into microservice units, and controls the input and output of the modules 110 , 120 and 130 to each module. It may be a containerized module that handles the process of setting up the operation of the system in units of microservices.

In order to set the operation for each module, the service control module 140 designates a url and a preprocessing format (preprocessing policy data defining a preprocessing format), and transmits it to the input processing module 110 .

In addition, the service control module 140 transmits a delivery method (or delivery policy data defining a delivery method) for delivering the pre-processed data to the AI model operation module 120 to the input processing module 110 .

In addition, the service control module 140 transmits the reception method (or reception policy data defining the reception method) of the preprocessed data to the AI model operation module 120 .

In addition, the service control module 140 requests the AI model calculation module 120 to perform the calculation of the AI model.

In addition, the service control module 140 transmits a method (or delivery policy data defining a delivery method) of delivering the operation result (output) of the AI model to the AI model operation module 120 .

Also, the service control module 140 transmits a reception method (reception policy data defining a reception method) for receiving result data indicating the operation result of the AI model to the output processing module 130 .

In addition, the service control module 140 specifies a post-processing format (post-processing policy data defining a post-processing format) for the result data representing the computation result of the AI model computation module 120 and sends it to the output processing module 130 . transmit

In addition, the service control module 140 sets a method for providing (displays) the final analysis result obtained through the post-processing process in the output processing module 130 to the user, or sets the setting method to the output processing module 130 . forward to Here, the setting method may include, for example, designation of a URL of the user terminal.

The service control module 140 may also be a containerized module implemented to perform an independent process in units of microservices, similarly to the above-described components 110 , 120 , and 130 .

In the case of the above configuration, the processes processed in the above two examples may be changed to the following independent processes in the microservice unit.

1) (Service Control Module --> Input Processing Module) The input URL is transmitted to the input processing module 110, and input terminal information of the AI model operation module is transmitted.

2) (Input processing module --> AI model calculation module) Pre-process the input data and transfer the pre-processed input data to the AI model calculation module.

3) (Service control module --> AI model operation module) Instructs to deliver the result of operation to the output processing module.

4) (AI model calculation module --> output processing module) The result of the operation is delivered to the output processing module.

5) (Service control module --> Output processing module) Set the post-processing method through the operation result.

6) (Output processing module) After post-processing, the data is delivered to the designated URL.

Through the above configuration, it is possible to interlock various inputs and outputs for one type of AI model calculation module, and through this, higher code reusability and setting method for AI service configuration can be suggested, and AI service-based activation is possible. can be presented.

Meanwhile, the application including the containerized modules 110 , 120 , 130 and 140 executed in the AI device 100 may be distributed from an external system. Here, the external system may be referred to as a microservice platform device (or a cloud platform device).

That is, the microservice platform device selects an existing application that can be divided into three specialized functions, such as a pre-processing function, an AI model calculation function, and a post-processing function, among existing applications, and converts the selected existing application into a containerized application divided by each function. It can be developed as a module and distributed to the AI device 100 . In this case, the AI device 100 may receive (download) all containerized modules from the microservice platform device, or receive some containerized modules as other microservice platform devices.

2 is an overall configuration diagram of a microservice platform device for distributing a microservice-based application according to an embodiment of the present invention.

Referring to FIG. 2 , the microservice platform system 200 according to an embodiment of the present invention includes an application selection unit 210 , an application analysis unit 220 , a container configuration unit 230 , and an operation setting unit 240 . do.

The application selector 210 responds to a request message for requesting the provision of a microservice-based application from the AI device 100 or a device including the same (eg, a user terminal, a desktop, or a server), among existing applications. Select the target application.

The target application includes specialized functions desired by the user of the AI device 100 or a device including the same (eg, a pre-processing function of input data, an AI model arithmetic processing function for pre-processed input data, and an arithmetic processing result of the AI model It is selected based on whether it has post-processing function for data).

In order to select such a target application, the application selector 210 requests application information desired by the user. Here, the application information requests information about an application that the user has previously used (name information or version information of an existing application) or information about an existing application that has a similar function but has no experience in using it.

When receiving application information from the AI device 100 or a device including the same, the application selecting unit 210 selects a target application based on the received application information.

The application analysis unit 220 analyzes the application selected by the application selection unit 210, extracts a minimum functional element classified as a microservice from the selected application according to the analysis result, and stores the extracted minimum functional element as a container Decide on the target of conversion. Here, the minimum functional element of the microservice unit includes, for example, an input pre-processing step (input pre-processing function), an AI model calculation step (AI model calculation function), and a post-processing step (post-processing function).

The container configuration unit 230 sets the container conversion target determined by the application analysis unit 220, for example, the input pre-processing step, the AI model calculation step, and the post-processing step to a module separation policy, module integration policy, module linkage policy, individual Each of the containerized modules is configured in consideration of the container configuration policy including the module's availability policy, the individual module's extensibility policy, and the security policy. Here, a method of configuring each step as a containerized module may be an open source change of an existing application, a code change, etc., and a description thereof is replaced with a known description of a container technology such as Docker. Meanwhile, although FIG. 2 illustrates a case in which the application analysis unit 220 and the container configuration unit 230 are separated, they may be integrated into one configuration. For example, the application analysis unit 220 may be integrated into the container configuration unit 230 .

The operation policy setting unit 240 sets the operation policy of the containerized modules. The operation policy may be, for example, a policy related to input/output control of each containerized module. For example, a delivery policy for the input processing module 110 that defines a method of transferring the preprocessed data to the AI model operation module 120, and the AI model operation module 120 that defines a method of receiving the preprocessed data A delivery policy for the AI model calculation module 120 that defines a reception policy for, a policy related to requesting the AI model calculation module 120 to perform calculation of the AI model, and a method of delivering the calculation result (output) of the AI model , a reception policy for the output processing module 130 that stipulates a method of receiving the result data indicating the operation result of the AI model, and the post-processing format (post-processing) for the result data indicating the operation result of the AI model operation module 120 Post-processing policy data defining the format) for the output processing module 130 that defines the post-processing policy for the output processing module 130 and the method of providing the final analysis result obtained through the post-processing process to the user provision policies, and the like.

In addition, the operation setting unit 240 reflects the overall monitoring view of the containerized microservice application, infrastructure operation monitoring, performance test, and performance test results to the containerized microservice application, error reflection, operating system transfer and application, etc. Set up operation.

As described above, in the present invention, the existing application performs pre-processing of input data and post-processing of output in a single application in the form of a single application, making it difficult to service and reducing the reusability of the developed code, while By proposing a service-based AI service architecture and providing input/output management technology based on it, it is possible to configure the existing application logic to respond to various service environments.

In addition, in the present invention, in order to improve the performance of the analysis application in some cases, instead of updating the entire existing application, only the models related to input/output control are replaced with the corresponding microservice, thereby enhancing code reusability for service configuration. By suggesting a systematic way to manage input/output models, it is possible to reduce the difficulty of developing AI-based services and promote service activation.

The present invention may be implemented in hardware components, software components, and/or combinations of hardware components and software components. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA). , a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers.

The software may include a computer program, code, instructions, or a combination of one or more of these, configuring the processing unit to operate as desired or processed independently or collectively. You can command the device. Software and/or data may be interpreted by a processing device or, to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. , Or may be permanently or temporarily embodyed in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

Although the embodiments have been described by the limited embodiments and drawings as described above, various modifications and variations can be made from the above description to those of ordinary skill in the art. For example, the described techniques are performed in a different order from the described method, and/or components such as systems, structures, devices, circuits, etc. described are combined or combined in a form different from the described method, or other components Alternatively, even if substituted or substituted by an equivalent, an appropriate result can be achieved. Therefore, other implementations, other embodiments, and those equivalent to the claims also fall within the scope of the claims to be described later.

Claims (12)

an input processing module for processing the pre-processing of input data;
an AI model calculation module for processing an AI model calculation process with respect to the input data preprocessed by the input processing module; and
An output processing module for processing post-processing of the result data calculated by the AI model calculation module;
The AI device wherein the pre-processing process, the AI model calculation process, and the post-processing process are processes defined according to microservices. In claim 1,
The input processing module, the AI model calculation module, and the output processing module are each a containerized module. In claim 1,
The AI device further comprising a service control module for processing input/output control between the input processing module, the AI model operation module, and the output processing module according to a predefined operation policy. In claim 3,
wherein the service control module is a containerized module. A method of deploying a containerized application on a microservices platform device, comprising:
selecting a target application from among existing applications;
analyzing the selected target application and extracting an input pre-processing function defined as a microservice, an AI model calculation function, and a post-processing function;
configuring the extracted input pre-processing function, AI model calculation function, and post-processing function into containerized modules, respectively;
setting an operation policy for each containerized module; and
deploying each containerized module;
How to deploy a containerized application containing In clause 5,
The configuring step is
Considering the container configuration policy including the module separation policy, module integration policy, module linkage policy, individual module availability policy, individual module scalability policy and security policy, the input pre-processing function, AI model calculation function, and post-processing function A method of deploying a containerized application, the step of configuring each into containerized modules. In clause 5,
The step of setting the operation policy is,
A method of deploying a containerized application, which is a step of setting policies related to input/output control of each containerized module. In clause 7,
The policies related to I/O control of each containerized module are:
A preprocessing policy that defines the preprocessing format of input data, a forwarding policy that specifies a method for transmitting preprocessed data, a reception policy that defines a method of receiving preprocessed data, a forwarding policy that defines the method of delivering the calculation result of the AI model, AI Reception policy that stipulates how to receive the calculation result of the model, the post-processing policy that stipulates the post-processing format of the AI model calculation result, and the provision policy that stipulates how to provide users with the final analysis result obtained through the post-processing A method of deploying a containerized application comprising: an application selection unit for selecting a target application having an input pre-processing function, an AI model calculation function, and a post-processing function;
a container configuration unit that configures the input pre-processing function, the AI model calculation function, and the post-processing function of the selected target application into containerized modules, respectively; and
Operation policy setting unit to set operation policy for each containerized module
A microservices platform device comprising a. In claim 9,
The operation policy setting unit,
A microservices platform device that sets policies related to I/O control of each containerized module. an input processing module for processing a preprocessing process of resizing and changing an image file or a video file loaded from a specific url into an array form;
an AI model calculation module for processing an AI model calculation process for the data generated according to the preprocessing process; and
an output processing module for processing post-processing of the result data calculated by the AI model calculation module;
The AI device wherein the pre-processing process, the AI model calculation process, and the post-processing process are processes defined according to microservices. In claim 11,
The post-processing process processed by the output processing module is,
The process of adding a bounding box and object name to an image file or video file using the calculated result data, and an image file or video file to which the bounding box and object name are added are provided to the user terminal in an HTTP streaming method AI device comprising a process for doing.

Images