KR20220070868A - Electronic device control method, device and program - Google Patents

Abstract

A method for controlling an electronic device includes: performing a service for transmitting requested video data to a user equipment, in response to a request by the user equipment, generating a generative neural network film required for an artificial neural network algorithm operation to improve a resolution of image information, based on retained video data, and transmitting, to the user equipment, lower-quality video data obtained by changing the generative neural network file and the resolution to be in a level lower than a preset level; receiving the lower-quality video data, performing a computation operation based on the artificial neural network obtained by applying the received generative neural network file to the received lower-quality data, improving a resolution of the lower-quality video data depending on the computation operation, and regenerating video data having the improved resolution; storing a testing image to be improved in image quality, and providing the testing image, which is to be improved in image quality, to a server to improve the image quality; training an AI image processing model to be used in improving image quality of an image, improving the image quality by introducing a specific testing image, which is received from a testing image managing server, into the AI image processing model, wherein the AI image processing model is formed by combining at least two AI models, and improving an image quality of a target image in a manner of finding an average of result data obtained, as the target image is introduced into each of a plurality of AI models combined.

Description

Electronic device control method, device and program {Electronic device control method, device and program}

The present invention relates to a method, apparatus and program for controlling an electronic device.

In order to improve the resolution of image data, various approaches such as interpolation techniques and machine learning techniques have been used. Recently, research using deep learning techniques has been actively conducted in the field of pattern recognition such as image recognition/analysis. In addition, it was proved that the technology using deep learning showed superior performance than the learning-based machine learning method in the upscaling method, which is one of the image quality improvement fields.

An object of the present invention is to provide a method, apparatus, and program for controlling an electronic device.

In order to achieve the above object, in an embodiment of the present invention, a method for controlling an electronic device performs a service of transmitting requested video data to a user device in response to a request from a user device, but based on the retained video data generating a general-purpose neural network file required for an artificial neural network algorithm operation for resolution improvement of image information, and transmitting the created general-purpose neural network file and low-quality video data obtained by changing the resolution to a preset level or less to the user device; Receive low-quality video data, perform a calculation operation based on an artificial neural network algorithm that applies the received general-purpose neural network file to the received low-quality data, and improve the resolution of the low-quality video data according to the calculation operation, and the resolution is reproducing the improved video data; Storing an image for inspection that is a target for image quality improvement, and providing an image for inspection that is a target for image quality improvement to the image quality improvement server; learning an AI image processing model to be used to improve image quality, and from the image management server for inspection The received specific inspection image is input to the AI image processing model to improve image quality, wherein the AI image processing model is configured by combining at least two or more AI models, and a target image is applied to each of a plurality of combined AI models. improving the target image quality by averaging the result data obtained by inputting; includes

According to an embodiment of the present invention, even when the image data provider transmits low-quality data, when the image is reproduced, the resolution can be improved based on the neural network file, so data transmission and reception is performed with low-quality data instead of high-quality data. It can provide an environmental foundation. Accordingly, the present invention is effective in solving the problem of a reduction in transmission speed that occurs when high-capacity high-definition image data is transmitted.

In addition, the present invention has the effect of occupying a relatively small amount of memory because even if only a low-quality image can be converted to a high-quality image through a neural network file at the time of running the video.

In addition, the present invention can solve the difference in illuminance for each image frame that occurs when the same subject is photographed by rotating 360 degrees through the AI image processing process.

1 is a diagram showing the configuration of an image quality improvement system according to an embodiment of the present invention.
2 is a diagram illustrating a configuration of a learning unit according to an embodiment of the present invention.
3 is a block diagram of an apparatus according to an embodiment of the present invention.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail.

However, this is not intended to limit the present invention to specific embodiments, and it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention. In describing each figure, like reference numerals have been used for like elements.

When a component is referred to as being 'connected' or 'connected' to another component, it may be directly connected or connected to the other component, but it is understood that other components may exist in between. it should be On the other hand, when it is mentioned that a certain element is 'directly connected' or 'directly connected' to another element, it should be understood that the other element does not exist in the middle.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as 'comprise' or 'have' are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, and one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

1 is a diagram showing the configuration of an image quality improvement system according to an embodiment of the present invention.

As shown in FIG. 1 , the resolution improvement system according to an embodiment of the present invention may be configured to include a server 100 and a user device 200 .

According to an embodiment, the server 100 may include a server that provides a VOD service to the user device 200 .

The server 100 may transmit video data to provide a VOD service to the user device 200 . And according to an embodiment of the present invention, the server 100 may calculate a neural network file required to improve the quality of video data and transmit it to the user device 200 together. Accordingly, the user device 200 can improve the image quality based on the neural network file that receives the video data provided from the server 100 .

In addition, the user device 200 may select video data to be transmitted to the server 200 and request transmission. In addition, the user device 200 may calculate user viewing pattern information calculated based on selection information and reproduction information of its own video data, and transmit it to the server 200 .

The user device 200 may generate a video file with an improved resolution through a neural network file for the video file. In this case, according to an embodiment of the present invention, the neural network file is a general-purpose file that can be used regardless of the type of video file, and accordingly, the neural network file can be combined with any video file transmitted to the user device 200 to improve resolution.

In addition, according to various embodiments, the user device 200 may load the general-purpose neural network file as embedded software and receive a video file to be improved in quality from the server 100 (eg, a video streaming server).

The server 100 according to an embodiment of the present invention may include a communication unit 110, a storage unit 120, and a control unit 130. The control unit 130 may include a data processing unit 131, a neural network learning unit 132, a result evaluation unit 133, and a usage pattern calculating unit 134.

First, the communication unit 110 may use a network for data transmission/reception between a user device and a server, and the type of the network is not particularly limited. The network may be, for example, an IP (Internet Protocol) network that provides a large-capacity data transmission/reception service through the Internet Protocol (IP) or an All IP network that integrates different IP networks. In addition, the network includes a wired network, a Wibro (Wireless Broadband) network, a mobile communication network including WCDMA, a High Speed Downlink Packet Access (HSDPA) network, and a Long Term Evolution (LTE) network including a mobile communication network, LTE advanced (LTE-A) ), a mobile communication network including 5G (Five Generation), a satellite communication network, and a Wi-Fi network, or a combination of at least one or more of them.

The communication unit 110 according to an embodiment of the present invention may perform data communication with an external web server and a plurality of user devices. For example, the communication unit 110 may receive content data (photos and videos) including images from other web servers or user devices (including administrator devices). In addition, since the server 100 includes a server that provides a VOD service, VOD content corresponding to a user request may be transmitted to the user device 200 .

According to various embodiments, the communication unit 110 may receive and transmit a VOD file for a VOD service, but is not limited thereto, and the communication unit 110 collects learning data required to generate a neural network file for resolution improvement. communication function for

The neural network file may contain information necessary to restore the resolution of image data damaged through an artificial neural network algorithm to be similar to the original data, and may include information on various parameters to be selected when the artificial neural network algorithm is driven. have.

The storage unit 120 may include, for example, an internal memory or an external memory. The internal memory includes, for example, a volatile memory (eg, dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM), etc.), non-volatile memory (eg, OTPROM (one time programmable ROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (such as NAND flash or NOR flash), hard drive, or a solid state drive (SSD).

External memory is a flash drive, for example, CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), XD (extreme digital), It may further include a multi-media card (MMC) or a memory stick. The external memory may be functionally and/or physically connected to the electronic device through various interfaces.

The storage unit 120 according to an embodiment of the present invention is processed data (original image data reduced at a certain rate) obtained by processing image data (eg, photo and video data) received from a user device (administrator device) or another web server. data, or data that has been reduced and then enlarged to the original data size) and the corresponding original data may be matched and stored. The original data and the processed data may be used to extract information about a lattice phenomenon that occurs when the resolution is reduced, respectively.

In addition, the storage unit 120 may store a neural network file, which is data for restoring the resolution to the original image level by removing the grid existing in the processed data through an artificial intelligence algorithm (eg, SRCNN) after extracting information on the grid phenomenon.

The controller 130 may also be referred to as a processor, a controller, a microcontroller, a microprocessor, a microcomputer, or the like. Meanwhile, the control unit may be implemented by hardware, firmware, software, or a combination thereof.

In the case of implementation by firmware or software, an embodiment of the present invention may be implemented in the form of a module, procedure, function, etc. that perform the functions or operations described above. The software code may be stored in the memory and driven by the control unit. The memory may be located inside or outside the user terminal and the server, and data may be exchanged with the controller by various known means.

The controller 130 according to an embodiment of the present invention may generate a general-purpose neural network file, which is a file required to improve the resolution of image data through an artificial neural network-based operation.

The controller 130 may include a data processing unit 131 , a neural network learning unit 132 , a result evaluation unit 133 , and a usage pattern calculating unit 134 .

First, the data processing unit 131 may collect and process learning data required to calculate a general-purpose neural network file required for image quality improvement of video data. The data processing unit 131 may perform a primary change (reduction change) and a secondary change (enlargement of the reduced image data again) of the collected data. More specific details of the operation performed by the data processing unit 131 will be described with reference to FIG. 4 .

The neural network learning unit 132 may perform a neural network learning operation based on artificial intelligence based on the processed data calculated after the data collection and processing operations performed by the data processing unit 131 are performed. The neural network learning unit 132 may perform a parameter setting operation and a neural network file calculation operation required for a learning process. A more detailed description of the neural network learning unit 132 will be described with reference to FIG. 5 .

The result evaluation unit 133 may perform an operation of evaluating a result value obtained by applying the general-purpose neural network file calculated by the neural network learning unit 132 in the user device 200 .

Specifically, the result evaluator 133 may determine the degree of improvement in the resolution of data as a result of applying the neural network file in the user device 200 . Also, the result evaluation unit 133 may determine an error rate between the result data and the original data after applying the neural network file. In this case, the comparison unit between the result data and the original data may be each frame constituting an image or a fragment unit divided for image transmission.

Alternatively, according to various embodiments, each image data is a plurality of frame bundles based on image identity (eg, when an image is displayed over 100 frames, the 100 frames may be set as one frame bundle, and such a frame bundle In addition, it may be divided into a plurality of pieces). Accordingly, the comparison unit of the result data and the original data may be a bundle unit divided based on image identity.

Furthermore, when it is determined that the error rate between the result data and the original data is equal to or greater than a reference value, the result evaluator 133 may request correction of weight and bias values constituting the neural network file. That is, the result evaluator 133 may determine whether to modify the parameters constituting the neural network file by comparing the original data and the result data.

According to an embodiment, the result evaluator 133 may calculate the importance of each image object required to understand the image from the original data. And, the error rate of one unit (eg, one frame, one frame bundle, etc.) of the original data and the result data (data whose resolution is improved by applying a neural network file in the user device 200) is equal to or greater than a preset value, and When it is determined that an image object greater than or equal to a preset value is included, it is possible to request correction of weight and bias values constituting the neural network file.

The importance of each image object may be calculated based on a size ratio occupied by one image object in one frame, a repetition ratio of the image object, and the like.

According to various embodiments, the result evaluator 133 may calculate the importance of each image object based on content characteristics of video data according to various embodiments. The result evaluator 133 may first check the content characteristics of the video data. In this case, the content characteristics of the private video data may be calculated by the data processing unit 131 . For example, the content characteristics of the video data include information on the path where the video data is uploaded to the server 100 (eg, the folder name selected by the user or administrator when uploading the video file to the server 100), and the user or administrator It may be calculated based on the content genre, field, etc. input at the time of uploading the video file to the server 100 . In addition, the content characteristics of the calculated video data may be managed as metadata of the corresponding video data.

Accordingly, the result evaluator 133 may check the content characteristic information of each video data extracted and stored at the time of being uploaded to the server 100, and thus may calculate the importance of each image object based on the content characteristic information. For example, the result evaluator 133 may classify the image object item into a face object of a person, a text object (eg, a subtitle), an object object, etc., and each object item matching the content characteristic information (eg, in the drama content) Correspondingly, it is possible to match the facial object of the person).

The usage pattern calculating unit 134 may calculate the user's usage pattern for the VOD service based on the user's streaming request record, download record, and user information calculated and transmitted from the user device 200 .

The usage pattern calculating unit 134 may calculate items such as a user's preferred genre, preferred content characteristics, main streaming request time period, and main viewing device as a usage pattern. The usage pattern calculator 134 may recommend a screen mode suitable for the user based on the calculated user's usage pattern information. According to various embodiments, the neural network file for image quality improvement may be used for general purpose, but depending on the screen mode of the result data (eg, movie size mode, subtitle-oriented mode, specific color (RGB) enhancement mode, person-oriented resolution improvement mode, etc.) It may be produced as a separate file. That is, the user may perform resolution improvement by downloading one general-purpose neural network file suitable for his/her usage pattern among a plurality of neural network files for each screen mode provided by the server 100 to the user device 200 . To this end, the usage pattern calculating unit 134 may calculate a usage pattern for each user and separately manage information on the usage pattern. After that, when a corresponding user requests streaming, a neural network file of a mode suitable for the user's usage pattern can be created and provided.

According to various embodiments, a general-purpose neural network file suitable for a user's usage pattern may be automatically set based on the user's usage pattern information calculated by the usage pattern calculating unit 134 of the server 100, or based on the screen mode selected by the user. may be set.

Meanwhile, the image quality improvement system according to an extended embodiment of the present invention may include an inspection image management server 110 and an image processing server 120 .

Specifically, the inspection image management server 110 may store and manage the generated inspection image, and when the image resolution is to be improved, a specific image is selected from among the images stored therein to select the image. may be forwarded to the processing server 120 .

Therefore, when the image management server 110 for inspection wants to transmit an image to the image processing server 120 for resolution improvement rather than learning the AI image processing model, one sheet at a time or a reduced number of presets (eg, Only certain images in Chapter 3) can be delivered.

Also, the image management server for inspection 110 may transmit a plurality of images for inspection to be used as image data for training to the image processing server 120 .

Specifically, the image processing server 120 includes a learning unit 121 that selects a target to be used as image data for learning from among a plurality of image data obtained from the image management server 110 for inspection, and puts it into an AI image processing model to learn; It may be configured to include a processing unit 122 that improves the image by inputting the target image to the AI image processing model that has been trained.

The AI image processing model according to an embodiment of the present invention may be configured by combining a plurality of sub-AI models, for example, a preset number of super-resolution models (SR models) and a preset number of denoise models to be included as sub-models. can However, the AI image processing model is not limited only to a combination of a super-resolution model and a denoise model as described above, and a preset number (eg, 4) of a super-resolution model or a preset number of denoise models is used as a sub-model. may be of the form.

On the other hand, the AI image processing model may be configured in a form in which the sub-models of 401 to 404 are combined. In this case, all of the 401 to 404 may be a super-resolution model or may be all denoise models. Alternatively, half of them may be a super-resolution model and the rest may be a denoise model.

According to the various configurations as described above, the learning unit 121 of the image processing server 120 may learn a plurality of sub-models, respectively, and the processing unit 122 converts the image to be improved in resolution into the plurality of sub-models. The result data is calculated by input to the configured AI image processing model, but the final result data can be generated by averaging the primary result data calculated through each of a plurality of sub-models.

Also, according to various embodiments, the learning unit 121 of the image processing server 120 may perform an additional learning operation including the primary result data calculated by the processing unit 122 . For example, the learning unit 121 of the image processing server 120 includes a new image that has never passed the AI image processing model, primary result data of passing the new image through a super-resolution (SR) model once, and the new image. Learning can also be performed by re-inputting the averaging data obtained by averaging the primary result data of the SR model and the primary result data of the DN model, passing the image through the denoise (DN) model, in the learning stage. have.

And, as the AI image processing model according to an embodiment of the present invention may be configured to include a sub-model including a super-resolution model and a denoise model, the learning unit 121 also corresponds to the super-resolution model learning unit 121a and a denoise model learning unit 121b. A detailed configuration of the learning unit 121 will be described later with reference to FIG. 2 .

On the other hand, the processing unit 122 may perform a resolution improvement operation through the trained AI image processing model, and input the target image of the resolution improvement operation obtained from the inspection image management server 110 as an input value to the AI image processing model. And, it is possible to obtain the result data by performing an operation according to the previously learned algorithm. As described above, in the process of obtaining the result data, the final result data can be obtained by averaging each of the primary result data calculated by passing through a plurality of sub-AI models.

Meanwhile, the processing unit 122 of the image processing server 120 according to an embodiment of the present invention may drive an AI image processing model in which a super-resolution model and a denoise model are combined. The super-resolution model and the de-noise model may be trained in different ways, and when learning is completed, the processing unit 122 receives an image to be improved in resolution and performs an AI operation on it. In addition, the processing unit 122 calculates final result data by averaging the primary result data that has been subjected to noise removal and upscaling through the denoise model and the super-resolution model.

Furthermore, the present invention can use a difference value between a pre-processed working image and an image obtained by averaging several images as learning data for the development of an improved AI model later, and use the structure value (std) of the difference image to generate noise Learning can also be carried out by composing an image dataset according to the noise structure by grasping the structure diagram of

Hereinafter, the components of the learning unit 121 will be described with reference to FIG. 2 .

2 is a diagram illustrating a configuration of a learning unit according to an embodiment of the present invention.

As shown in FIG. 2 , the learning unit 121 may include a super-resolution model learning unit 121a and a denoise model learning unit 121b.

As mentioned above, the AI image processing model according to an embodiment of the present invention may be configured in a form including a plurality of sub-models, and the final result data is an average value of the primary result data output from each sub-model. method to calculate .

And, the plurality of models may be a structure in which a plurality of models of the same type of learning method are included (eg, a case in which a plurality of super-resolution models are composed of sub-models or a case in which a plurality of denoise models are composed of sub-models). , two models with different learning methods (super-resolution model and denoise model) may be combined as a sub-model. Accordingly, it is necessary for the learning unit 121 according to an embodiment of the present invention to be performed separately when learning the super-resolution model and the denoise model that can be used as a sub-model. Accordingly, the learning unit 121 may separate the super-resolution model learning unit 121a and the denoise model learning unit 121b to perform a learning operation through separate processes.

First, the super-resolution model learning unit 121a is a component of the learning unit 121 for learning the super-resolution model (SR model).

The super-resolution model (SR model) inputs the target value image data and LR (low-resolution image data) as a pair, and when the low-resolution image passes the convolutional neural network layer, the target value image data is calculated. ) is a model that is trained to obtain a value.

Accordingly, the super-resolution model learning unit 121a may select a target image and low-resolution image data, which are target values, as a pair of image data for training, and input them to the SR model in the learning step.

First, the super-resolution model learning unit 121a may select a conventional finished image photographed by accumulating a plurality of frames as target image data. In addition, as the low-resolution image data, an individual frame image obtained from the image management server 110 for inspection or a low-resolution version of a finished image obtained through data manipulation may be selected.

That is, the super-resolution model learning unit 121a selects the target value that is considered to be the completed inspection image obtained for the same subject, and the data obtained by converting the complete inspection image to low quality through data manipulation as a training image data pair. and can be fed into the SR model at the learning stage.

In addition, the super-resolution model learning unit 121a may perform learning by applying an additional effect to the image data for training in order to improve the image recognition performance of the AI model as a learning target. For example, the super-resolution model learning unit 121a may set a pair of training data in consideration of the illuminance, an image having a low image quality less than the reference value and a low illuminance value less than the reference value, high quality more than the reference value, and a high illuminance value more than the reference value The AI model can be trained by inputting the image pair with the image data for training. In this way, the super-resolution model learning unit 121a may perform learning with image data in which the illuminance of the image is adjusted, and the AI model that has been trained in this way may later produce a result image having an even illuminance value. .

In addition, the super-resolution model learning unit 121a selects only images having a data structure value in a preset range based on the data structure value (standard deviation; std value) of the image for learning to perform the learning of the super-resolution model. .

More specifically, each of the arbitrary image data may have a unique data structure value (std value).

The data structure value may show an increase in size as the complexity of the image increases, and the data structure value of the image to be trained depends on the purpose of the AI model to be finally created (property of the image to be improved in resolution). The range of (std value) can be changed.

Specifically, the super-resolution model learning unit 121a changes the range of the data structure value (std value) of the image to be selected for learning according to the properties of the image to be improved in quality, but the average complexity of the image to be improved is As it increases, the range of the data structure value (std) of the image to be selected for learning can be increased and set.

For example, assuming that the range of the data structure value (std) of the image data to be selected for training is selected for the AI model to be created as '20 to 50', the AI model to be created is the In the case of a model, an image having a std value of '20 to 80' may be selected for learning. As such, the range of the data structure value std selected as the learning image may be increased or decreased according to the properties of the target image for image quality improvement.

According to various embodiments, when the balance of the training image input in the learning step is broken, the image improvement performance of the AI model may be deteriorated. In order to prevent such a problem, the super-resolution model learning unit 121a according to an embodiment of the present invention may select only images having an std value within a reference range as image data for training. To this end, the super-resolution model learning unit 121a may exclude data having an extreme std value from the learning step. The super-resolution model learning unit 121a may remove images having a predetermined std value (eg, 0-20, 80-100) considered as extreme values from the training image.

The denoise model learning unit 121b may train the denoise model.

In this case, the denoise model may be an AI model based on a convolutional neural network (CNN) among deep learning models similar to a super-resolution model (SR model). However, the denoise model is somewhat different from the SR model in that it can be learned in a way that does not input target data first. The SR model aims to improve the resolution of the input target image, but the denoise model is an AI image processing model whose goal is to remove noise from the input model. Accordingly, unlike the SR model, the denoise model may be trained to remove noise.

The SR model was trained by inputting a pair of a low-quality image and a high-quality image (target image) as training data, but the denoise model learning unit 121b does not input a separate target image during the learning operation of the denoise model, and noise A number of images in which are present can be input as training data. The denoise model learning unit 121b may learn a plurality of noise images to find a noise deterioration pattern of the corresponding images, and calculate a weight value for performing a noise removal operation.

On the other hand, in general, when a loss value is calculated in the learning step through one specific neural network model, the learning is completed through a process in which a backpropagation operation is performed through the same model. However, the learning method of the denoise model according to an embodiment of the present invention may be performed in a different manner from the conventional backpropagation method.

That is, the denoise model learning unit 121b according to an embodiment of the present invention can train two denoise models, respectively. In the process of calculating the image data for training through the first denoise model 601, a loss value of 611 will be obtained. For reference, the loss value may mean a value calculated by how far the output value is from the correct answer. That is, the learning operation changes the weight value in the direction of reducing the loss value, and in this process, the back propagation operation is performed to calculate the optimal weight value.

The denoise model learning unit 121b according to an embodiment of the present invention uses the loss value 611 obtained through the first denoise model 601 as the basis of the backpropagation operation of the second denoise model 602 rather than the first denoise model. can be used Similarly, a loss value 612 calculated as a result of inputting arbitrary noise images to the second denoise model 602 as a training image may be the basis of a backpropagation operation in the first denoise model rather than the second denoise model.

As described above, the denoise model learning unit 121b according to an embodiment of the present invention repeats the process of performing the backpropagation operation by applying the loss value calculated through each of the two denoise models to the opposing denoise model. learning can be done through

In addition, according to various embodiments, the denoise model learning unit 121b may perform other more various learning methods. have. For example, in the first learning, frame 1 is put into the first denoise model and frame 2 is put into the second denoise model, and in the second learning, frames 1 and 3 are applied to the first denoise model, and the second In the denoise model, the configuration of the image data for training may be changed by sequentially changing the number of input frames as in the method of inputting the second and fourth frames.

An image quality improvement system comprising an image management server for inspection and an image processing server, wherein the image management server for inspection stores an image for inspection that is a target for image quality improvement, and the inspection that is a target for image quality improvement in the image quality improvement server provides an image for use, the image processing server trains an AI image processing model to be used for image quality improvement, and inputs a specific inspection image received from the inspection image management server into the AI image processing model to improve image quality However, the AI image processing model is configured by combining at least two or more AI models, and the target image quality can be improved by inputting the target image to each of a plurality of combined AI models and averaging the result data. have.

And the image processing server receives a plurality of image data from the image management server for inspection, and trains a sub-model constituting an AI image processing model with images selected for learning among the received image data, the AI image processing Includes a learning unit for learning each of the super-resolution model and the denoise model used in the model in different learning methods, and a processing unit for calculating the target image received from the image management server for inspection by inputting the pre-trained AI image processing model can be configured.

And the learning unit confirms the size of the data structure value of each image of a plurality of acquired image data, selects only images having a data structure value in a preset range for learning, a super-resolution model to perform learning of the super-resolution model The learning unit and a plurality of image frames photographed of the same subject are set as image data for training, but the entire frame is classified in half and put into each of the two AI models. It may be configured to include a denoise model learning unit that performs learning of the denoise model in a manner of performing .

The super-resolution model learning unit changes the data structure value range of the image to be selected for learning according to the properties of the image quality improvement target image, but as the complexity of the image quality improvement target image increases, the data structure value range of the image to be selected for learning can be set to increase.

In addition, although not shown in the drawings according to various embodiments, the image processing server 120 may further include a communication unit and a storage unit.

The communication unit may use a network for data transmission/reception between the user device and the server, and the type of the network is not particularly limited. The network may be, for example, an IP (Internet Protocol) network that provides a large-capacity data transmission/reception service through Internet Protocol (IP) or an All IP network that integrates different IP networks. In addition, the network includes a wired network, a Wibro (Wireless Broadband) network, a mobile communication network including WCDMA, a High Speed Downlink Packet Access (HSDPA) network, and a Long Term Evolution (LTE) network including a mobile communication network, LTE advanced (LTE-A) ), a mobile communication network including 5G (Five Generation), a satellite communication network, and a Wi-Fi network, or a combination of at least one or more of them.

The communication unit according to an embodiment of the present invention may perform a communication function required to receive an image for examination from the image management server 110 for examination.

In addition, the communication unit may perform a communication operation for transmitting the result data obtained by improving the resolution of the image in the image processing server 120 to the image management server 110 for inspection.

The storage unit may include, for example, an internal memory or an external memory. The internal memory includes, for example, a volatile memory (eg, dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM), etc.), non-volatile memory (eg, OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (such as NAND flash or NOR flash), hard drive, or a solid state drive (SSD).

External memory is a flash drive, for example, CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), XD (extreme digital), It may further include a multi-media card (MMC) or a memory stick. The external memory may be functionally and/or physically connected to the electronic device through various interfaces.

The storage unit according to an embodiment of the present invention may store algorithms and data related to a learning operation for the AI image processing model. Also, the storage unit may store an algorithm and data related to a resolution improvement operation of a target image performed through a pre-learned AI image processing model. Also, the storage unit may store result data calculated through the AI image processing model.

According to more various embodiments, the storage unit may further store information for determining the contents of the examination result (eg, whether there is an abnormality in the body, whether the product can be shipped, etc.) that the result data calculated through the AI image processing model means have. According to various embodiments of the present disclosure, not only the resolution improvement operation of the image is performed as described above, but also the operation of automatically checking information meaning the image based on the image as a result of the resolution improvement may be performed.

Although the present invention has been described in detail with reference to the above-described examples, those skilled in the art can make modifications, changes, and modifications to these examples without departing from the scope of the present invention. In short, in order to achieve the intended effect of the present invention, it is not necessary to separately include all the functional blocks shown in the drawings or follow all the orders shown in the drawings. Note that it may fall within the scope.

110: image management server for inspection
120: image processing server
121: study department
121a: super-resolution model training unit
121b: Denoise model learning unit
122: processing unit

Claims (5)

In response to a request from the user device, a service of transmitting the requested video data to the user device is performed, and a general-purpose neural network file required for an artificial neural network algorithm operation for improving the resolution of image information is generated based on the video data possessed; transmitting the generated general-purpose neural network file and low-quality video data obtained by changing the resolution to a preset level or less to the user device;
Receive low-quality video data, perform a calculation operation based on an artificial neural network algorithm that applies the received general-purpose neural network file to the received low-quality data, improve the resolution of the low-quality video data according to the calculation operation, and the resolution is reproducing the improved video data;
storing an image for inspection as a target for image quality improvement, and providing the image for inspection as a target for image quality improvement to the image quality improvement server;
Train an AI image processing model to be used to improve image quality,
A specific inspection image received from the inspection image management server is input to the AI image processing model to improve image quality, wherein the AI image processing model is configured by combining at least two AI models, and a plurality of combined improving the image quality of the target image by inputting the target image to each AI model and averaging the result data obtained; A control method comprising a. The method of claim 1,
The image for inspection is
A control method characterized in that it is a method of obtaining a finished image by accumulating a plurality of frames. The method of claim 1,
Receive a plurality of image data from the image management server for inspection, train a sub-model constituting an AI image processing model with images selected for learning among the received image data, super-resolution used in the AI image processing model training each of the model and the denoise model in different learning methods; and
A control method comprising a; inputting the target image received from the inspection image management server into the pre-trained AI image processing model and calculating. a memory storing one or more instructions; and
a processor executing the one or more instructions stored in the memory;
The processor by executing the one or more instructions,
An apparatus for performing the method of claim 1 . A computer program stored in a computer-readable recording medium in combination with a computer, which is hardware, to perform the method of claim 1.

Images