WO2020111333A1 - Image quality improvement system and method thereof - Google Patents

Abstract

The present invention relates to a system for improving image quality and a method thereof. The image quality improvement system according to the present invention comprises: a receiving unit for receiving image data and a user command; a memory for storing a program that performs image quality improvement on an object of interest of the image data according to a user command; and a processor for executing the program, wherein the processor analyzes the user command to determine a detection region of the image data and how to restore image quality, and performs image quality improvement.

Description

Image quality improvement system and method

The present invention relates to a system and method for improving image quality.

In order to improve the image quality according to the prior art, it is necessary to install a licensed software on a local computer and to learn how to use the software.

In addition, according to the prior art, in order to improve the image quality, the user has to perform a series of processes to select a specific area while viewing the image individually and select and apply a filter or an image quality improvement measure, so that the image information to perform the image quality improvement If exists in large quantities, there is a problem that it is difficult to perform all the tasks individually.

The present invention has been proposed to solve the above-mentioned problems, and it is possible to indicate an image quality improvement direction or an image quality improvement purpose through text or voice received from a service requester, and a large amount of data without a complicated user interface. An object of the present invention is to provide a system and method for improving image quality capable of processing.

The image quality improvement system according to the present invention includes a receiver for receiving image data and a user command, and a processor for executing a memory and a program in which a program for performing image quality improvement for an object of interest in the image data according to the user command is stored, The processor is characterized in that it analyzes a user command to determine a detection area and an image quality restoration method of image data, and performs image quality improvement.

The image quality improvement method according to the present invention includes receiving image data and a user command, analyzing a user command to determine a user's intention, and performing image restoration on a detection area of the image data according to the user's intention And, it characterized in that it comprises a step of performing an image quality improvement.

According to an embodiment of the present invention, it is possible to grasp a user's intention by analyzing a user command (voice, text, etc.), and perform image quality improvement on an object of interest in an image according to the user's intention, without a complicated user interface. It is possible to provide an image quality improvement service.

According to an embodiment of the present invention, when a large amount of data needs to be frequently analyzed and high-quality information about an object of interest needs to be obtained, such as in the case of VMS (Video Management System) operation or monitoring, it is convenient to use and improves image quality. It has the effect of increasing reliability.

The effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 and 2 are block diagrams illustrating an image quality improvement system according to an embodiment of the present invention.

3 is a flowchart illustrating a method for improving image quality according to an embodiment of the present invention.

The above-mentioned objects and other objects, advantages and features of the present invention and methods for achieving them will be clarified with reference to the embodiments described below in detail together with the accompanying drawings.

However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the following embodiments are intended for those skilled in the art to which the present invention pertains. It is merely provided to easily inform the configuration and effect, the scope of the present invention is defined by the description of the claims.

Meanwhile, the terms used in the present specification are for explaining the embodiments and are not intended to limit the present invention. In the present specification, the singular form also includes the plural form unless otherwise specified in the phrase. As used herein, "comprises" and/or "comprising" refers to the components, steps, operations and/or elements in which one or more other components, steps, operations and/or elements are present. Or added.

1 is a block diagram showing an image quality improvement system according to an embodiment of the present invention.

An image quality improvement system according to an embodiment of the present invention includes a receiver 100 that receives image data and a user command, and a program that performs image quality improvement on an object of interest in image data (still image, video) according to a user command. It includes a stored memory 200 and a processor 300 for executing a program, and the processor 300 analyzes a user command to determine a detection area and an image quality restoration method of image data, and performs image quality improvement.

The receiving unit 100 according to an embodiment of the present invention receives a user command in the form of voice or text, and when the receiving unit 100 receives the voice information, the processor 300 converts it into a text type sentence.

The processor 300 analyzes the user command in a morphological unit to obtain a key word, and detects the similarity to the word included in the first dictionary related to the detectable area and the second dictionary related to the image quality restoration method for the key word do.

The first dictionary is a dictionary mapping related words for detectable regions, and the second dictionary is a dictionary mapping related words related to image quality restoration processing technology.

The processor 300 calculates the similarity between the key words obtained as a result of the analysis of the user command and the words included in the first dictionary and the second dictionary, and performs area detection on areas where the similarity is greater than or equal to a preset threshold. , The image quality restoration processing is performed using a picture quality restoration processing technique in which the similarity is equal to or greater than a predetermined threshold.

When the image data is a moving image, the processor 300 extracts a still image from the moving image and repeatedly processes region detection and image quality restoration.

2 is a block diagram showing an image quality improvement system according to an embodiment of the present invention.

According to an embodiment of the present invention, for a video (still image or video) received from a service requester, a blurry object of interest (O_0) is converted into an object of interest (O_1) with improved image quality.

According to an embodiment of the present invention, the image quality to be improved for the detection area A_0 is improved, but the final image to be provided is provided only to the object of interest (O_1) having improved image quality, or as the image I_1 having the same size as the received image. You may.

To help those of ordinary skill in the art understand, the service requester is a monitoring agent of the Video Management System (VMS). Suppose it is a request such as "Please find and improve the picture quality", "Find and show the runaway teenagers", "Please restore the compressed video with JPEG".

In addition, the service requester is an employee who works for the police station, "Please increase the image quality of a person with a black hair 180cm tall", "OOO OO minutes, increase the image quality of cars in certain OO regions", "Red cars "Please show me clearly".

According to an embodiment of the present invention, a voice recognition unit 310 for receiving voice information as a user command and a sentence analysis unit 320 for receiving text information as a user command are included.

The speech recognition unit 310 uses the Speech To Text engine to create a text-type sentence, and the text-type sentence is provided as an input of the sentence analysis unit 320.

As described above, the sentence analysis unit 320 may receive a sentence directly from the service requester, and may also be provided with a text sentence through the voice recognition unit 310.

The speech recognition unit 310 and the sentence analysis unit 320 according to an embodiment of the present invention acquire "information for detecting which areas to be detected (detection area)" and "how to restore image quality" Information (how to restore image quality)".

Hereinafter, an embodiment of acquiring "information (detection area) for grasping which areas should be detected" will be described.

The area detection unit 330 according to an embodiment of the present invention receives a result of analyzing a user command from the sentence analysis unit 320.

At this time, the area detection unit 330 detects "face area", "body area", "leg area", "whole area", and "car area" through image processing, and detectable areas are continuously added to perform This can be improved.

The region detection unit 330 determines a detection region by using a dictionary (first dictionary) that maps related words to detectable regions.

For example, as shown in [Table 1] below, related words for detectable regions are mapped, and such information may be periodically updated or newly added.

Related words may be used several times for the detectable areas. In [Table 1] below, it can be seen that words such as "adult" and "child" are used together in related words related to the face area and the whole body area.

Face area	{Face, old man, elderly, infant, child, impression, face, head, glasses, nose, mouth, ear, cheek, cheek, sunglasses, hat, muffler, scarf, mask, ....}
Body area	{Body, shoulder, belly, belly, chest, chest, ...}
Leg area	{Legs, feet, knees, shins, thighs, ...}
Whole body area	{Elderly, elderly, infant, child, full body, whole body, person, man, woman, human, ...}
Automotive area	{Car, car, wheel, license plate, vehicle, escape vehicle, taxi, bus, truck ...}

The sentence analysis unit 320 obtains a key word by decomposing sentence information (a sentence as a result of converting an input sentence or an input voice) corresponding to a user command provided by a service requester into morphological units.

For example, if a sentence such as "Find a face of a 7-year-old child lost and make it clear without blurring" is input as a user command, {"road", "lost", "7 years old", "child", "face", "Blud" and "Clear"} are key keywords.

According to an embodiment of the present invention, the area detection unit 330 compares the obtained key keyword with a word included in the first dictionary, and calculates the similarity using a metric such as cosine similarity.

The area detection unit 330 according to an exemplary embodiment of the present invention performs area detection on areas having a similarity level greater than or equal to a threshold.

According to the above-described example sentence, it is highly likely that the request is made to detect the face region and the whole body region.

When it comes to calculating similarity, we use word analysis techniques such as word2vec technology, TF-IDF (Term Frequency-Inverse Document Frequency), Latent Semantic Analysis, and Latent Dirichlet allocation, which map word sets to a high-dimensional vector space and support similarity calculation. .

Hereinafter, an embodiment of acquiring "information for obtaining a picture quality restoration method (image quality restoration method)" will be described.

The image quality restoration unit 340 according to an embodiment of the present invention can improve JPEG image quality, improve super resolution, improve blur area, improve noise, and improve depth of field as an image processing method.

These image processing methods are continuously added, and performance can be improved.

As shown in [Table 2] below, the image quality restoration unit 340 uses a dictionary (second dictionary) including words related to image quality restoration processing technology.

JPEG quality improvement	{Linging phenomenon, blocking shape, JPEG, dices, dices, blocking, grid pattern removal, ...}
Super Resolution Improvement	{Low pixel, high definition, resolution increase, super resolution, image growth, ...}
Blur area improvement	{Night, Cloudy, Blur, Motion, Motion Blur, Motion Blur, ....}
Noise improvement	{Noise, sizzling, jizzling, salt pepper, low light, darkness, gain, ...}
Depth improvement	{Multifocal, depth improvement, DOF, depth of field, ...}

The sentence analysis unit 320 obtains a key word by decomposing sentence information (a sentence as a result of converting an input sentence or an input voice) corresponding to a user command provided by a service requester into morphological units.

For example, as described above, if the sentence "Please find the face of a 7-year-old child lost and make it clear and not blurry" is input as a user command, "blurred" for the related dictionary related to image quality restoration (second dictionary), "Vivid" is the main key keyword.

The image quality restoration unit 340 calculates the similarity using metrics such as cosine similarity by comparing key keywords with words in the second dictionary.

As a result of the similarity calculation, the image quality restoration unit 340 performs image quality restoration processing using an image quality restoration processing technique having a similarity of a threshold or higher.

In the above-described example, there is a high possibility that image quality improvement processing related to blur area improvement is used.

According to an embodiment of the present invention, a service requester may be provided with a plurality of processed images. For example, if “face area” and “whole area” are detected and blur area improvement and noise improvement are performed for each area, Results such as "Block area improvement" for "Face area", "Noise improvement" for "Face area", "Blur area improvement" and "Noise improvement" for "Full area" are processed simultaneously Can be.

According to an embodiment of the present invention, it is possible for a service requester who wants to receive a service to provide a still image or video that needs to be improved to the service platform, and to indicate the direction or purpose of improving the image quality through text or voice. .

Therefore, it is possible for a service requester to request processing of a large amount of data without a complicated user interface.

According to an embodiment of the present invention, even if the service requester does not have detailed technical know-how or experience in improving the image quality, it is possible to easily obtain an image with improved image quality.

As described above, when the service requester according to the embodiment of the present invention is a monitoring agent of the Video Management System (VMS) operated by a specific local government, "find the face of a 7-year-old child who is lost and make it clear without blurring." If instructed by text or voice, the service platform according to an embodiment of the present invention may perform a quality improvement process on an object of interest to meet a user's intention, thereby providing a reconstructed image Do.

3 is a flowchart illustrating a method for improving image quality according to an embodiment of the present invention.

An image quality improvement method according to an embodiment of the present invention includes receiving image data and a user command (S310), analyzing a user command to determine a user's intention (S320), and displaying the image data according to the user's intention. And performing image quality restoration on the detection area to perform image quality improvement (S330).

In step S310, when voice information is received by a user command, the voice information is converted into a text-type sentence.

Step S320 calculates the similarity between the key word obtained by analyzing the user command in morphological units and the second dictionary including the first dictionary including the related word for the detectable region and the related word for the image quality restoration method, Determine the user's intention (requests for objects of interest and how to restore image quality).

In step S330, when the image data is a moving image, a process of restoring the image quality of the extracted detection region of the still image is repeatedly performed.

Meanwhile, a method for improving image quality according to an embodiment of the present invention may be implemented in a computer system or recorded on a recording medium. The computer system may include at least one processor, memory, a user input device, a data communication bus, a user output device, and storage. Each of the above-described components communicates through a data communication bus.

The computer system can further include a network interface coupled to the network. The processor may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in memory and/or storage.

The memory and storage may include various types of volatile or nonvolatile storage media. For example, the memory may include ROM and RAM.

Accordingly, a method for improving image quality according to an embodiment of the present invention may be implemented in a method executable on a computer. When the method for improving image quality according to an embodiment of the present invention is performed on a computer device, computer-readable instructions may perform the method for improving image quality according to the present invention.

On the other hand, the above-described image quality improvement method according to the present invention can be implemented as a computer-readable code on a computer-readable recording medium. Computer-readable recording media includes all kinds of recording media storing data that can be read by a computer system. For example, there may be a read only memory (ROM), a random access memory (RAM), a magnetic tape, a magnetic disk, a flash memory, and an optical data storage device. In addition, the computer-readable recording medium may be distributed over computer systems connected through a computer communication network, and stored and executed as code readable in a distributed manner.

So far, we have focused on embodiments of the present invention. Those skilled in the art to which the present invention pertains will understand that the present invention may be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered in terms of explanation, not limitation. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the equivalent range should be interpreted as being included in the present invention.

Claims (8)

1. A receiving unit receiving image data and a user command;

   A memory in which a program for improving image quality of an object of interest is stored according to the user command; And

   It includes a processor for executing the program,

   The processor analyzes the user command to determine a detection area and an image quality restoration method of the image data, and performs image quality improvement

   Image quality improvement system.
2. According to claim 1,

   When the receiving unit receives the voice information by the user command, the processor converts it into a text-type sentence, and determines the detection area and image quality restoration method

   Image quality improvement system.
3. According to claim 1,

   The processor analyzes the user command, and determines the detection area using a first dictionary that maps an associated word for a detectable area, and determines the picture quality restoration method using a second dictionary related to picture quality restoration.

   Image quality improvement system.
4. According to claim 3,

   The processor calculates a degree of similarity between a key word obtained by analyzing the user command in a morphological unit and a word included in the first dictionary and the second dictionary, and a method of restoring an area and image quality having a similarity level higher than a preset threshold. To improve image quality according to

   Image quality improvement system.
5. (a) receiving image data and a user command;

   (b) analyzing the user command to grasp the user's intention; And

   (c) performing image quality improvement by performing image quality restoration on a detection area of the image data according to the intention of the user

   Image quality improvement method comprising a.
6. The method of claim 5,

   In step (a), when the voice information is received by the user command, the voice information is converted into a text-type sentence.

   How to improve the image quality.
7. The method of claim 5,

   In step (b), the similarity of the second dictionary including the first word including the key word obtained by analyzing the user command in morphological units and the related word for the detectable area and the related word for the image quality restoration method By calculating, to grasp the intention of the user

   How to improve the image quality.
8. The method of claim 5,

   In step (c), when the image data is a video, repeating a process of restoring image quality for the detection area of the extracted still image.

   How to improve the image quality.

Images