RU2779609C2 - Electronic device and method for its control - Google Patents

Abstract

FIELD: computer technology.

SUBSTANCE: invention relates to the field of computer technology for image processing. An electronic device includes: a storage device configured to store a set of filters, each filter corresponds to a set of image patterns; and a processor configured to classify an image block including a target pixel and a set of surrounding pixels to one of the set of image patterns based on the ratio between pixels in the image block and obtain a final image block, in which the target pixel is processed in the image by means of application of at least one filter corresponding to the classified image pattern among the set of filters to the image block. At the same time, the set of filters is obtained by means of studying by an artificial intelligence algorithm the ratio between a set of the first discrete image blocks and a set of the second discrete image blocks corresponding to the set of the first discrete image blocks based on each of the set of image patterns.

EFFECT: improvement of an image processing speed.

15 cl, 23 dwg

Description

The field of technology to which the invention belongs

[1] The present invention relates generally to an electronic device and a method for controlling the same, and, for example, an electronic device performing image processing and a method for controlling the same.

[2] The present invention also relates to an artificial intelligence (AI) system simulating a recognition function and a decision function of a human brain with a machine learning algorithm and its application.

State of the art

[3] Recently, an artificial intelligence system realizing human-level intelligence has been used in various fields. An artificial intelligence system is a system in which a machine performs learning and makes a decision and becomes intelligent in itself, unlike the existing rule-based intelligent system. As the artificial intelligence system is used more and more, the recognition speed improves and the user's manner can be more accurately understood, so that the existing rule-based intelligent system is gradually replaced by a deep learning-based artificial intelligence system.

[4] Artificial intelligence technology may include machine learning (eg, deep learning) and elemental technologies using machine learning.

[5] Machine learning may include the algorithmic technology of classifying/learning the distinguishing features of input data by itself, and elemental technology may include the technology of modeling functions such as recognition, decision making, etc., of the human brain using a machine learning algorithm such as deep learning, or the like, and may include technical fields such as linguistic understanding, visual understanding, logical reasoning/prediction, knowledge representation, motion control, and the like.

[6] The various fields to which artificial intelligence technology can be applied are as follows. Linguistic understanding may refer to human language recognition and application/processing technology, and may include natural language processing, machine translation, conversational system, question-answer, speech recognition/synthesis, or the like. Visual understanding may refer to object recognition and processing technology similar to human vision, and may include object recognition, object tracking, image search, person recognition, scene understanding, space understanding, image enhancement, or the like. Inference/prediction may refer to decision and inference and information prediction technology, and may include knowledge/probability based inference, optimization prediction, preference based planning, recommendation, or the like. The knowledge representation may refer to human experience automation and processing technology as knowledge data, and may include knowledge construction (data generation/classification), knowledge management (data exploitation), or the like. The motion control may refer to a vehicle self-driving control technology and motion control (navigation, collision, driving), manipulation control (behavior control), or the like.

[7] Meanwhile, the conventional image processing method can be divided into non-training technology and training-based technology. The technology without training has the advantage that the image processing speed is fast, but has the problem that flexible image processing depending on the characteristics of the image is not possible. The learning-based technology has the advantage that flexible image processing is possible, but has the problem that real-time processing is difficult.

[8] For example, consider the case of increasing the resolution of an image, by an interpolation method that is a characteristic method of technology without learning, the brightness of a pixel corresponding to a position at which the resolution is increased is calculated with a filter having low-pass characteristics. In detail, there is a spline-based bicubic interpolation method, a Lanczos filter resampling method formed by simplifying an ideal low-pass filter (Sinc Kernel), or the like. Such a non-training technique shows a stable image magnification performance due to low complexity, but may not reflect the previous information that only the image has, so that the edge sharpness is blurred, the edge is jagged, aliasing or ringing occurs near the edge.

[9] As a characteristic method of learning-based technology, there is a method of directly using a high-quality image database for reconstruction, a method of learning and using a high-resolution transform rule for each classified class, a method of learning a low-resolution/high-resolution transform in the form of an end-to-end matching through a deep learning network and zooming in using the trained network during image zooming.

[10] In learning-based technologies, the unique characteristics of the image signal are reflected in training and used during image enlargement, and learning-based technologies can thus recreate a sharp, jagged, and smooth edge compared to non-trained image enlargement methods. However, learning-based technologies are suitable for non-real-time applications due to high complexity, but it is difficult to apply learning-based technologies to real-time devices such as a television set (TV). In addition, it is difficult to apply learning-based technologies to a system-on-a-chip (SoC) implementation for real-time implementation.

[11] In addition, learning-based technologies show excellent performance with respect to edge components whose local features are clear, but exhibit interference components relative to a flat region whose feature is unclear or shows poor performance on the detail surface. In addition, learning-based technologies have a problem in that an image may not be enlarged relative to zoom without learning.

[12] Therefore, it was required to develop a technology capable of performing flexible image processing and improving image processing speed.

The essence of the invention

Technical task

[13] Exemplary embodiments of the present invention overcome the disadvantages described above and other disadvantages not described above.

[14] The present invention provides an electronic device performing image processing based on real-time learning and a method for controlling the same.

The solution of the problem

[15] According to an exemplary aspect of the present invention, the electronic device includes: a storage device configured to store a plurality of filters, each filter corresponding to a plurality of image patterns; and a processor configured to classify the image block including the target pixel and the plurality of surrounding pixels into one of the plurality of image templates based on the relationship between pixels in the image block, and obtain a final image block in which the target pixel is processed in the image by applying, of at least one filter corresponding to the classified image template, among the plurality of filters to the image block, wherein the plurality of filters are obtained by learning the relationship between the plurality of first discrete image blocks and the plurality of second discrete image blocks corresponding to the plurality of first discrete image blocks by means of an artificial intelligence based on each of the many image templates.

[16] The processor may compute (determine) the gradient vector from the image block, calculate (determine) the image block matches each of the plurality of index vectors based on an index matrix stored in the memory including the plurality of index vectors and the gradient vector, and classify the block images into one of the plurality of image templates based on the calculated (determined) matches, and the plurality of index vectors can be obtained based on the plurality of discrete gradient vectors calculated (determined) from the plurality of first discrete image blocks and corresponding to the plurality of image templates, respectively.

[17] The processor may compute a gradient vector from the image block when the image block's horizontal directivity intensity is greater than the image block's vertical directivity intensity, and may transpose the image block and compute a gradient vector from the transposed image block when the image block's horizontal directivity intensity is less than the vertical directivity intensity of the image block. Images.

[18] The processor may reduce the size of the gradient vector with the eigenvector stored in the memory, and compute image block mappings to each of the plurality of index vectors based on the reduced gradient vector, and the eigenvector may be created by applying principal component analysis from sets of discrete gradient vectors.

[19] The processor may calculate a first image block correspondence to each of the plurality of index vector groups based on the first index matrix stored in the memory and the gradient vector, obtain one of the plurality of index vector groups based on the first correspondence, calculate a second image block correspondence to each of a plurality of index vectors included in the obtained index vector group, based on a second index matrix corresponding to the obtained index vector group, among a plurality of second index matrices stored in the memory and a gradient vector, and obtain a final image block by applying at least one of the plurality of filters to the image block based on the second match, the plurality of index vector groups may be obtained by dividing the plurality of index vectors into a predetermined number of groups, the first index matrix may include index vectors, n representing each of the plurality of index vector groups, and each of the plurality of second index matrices may include a plurality of index vectors corresponding to each of the plurality of index vector groups.

[20] The processor can obtain one of the plurality of index vectors included in the obtained index vector group, based on the second match, obtain at least one additional index vector included in the others of the plurality of index vector groups and corresponding to the obtained index vector, on based on the same index vector information stored in the memory, compute a third image block match based on the obtained index vector, the obtained additional index vector, and the gradient vector, and obtain a final image block by applying at least one of the plurality of filters to the image block based on third match.

[21] The processor may obtain at least two of the plurality of filters based on the plurality of calculated matches, calculate the final filter based on at least two filters and the matches corresponding to each of the at least two filters, and obtain the final the image block by applying the computed final filter to the image block.

[22] The plurality of first discrete image blocks may be images in which the resolutions of the respective second discrete image blocks are reduced, respectively, and the processor may obtain a final image block in which the resolution of the target pixel is increased by applying at least one of the plurality of filters. to the image block.

[23] The processor may obtain an additional image block in which the resolution of the target pixel is upscaled by applying a learningless resolution upscaling technique to the image block, and update the final image block based on the best match of the set of calculated matches and the additional image block.

[24] The processor may calculate horizontal gradients and vertical gradients for the target pixel and each of the plurality of surrounding pixels, and calculate a gradient vector based on the horizontal gradients and vertical gradients.

[25] Each of the plurality of filters can be obtained by obtaining a plurality of first subsample image blocks corresponding to one of the plurality of index vectors, among the plurality of first discrete image blocks, obtaining a plurality of second subsample image blocks corresponding to the plurality of first subsample image blocks, among the plurality of second discrete image blocks, and learning the relationship between the plurality of first sub-sample image blocks and the plurality of second sub-sample image blocks by an artificial intelligence algorithm.

[26] According to another exemplary aspect of the present invention, a method for controlling an electronic device includes: classifying an image block including a target pixel and a plurality of surrounding pixels into one of a plurality of image templates based on a relationship between pixels in the image block; and obtaining a final image block in which the target pixel is processed in the image by applying at least one filter corresponding to the classified image template, among the plurality of filters, each corresponding to the plurality of image templates, to the image block, wherein the plurality of filters are obtained by studying the relationship between the plurality of first discrete image blocks and the plurality of second discrete image blocks corresponding to the plurality of first discrete image blocks by an artificial intelligence algorithm based on each of the plurality of image templates.

[27] The classification may include: calculating (determining) a gradient vector from an image block; calculating (determining) correspondences of the image block to each of the plurality of index vectors based on an index matrix including the plurality of index vectors and the gradient vector; and classifying the image block into one of the plurality of image templates based on the calculated (determined) matches, and the plurality of index vectors can be obtained based on the plurality of discrete gradient vectors calculated from the plurality of first discrete image blocks and corresponding to the plurality of image templates, respectively.

[28] When calculating the gradient vector, the gradient vector can be calculated from the image block when the intensity of the horizontal directivity of the image block is greater than the intensity of the vertical directivity of the image block, and the image block can be transposed, and the gradient vector can be calculated from the transposed image block when the intensity of the horizontal directionality of the image block is less than the intensity of the vertical directionality of the image block.

[29] The method may further include reducing the size of the gradient vector with an eigenvector, wherein, in the correspondence calculation, the image block's correspondences to each of the plurality of index vectors are calculated based on the gradient vector whose size is reduced, and the eigenvector is generated by applying analysis principal components from a set of discrete gradient vectors.

[30] Computing the correspondences may include: calculating a first correspondence of the image block to each of the plurality of index vector groups based on the first index matrix and the gradient vector; obtaining one of the plurality of index vector groups based on the first match; calculating a second correspondence of the image block to each of the plurality of index vectors included in the obtained index vector group, based on the second index matrix corresponding to the obtained index vector group, among the plurality of second index matrices and the gradient vector; and obtaining the final image block, the final image block is obtained by applying at least one of the plurality of filters to the image block based on the second match, the plurality of index vector groups can be obtained by dividing the plurality of index vectors into a predetermined number of groups, the first index matrix may include index vectors representing each of the plurality of index vector groups, and each of the plurality of second index matrices may include a plurality of index vectors corresponding to each of the plurality of index vector groups.

[31] Computing matches may include: deriving one of a plurality of index vectors included in the obtained index vector group based on a second match; obtaining at least one additional index vector included in others of the plurality of index vector groups and corresponding to the obtained index vector based on the same index vector information; and calculating a third image block match based on the obtained index vector, the obtained additional index vector, and the gradient vector, and, upon obtaining the final image block, the final image block can be obtained by applying at least one of the plurality of filters to the image block based on third match.

[32] Obtaining a final image block may include: obtaining at least two of the plurality of filters based on the plurality of computed matches; calculating a final filter based on the at least two filters and the matches corresponding to each of the at least two filters; and obtaining a final image block by applying the computed final filter to the image block.

[33] The plurality of first discrete image blocks may be images in which the resolutions of the respective second discrete image blocks are reduced, respectively, and when obtaining the final image block, the final image block in which the resolution of the target pixel is increased can be obtained by applying, by at least one of the plurality of filters to the image block.

[34] The method may further include: obtaining an additional image block in which the resolution of the target pixel is increased by applying a learningless resolution increasing technique to the image block; and updating the final image block based on the highest match of the set of calculated matches and the additional image block.

[35] When calculating the gradient vector, horizontal gradients and vertical gradients for the target pixel and each of the plurality of surrounding pixels can be calculated, and the gradient vector can be calculated based on the horizontal gradients and vertical gradients.

[36] Each of the plurality of filters can be obtained by obtaining a plurality of first subsample image blocks corresponding to one of the plurality of index vectors, among the plurality of first discrete image blocks, obtaining a plurality of second subsample image blocks corresponding to the plurality of first subsample image blocks, among the plurality of second discrete image blocks, and learning the relationship between the plurality of first sub-sample image blocks and the plurality of second sub-sample image blocks by an artificial intelligence algorithm.

Useful results of the invention

[37] According to various exemplary embodiments of the present invention, as described above, an electronic device can perform real-time image processing and thus be used in various applications, and can improve image processing quality compared to image processing without training by performing processing images based learning.

Brief description of the drawings

[38] The foregoing and/or other aspects, features and accompanying advantages of the present invention will be more apparent and easily understood from the following detailed description taken in conjunction with the accompanying drawings, in which like reference numerals refer to like elements, and in which:

[39] FIG. 1A is a block diagram illustrating components of an exemplary electronic device according to an exemplary embodiment of the present invention;

[40] FIG. 1B is a block diagram illustrating an example of the components of an electronic device;

[41] FIG. 2 is a diagram illustrating exemplary operations of a processor according to an exemplary embodiment of the present invention;

[42] FIG. 3A, 3B, 3C, 3D and 3E are diagrams illustrating an exemplary method for deciding on the directionality of an image block and generating a gradient vector according to an exemplary embodiment of the present invention;

[43] FIG. 4A and 4B are diagrams illustrating an exemplary size reduction of a gradient vector feature according to an exemplary embodiment of the present invention;

[44] FIG. 5A and 5B are diagrams illustrating an exemplary filter search with an index matrix according to an exemplary embodiment of the present invention;

[45] FIG. 6A, 6B, and 6C are diagrams illustrating an exemplary method for reducing the amount of computation for filter search according to an exemplary embodiment of the present invention;

[46] FIG. 7A and 7B are diagrams illustrating an exemplary computational reduction method for filter search according to another exemplary embodiment of the present invention;

[47] FIG. 8A and 8B are diagrams illustrating an exemplary method for applying a filter to an image block according to an exemplary embodiment of the present invention;

[48] FIG. 9A and 9B are diagrams illustrating an exemplary training method for a filter according to an exemplary embodiment of the present invention;

[49] FIG. 10 is a flowchart illustrating an exemplary streaming operation according to an exemplary embodiment of the present invention; and

[50] FIG. 11 is a flowchart illustrating an exemplary method for controlling an electronic device according to an exemplary embodiment of the present invention.

Optimal Mode for Carrying Out the Invention

[51] Mode of carrying out the invention

[52] Various exemplary embodiments of the present invention may be modified in various ways. Accordingly, specific exemplary embodiments are illustrated in the drawings and described in detail in the specification. However, it should be understood that the present invention is not limited to the specific exemplary embodiment, but includes all modifications, equivalents, and substitutions without departing from the scope and spirit of the present invention. Also, well-known functions or constructions may not be described in detail when they confuse the description of the invention with unnecessary detail.

[53] Hereinafter, various embodiments of the present invention will be described in more detail with reference to the accompanying drawings.

[54] FIG. 1A is a block diagram illustrating exemplary components of an electronic device 100 according to an exemplary embodiment of the present invention. As illustrated in FIG. 1A, electronic device 100 includes a memory 110 and a processor (eg, including processing circuitry) 120.

[55] The electronic device 100 may be an image processing device. For example, the electronic device 100 may be a device that performs image processing based on data learned by an artificial intelligence algorithm. For example, the electronic device 100 may, for example, and without limitation, be a desktop personal computer (PC), a laptop computer, a smartphone, a tablet PC, a server, a refrigerator, a washing machine, a sign, or the like. In addition, the electronic device 100 may, for example and without limitation, be the system itself in which the cloud computing environment is configured. However, the electronic device 100 is not limited to this, but may be a device that can perform image processing.

[56] The memory 110 may store a plurality of filters corresponding to each of the plurality of image templates. Here, a plurality of image patterns can be classified depending on the characteristics of the image. For example, the first image template may be an image template having a plurality of lines in a horizontal direction, and the second image template may be an image template having a plurality of lines in a rotation direction.

[57] The memory 110 may further store an index matrix including a plurality of index vectors. A plurality of index vectors may be derived based on a plurality of discrete gradient vectors calculated (determined) from a plurality of first discrete image blocks, and may correspond to a plurality of image templates, respectively. A plurality of filters can be obtained by learning a relationship between a plurality of first discrete image blocks and a plurality of second discrete image blocks corresponding to the plurality of first discrete image blocks by an artificial intelligence algorithm based on each of the plurality of index vectors.

[58] The index vector plurality and the filter plurality may be created by the external server instead of the electronic device 100. However, the index vector plurality and the filter plurality may also be created by the electronic device 100. In this case, the storage device 110 may store a plurality of first discrete image blocks; and a plurality of second discrete image blocks.

[59] A method for generating gradient vectors, a plurality of index vectors, and a plurality of filters are described below.

[60] The storage device 110 may store at least one image block. Memory 110 may store a plurality of first discrete image blocks and a plurality of second discrete image blocks. In addition, the memory 110 can store the filter without learning, and the kind of the memory is not limited.

[61] The storage device 110 may, for example, and without limitation, be implemented by a hard disk, non-volatile memory, volatile memory, or the like. and can be any component that can store data.

[62] The processor 120 may include various processing circuits and generally controls the operation of the electronic device 100.

[63] According to an exemplary embodiment, processor 120 may be implemented, for example, and without limitation, by a digital signal processor (DSP), a microprocessor and/or a timing controller (TCON), or the like. However, the processor 120 is not limited to this, but may include, for example and without limitation, one or more of a dedicated processor, a central processing unit (CPU), a microcontroller unit (MCU), a microprocessor (MPU), a controller, an application processor (AP ), a communications processor (CP), an ARM processor and/or may be defined by these terms, or the like. In addition, processor 120 may be implemented by a system on a chip (SoC) or large scale integrated circuit (LSI) in which a processing algorithm is embedded, and/or may be implemented in the form of a field programmable gate array (FPGA).

[64] The processor 120 may receive an image block including a target pixel and a plurality of surrounding pixels input from an external server. In this case, processor 120 may store an image block in memory 110. Processor 120 may read an image block stored in memory 110. Processor 120 may read an image block having a predetermined size from an image frame stored in memory 110. For example, processor 120 may read the first 3×3 image block for the upper left edge of the image frame and may process the image of the first image block. In addition, the processor 120 may read the second 3×3 image block moved to the right from the upper left edge of the image frame by a single pixel, and may process the image of the second image block. Thus, processor 120 can perform image processing over the entire image frame.

[65] The target pixel may be the pixel that becomes the target of the filtering to be described below. That is, the target pixel may be filtered based on pixel values for a plurality of surrounding pixels.

[66] Processor 120 may classify an image block including a target pixel and a plurality of surrounding pixels into one of a plurality of image templates based on a relationship between pixels in the image block. For example, when the difference between pixel values for pixels adjacent to each other in the horizontal direction is smaller than the difference between pixel values for pixels adjacent to each other in the vertical direction, processor 120 may classify the image block into an image template having vertical direction characteristics.

[67] However, the present invention is not limited to this, and the processor 120 may also obtain a ratio between pixels in an image block depending on various methods.

[68] In addition, the processor 120 can obtain a final image block in which a target pixel is processed in the image by applying at least one filter corresponding to the classified image template among the plurality of filters to the image block.

[69] As another example, to classify an image block, processor 120 may compute (determine) a gradient vector from an image block. For example, processor 120 may calculate horizontal gradients and vertical gradients for the target pixel and each of the plurality of surrounding pixels, and calculate a gradient vector based on the horizontal gradients and vertical gradients. The gradient vector may indicate change amounts for pixels existing in a predetermined direction with respect to each pixel. That is, the characteristics of the image block can be detected by the gradient vector.

[70] For example, processor 120 may compute a horizontal gradient and a vertical gradient for a 3×3 image block using the Sobel operator. Each of the horizontal gradient and vertical gradient may include nine elements, and the processor 120 may arrange the elements of each of the horizontal gradient and vertical gradient in a predetermined sequence and convert the elements into vector form. The predetermined sequence may be specified in various ways. However, the predetermined sequence may be the same as the sequence for calculating the gradient vector in the process of creating the index vector set and the filter set, which will be described below.

[71] The processor 120 may calculate the image block gradient vector by various methods in addition to the Sobel operator, and the various methods are not particularly limited. In addition to the horizontal and vertical gradient vectors, a gradient vector having a different angle can be calculated, and any method can be used, as long as the gradient vector is calculated by the same method as the method used in the process of creating the index vector set and the filter set. which should be described below. Hereinafter, the case in which the gradient vector c × 1 is calculated is described for the convenience of explanation. Here, c may vary depending on the size of the image block.

[72] Processor 120 may calculate image block mappings to each of the plurality of index vectors based on the index matrix stored in memory 110 and the gradient vector. The index matrix may include a plurality of index vectors, and the plurality of index vectors may be derived based on the plurality of discrete gradient vectors computed from the plurality of first discrete image blocks.

[73] For example, in a case where 1,000,000 discrete gradient vectors calculated from the first 1,000,000 discrete image blocks are divided into groups for each of the characteristics of the image block, each of the plurality of index vectors may be a vector representing each group. That is, the image block's correspondences to each of the plurality of index vectors may mean the similarities of the image block to each of the groups. In addition, an image block can be considered to have characteristics similar to those of the group corresponding to the highest match of the matches.

[74] However, this is only an example, and the number of first discrete image blocks and the number of groups can be changed. In addition, the number of groups can also be changed depending on the type of filter. For example, in the case of a resolution-enhancing filter, the number of groups is a, but in the case of a sharpening filter, the number of groups may not be a.

[75] Processor 120 may obtain a final image block in which a target pixel is processed in the image by applying at least one filter from a plurality of filters to the image block based on the computed matches. Here, the filter may be a resolution-enhancing filter, a sharpening filter, or the like, but is not limited to this, and may also be any filter associated with image processing.

[76] Further, the filter set can be obtained by learning the relationship between the first discrete image block plurality and the second discrete image block plurality corresponding to the first discrete image block plurality by an artificial intelligence algorithm based on each of the plurality of index vectors.

[77] For example, each of the plurality of filters can be obtained by obtaining a plurality of first subsample image blocks corresponding to one of the plurality of index vectors, among the plurality of first discrete image blocks, obtaining a plurality of second subsample image blocks corresponding to the plurality of first subsample image blocks, among the plurality of second discrete image blocks, and learning the relationship between the plurality of first sub-sample image blocks and the plurality of second sub-sample image blocks by an artificial intelligence algorithm.

[78] In the example described above, each of the plurality of filters may, for example and without limitation, be a resolution-enhancing filter obtained by examining the image characteristics of one of the groups by an artificial intelligence algorithm. In the case in which the first image features are represented by the first index vector, and the first filter corresponding to the first index vector exists, and the second image features are represented by the second index vector, and the second filter corresponding to the second index vector exists, when the correspondence between the block image and the first image characteristics is high, the image block can show more improved quality in the case in which its resolution is increased by the first filter compared to the case in which its resolution is increased by the second filter. This is because the first filter and the second filter are filters trained as suitable for the first image features and the second image features, respectively.

[79] Processor 120 may compute a gradient vector from the image block when the horizontal directivity intensity of the image block is greater than the vertical directivity intensity of the image block, and may transpose the image block and calculate a gradient vector from the transposed image block when the horizontal directivity intensity of the image block is less than the vertical directivity intensity. image block.

[80] For example, processor 120 may transpose an image block whose vertical directionality is dominant with respect to an image block whose horizontal directionality is dominant, and compute a gradient vector from an image block whose horizontal directionality is dominant. In this case, the number of index vectors can be reduced and the calculation speed can thus be improved.

[81] For example, in a case in which an image block having a horizontal directionality is divided into first groups for each of the image characteristics, an image block having a vertical directionality may also be divided into second groups for each of the image characteristics. Therefore, a total of 2×a groups can be generated for each of the image features, and a total of 2×a index vectors can be generated.

[82] Therefore, in the case where the processor 120 transposes an image block whose vertical directionality is dominant, the transposed image block may have a horizontal directionality and may be included in one of the first groups. That is, processor 120 can process an image block belonging to one of the second groups in one of the first groups by transposing an image block whose vertical directionality is dominant, and can process an image block whose horizontal directionality is dominant in one of the first groups. , without transposing the image block whose horizontal orientation is dominant. Therefore, the number of index vectors can be reduced by half depending on the transposition operation, and the calculation speed can be improved.

[83] The processor 120 may obtain a final image block in which the target pixel is processed in the image by applying a filter to the transposed image block, and transpose the image block again. In the case of an image block that is not transposed, the operation of applying a filter to the image block and then transposing the image block again may be skipped.

[84] Meanwhile, although the case in which processor 120 transposes an image block whose vertical directionality is dominant has been described above herein, this is only an example, and processor 120 may also transpose an image block whose horizontal directionality is dominant.

[85] Meanwhile, the processor 120 may reduce the size of the gradient vector with the eigenvector stored in the memory 110 and calculate image block correspondences to each of the plurality of index vectors based on the gradient vector whose size is reduced. Here, an eigenvector may be created by applying principal component analysis from a plurality of discrete gradient vectors.

[86] When the eigenvector is not used in the state in which the index vectors exist, the index matrix may be of the form a × c. In this regard, when the gradient vector c × 1 is reduced to the gradient vector c' × 1 using an eigenvector, the index matrix may be of the form a × c'. In detail, the sizes of features of the plurality of index vectors included in the index matrix a × c may be reduced by the same eigenvector, and the index matrix including the plurality of index vectors whose features are reduced in size may be of the form a × c'. Here, the eigenvector may be of the form c' × c, and c' may be less than c.

[87] The calculation speed can be improved in the case of using an index matrix a × c' compared to the case of using an index matrix a × c.

[88] Meanwhile, principal component analysis is a statistical method of extracting principal components that succinctly expresses the variability of sample types of a set of variables as a linear combination of the original variables. That is, in the case where there are p variables, the information obtained from the p variables can be reduced to k variables significantly less than p. In the above example, the dimensions of the features c can be considered reduced to the dimensions c' of the features. However, the number of reduced feature sizes is not limited, but may also be changed.

[89] Next, this document describes the case of reducing the matrix of indices and gradient vectors using an eigenvector. However, the present invention is not limited to this, and a configuration that does not use an eigenvector corresponds to an exemplary embodiment of the present invention.

[90] Meanwhile, the processor 120 may calculate a first image block correspondence to each of the plurality of index vector groups based on the first index matrix stored in the memory 110 and the gradient vector, obtain one of the plurality of index vector groups based on the first correspondence, and obtain a second an index matrix corresponding to the obtained index vector group among the plurality of second index matrices stored in the memory 110.

[91] Here, the index vector group set can be obtained by dividing the index vector set into a predetermined number of groups. For example, a plurality of index vector groups may be divided into a total of four groups depending on characteristics, and each of the four groups may include b index vectors. That is, b index vectors in the same group can be similar to each other.

[92] The first index group may include index vectors representing each of the plurality of index vector groups. In the example described above, in each of the four groups, one of the b index vectors may be set to represent, and the first index matrix may be formed with the four index vectors representing the respective groups. Here, the first index matrix may be of the form 4 × c'.

[93] Each of the plurality of second index matrices may include a plurality of index vectors corresponding to each of the plurality of index vector groups. For example, each of the set of index vectors may be of the form b × c'.

[94] In the example described above, processor 120 may determine which of the plurality of index vector groups the image block belongs to based on the first match, and obtain a second index matrix corresponding to the corresponding group from the plurality of second index matrices.

[95] The processor 120 may calculate a second image block correspondence to each of the plurality of index vectors included in the obtained index vector group based on the obtained second index matrix and the gradient vector, and may obtain the final image block by applying at least one of the plurality of filters to the image block based on the second match. The calculation speed can be improved by the method as described above.

[96] However, this is only an example, and the number of index vector groups can be changed. In addition, the numbers of index vectors included in the plurality of index vector groups may differ from each other.

[97] Meanwhile, according to the exemplary embodiment described above, matches with only some of the total a index vectors can be calculated. Therefore, the calculation of matches with other index vectors is skipped, and in the case in which the grouping is inappropriate, the index vector having the highest match may not be found. Further in this document, a method of supplementing this is described.

[98] The processor 120 can obtain one of the plurality of index vectors included in the obtained index vector group, based on the second match, obtain at least one additional index vector included in the others of the plurality of index vector groups and corresponding to the obtained index vector, based on the same index vector information stored in memory 110, calculate a third image block match based on the obtained index vector, the obtained additional index vector, and the gradient vector, and obtain a final image block by applying at least one of the plurality of filters to the image block based on the third match.

[99] That is, the memory 110 can further store similar index vector information, which is information about additional index vectors included in different index vector groups for each index vector and having similar characteristics.

[100] When one index vector is obtained based on the second match, processor 120 may calculate an image block to correspond to additional index vectors having characteristics similar to those of the obtained index vector based on information stored in memory 110. Processor 120 may obtain at least at least one filter based on the image block match for the resulting index vector and the image block match for additional index vectors.

[101] In this case, the additional multiplication is performed in the process of calculating the correspondence of the image block to the additional index vectors, so that the speed can be reduced compared to the previous exemplary embodiment, but the accuracy can be improved compared to the previous exemplary embodiment. In addition, in the case where the number of additional index vectors is not numerous, the speed reduction may be small.

[102] Processor 120 may obtain at least two of the plurality of filters based on the plurality of calculated matches, compute a final filter based on at least two filters and the matches corresponding to each of the at least two filters, and obtain a final image block by applying the computed final filter to the image block.

[103] For example, processor 120 may obtain a first filter and a second filter and calculate a final filter by weighting and summing the first filter and the second filter based on the first match and the second match corresponding to each of the first filter and the second filter.

[104] However, the present invention is not limited to this, and the processor 120 may also calculate the final filter from the first filter and the second filter without regard to matches.

[105] Meanwhile, the plurality of first discrete image blocks may be images in which the resolutions of the respective second discrete image blocks are reduced, respectively, and the processor 120 may obtain a final image block in which the resolution of the target pixel is increased by applying at least one of the many filters to the image block.

[106] In this case, the plurality of filters may be filters learned and obtained by an artificial intelligence algorithm to obtain respective second discrete image blocks in which the resolutions are increased from each of the plurality of first discrete image blocks.

[107] The processor 120 may obtain an additional image block in which the resolution of the target pixel is increased by applying a learningless resolution upscaling technique to the image block, and update the final image block based on the best match of the set of calculated matches and the additional image block.

[108] For example, processor 120 may select the largest match from the set of calculated matches and determine the level of change of the final image block based on the selected match. In detail, processor 120 can minimize and/or reduce the change of the target image block by decreasing the impact of the additional image block when the selected match becomes large, and increase the change of the end image block by increasing the influence of the additional image block when the selected match becomes small.

[109] FIG. 1B is a block diagram illustrating exemplary components of an electronic device 100. As illustrated in FIG. 1B, electronic device 100 includes a storage device 110, a processor (for example, including processing circuits) 120, a communication device (for example, including communication circuits) 130, a display 140, a user interface 150, an audio processor (for example, including audio processing circuitry) 160; and a video processor (eg, including video processing circuitry) 170. A detailed description of components overlapping with those illustrated in FIG. 1A, among the components illustrated in FIG. 1B may not be repeated here.

[110] The processor 120 may include various processing circuits and generally controls the operation of the electronic device 100 through various programs stored in the storage device 110.

[111] For example, processor 120 may include, without limitation, random access memory (RAM) 121, read only memory (ROM) 122, main central processing unit (CPU) 123, graphics processor 124, first through n-th interfaces 125-1 to 125-n and bus 126.

[112] RAM 121, ROM 122, main CPU 123, GPU 124, first to n-th interfaces 125-1 to 125-n, and the like, can be connected to each other via bus 126.

[113] First to n-th interfaces 125-1 to 125-n are connected to various components described above. One of the interfaces may be a network interface connected to an external server via a network.

[114] The main CPU 123 accesses the storage device 110 to perform initial boot using the operating system (O/S) stored in the storage device 110. In addition, the main CPU 123 performs various operations with various programs or the like. ., stored in the storage device 110.

[115] A set of instructions for booting a system or the like. may be stored in the ROM 122. When a power-on command is input to supply power to the main CPU 123, the main CPU 123 copies the operating system (O/S) stored in the memory 110 to the RAM 121 depending on the instruction stored in the ROM 122, and performs an O/S to boot the system. When the boot is completed, the main CPU 123 copies the various application programs stored in the memory 110 to the RAM 121 and executes the application programs copied to the RAM 121 to perform various operations.

[116] The graphics processor 124 renders a screen including various objects such as an icon, an image, text, and the like with a calculation unit (not illustrated) and a playback unit (not illustrated). A calculation module (not illustrated) calculates attribute values such as coordinate values in which corresponding objects will be displayed, shapes, sizes, colors, and the like. corresponding objects depending on the screen layout based on the received control command. A rendering module (not illustrated) reproduces screens of various layouts including objects based on attribute values calculated in a calculation module (not illustrated). The screen reproduced by the playback unit (not illustrated) is displayed in the display area of the display 140.

[117] Meanwhile, the operation of the processor 120 described above can be performed by a program stored in the memory 110.

[118] The storage device 110 stores various data such as an operating system (O/S) program module for driving the electronic device 100, an image block analysis module, a filter module, and the like.

[119] The communication device 130 may include various communication schemes and is a component that communicates with various types of external servers in various types of communication methods. Communication device 130 may include various communication schemes included in various communication chips, such as, for example, and without limitation, wireless fidelity (WiFi) chip 131, Bluetooth chip 132, wireless chip 133, near field communication chip 134 (NFC) or the like. The processor 120 communicates with various external servers via the communication device 130 .

[120] The WiFi chip 131 and the Bluetooth chip 132 communicate in the WiFi method and the Bluetooth method, respectively. In the case of using the WiFi chip 131 or the Bluetooth chip 132, various kinds of connection information such as a service set identifier (SSID), session key, and the like are first transmitted and received, communication is established using the connection information, and different kinds of information can then be transmitted and received. The wireless communication chip 133 means a chip that communicates depending on various communication protocols such as Institute of Electrical and Electronics Engineers (IEEE) protocol, Zigbee, 3rd generation (3G), 3rd generation partnership project (3GPP), long-term development (LTE), etc. NFC chip 134 means a chip operating in an NFC way using a 13.56MHz band among various radio frequency identification (RFID) frequency bands such as 135kHz, 13.56MHz, 433MHz, 860-960MHz, 2.45GHz etc.

[121] Processor 120 may receive an index matrix, a plurality of filters, and the like. from an external server through the communication device 130. Alternatively, processor 120 may receive image blocks and the like. in real time from an external server through the communication device 130 .

[122] The display 140 may be implemented by various types of displays, such as, for example, and without limitation, a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display panel (PDP), or the like. A drive circuit, a backlight unit, and the like, which may be implemented in a form such as amorphous silicon, low temperature polycrystalline silicon (LTPS), a thin film transistor (TFT), an organic transistor (OTFT), or the like, may be included in the display 140. Meanwhile, the display 140 may be realized by a touch screen by combining with a touch sensor.

[123] The processor 120 may control the display 140 to display the image block and the end image block.

[124] The user interface 150 receives various user instructions. Here, the user interface 150 may be implemented in various forms depending on the implementation of the electronic device 100. For example, the user interface 150 may include various interface circuits such as, for example, and without limitation, a button included in the electronic device 100, a microphone , receiving the user's voice, a camera detecting the user's movement, or the like. In the case where the electronic device 100 is implemented by a touch electronic device, the user interface 150 may also be implemented in the form of a touch screen forming a layered structure together with the touch panel. In this case, the user interface 150 may be used as the display 140 described above.

[125] The audio processor 160 is a component that performs audio data processing. In the audio processor 160, various audio processing circuits may be enabled and various kinds of processing, such as, for example, and without limitation, decoding, amplification, noise filtering, or the like, may be performed on the audio data.

[126] The video processor 170 is a component that performs video data processing. Various video processing circuits may be included in the video processor 170, and various kinds of image processing may be performed, such as, for example, and without limitation, decoding, scaling, noise filtering, frame rate conversion, resolution conversion, or the like, may be performed on the video data. .

[127] Meanwhile, the model including the index matrix, the plurality of filters, and the like provided by the external server as described above is a decision model learned based on the artificial intelligence algorithm, and may be, for example, a model based on a neural network. The learned decision model may be designed to simulate the structure of the human brain on a computer, and may include a plurality of network nodes that model the neurons of a human neural network and have weights. A plurality of network nodes can form a connection relationship among themselves to model the synaptic activity of neurons transmitting and receiving signals across synapses. In addition, the learned decision model may include, for example, a neural network model or a deep learning model developed from a neural network model. In a deep learning model, many network nodes can be located at different depths (or layers) and can transmit and receive data among themselves depending on the connection ratio of the gyrus. An example of a learned decision model may include, for example, and without limitation, a deep neural network (DNN), a recurrent neural network (RNN), a bidirectional recurrent deep neural network (BRDNN), or the like, but is not limited to this.

[128] In addition, the electronic device 100 may use a personal assistant program, which is a dedicated artificial intelligence program (or artificial intelligence agent), to receive a final image block obtained by applying a filter to the image block as described above. In this case, the personal assistant program, which is a dedicated program for providing an artificial intelligence (AI) service, may be executed by an existing general purpose processor (for example, a central processing unit (CPU)) or a separate dedicated AI processor (for example, a GPU (GPU)).

[129] In a case in which predetermined user input (for example, photographing an image depending on the manipulation of a photographing button, or the like) is input, or an image block is input from an external device, the artificial intelligence agent may be activated (or performed ).

[130] According to another exemplary embodiment of the present invention, the electronic device 100 may transmit an input image block to an external server. The external server may obtain the final image block by directly applying the filter to the image block and transmit the resulting final image block to the electronic device 100.

[131] The processor 120 can improve image processing quality by performing image processing based on real-time learning through the method described above.

[132] Hereinafter, exemplary operations of the processor 120 are described in more detail with reference to the drawings.

[133] FIG. 2 is a diagram illustrating exemplary operations of a processor 120 according to an exemplary embodiment of the present invention.

[134] When an image block is input, processor 120 may first perform gradient extraction 210 for the image block. For example, processor 120 may decide on the directionality of the image block and create a gradient vector. In addition, the processor 120 may also transpose the image block based on the result of the directionality decision of the image block.

[135] Processor 120 may perform variance size reduction 220 of the gradient vector with eigenvector 260.

[136] Processor 120 may search 230 for a filter on a filter to be applied to a gradient vector whose feature size has been reduced using an index matrix 270 .

[137] The processor 120 may obtain at least one of the plurality of filters included in the filter database DB 280 based on the search result and apply at least one filter to the image block (filter application 240).

[138] The plurality of filters included in the filter database 280 may be learning-based filters. For example, a plurality of filters can be obtained by learning the relationship between a plurality of first discrete image blocks and a plurality of second discrete image blocks corresponding to the plurality of first discrete image blocks through an artificial intelligence algorithm. That is, the operation described above can be considered as applying a learning-based filter to an image block.

[139] Meanwhile, the processor 120 may apply a non-training filter to the image block (application of the non-training filter 250) separately from the operation described above. For example, processor 120 may perform a learning-based filter application and a non-training filter application on the same image block in parallel.

[140] In addition, the processor 120 may weight and sum the first result, depending on the application of the training-based filter, and the second result, depending on the application of the filter without training.

[141] Meanwhile, the eigenvector 260, the index matrix 270, and the filter database 280 may be created with a plurality of first discrete image blocks and a plurality of second discrete image blocks corresponding to the plurality of first discrete image blocks.

[142] FIG. 2, transposing the image block depending on the result of the directionality decision, reducing the size of the gradient vector feature, and applying the filter without training are optional operations and can be skipped.

[143] FIG. 3A, 3B, 3C, 3D and 3E are diagrams illustrating an exemplary method for deciding on the directionality of an image block and generating a gradient vector according to an exemplary embodiment of the present invention.

[144] As illustrated in FIG. 3, processor 120 may decide the directionality of the image block using masks such as Sx and Sy. For example, processor 120 may decide on the directionality of an image block by applying a 5×5 image block and masks to the following Equation 1:

[145] [Equation 1]

[146]

[147] Here, GV indicates the vertical directionality of the image block, GH indicates the horizontal directionality of the image block, n and m indicate indexes for identifying row and column, respectively, and i and j indicate the start points of the image block in the image frame.

[148] The processor 120 can obtain the directionality of the image block by comparing the vertical directionality amount and the horizontal directionality amount with each other.

[149] The processor 120 may determine whether or not to transpose the image block based on the directionality used in the case of creating an eigenvector, an index matrix, and a plurality of filters. For example, in the case where only horizontal directionality is used and vertical directionality is not used during creation of the eigenvector, index matrix, and filter set, processor 120 does not transpose the image block when the image block is horizontal, and may transpose the image block when the image block has a vertical orientation. In the case in which only vertical directionality is used and horizontal directionality is not used during creation of the eigenvector, index matrix, and filter set, processor 120 does not transpose the image block when the image block is vertically directed, and may transpose the image block when the image block has a horizontal direction.

[150] FIG. 3B, the base block indicates the directionality used in the case of creating an eigenvector, an index matrix, and a filter set.

[151] In the case where the base unit is vertically oriented, as in the upper edge in FIG. 3B, processor 120 may only transpose the right image block from the left two image blocks. In the case where the base unit has a horizontal orientation, as in the bottom edge in FIG. 3B, processor 120 may only transpose the right image block from the left two image blocks.

[152] Since only one of vertical directivity and horizontal directivity is used, the data volumes for the eigenvector, the index matrix, and the filter set can be reduced. In addition, additional calculation depending on the directionality decision and the transposition of the image block can be generated, but the amount of calculation can be reduced in subsequent operations due to the reduction in data amounts for the eigenvector, index matrix, and filter set. For example, the total amount of computation may be reduced depending on the direction decision operation.

[153] Meanwhile, masks such as Sx and Sy illustrated in FIG. 3A are just an example, and processor 120 may decide the directionality of the image block in a different manner. Furthermore, although only 3×3 pixels in a 5×5 image block are illustrated in FIG. 3B, this is to illustrate directionality. Processor 120 may decide on the directionality of the image block by applying the 5×5 image block and masks, as illustrated in FIG. 3A.

[154] However, the present invention is not limited to this, and the processor 120 may also decide on the directionality of the image block using only some pixels of the image block to improve the calculation speed. For example, processor 120 can decide the directionality of an image block by applying a 3x3 directionality decision mask to 3x3 pixels except for pixels near the edges of the 5x5 image block.

[155] Hereinafter, a case in which a 3×3 image block is input is described for the convenience of explanation. However, the present invention is not limited to this, and processor 120 may operate in the same manner with respect to image blocks having different sizes. In addition, the memory 110 may store masks corresponding to different image sizes, a gradient calculation operator to be described below, an index matrix, a plurality of vectors, and the like.

[156] Processor 120 may compute a horizontal gradient and a vertical gradient for a 3×3 image block using the Sobel operator, as illustrated in FIG. 3C. Processor 120 may obtain a 3×3 horizontal gradient and a 3×3 vertical gradient through the Sobel operator and 3×3 image block multiplication, as illustrated in FIG. 3D.

[157] Processor 120 may compute a c×1 gradient vector from a 3×3 horizontal gradient and a 3×3 vertical gradient, as illustrated in FIG. 3E.

[158] Although the case in which the gradient vector is calculated after the transposition of the image block is performed is described above in this document, the present invention is not limited to this. For example, processor 120 may only decide on the directionality of the image block, and calculate the horizontal gradient and vertical gradient of the image block. In addition, processor 120 may reorder the gradient vector calculation to reflect the transposition from a horizontal gradient and a vertical gradient when the image block is to be transposed.

[159] FIG. 4A and 4B are diagrams illustrating an exemplary size reduction of a gradient vector feature according to an exemplary embodiment of the present invention.

[160] Processor 120 may reduce the size of the gradient vector feature c×1 with the eigenvector c'×c, as illustrated in FIG. 4A. For example, processor 120 may obtain a c' × 1 gradient vector whose feature size is reduced by multiplying between the c' × c eigenvector and the c × 1 gradient vector.

[161] FIG. 4B illustrates an example of a Gradient vector' whose feature size is reduced.

[162] The eigenvector may be generated by the external server principal component analysis method, and the electronic device 100 may receive the eigenvector from the external server and store the eigenvector in memory 110. For example, the external server may generate a gradient vector for each of the first 10,000 discrete image blocks. The method for creating the gradient vector is the same as the method described above. Here, the external server may decide on the directionality of each of the first discrete image blocks, transpose the first discrete image blocks based on the estimated directivity, and calculate gradient vectors for the transposed first discrete image blocks. Then, the external server may calculate the eigenvector by applying the principal component analysis method to each of the plurality of computed discrete gradient vectors.

[163] However, the present invention is not limited to this, and the processor 120 may also directly create an eigenvector through a principal component analysis method. In this case, the storage device 110 may store a plurality of first discrete image blocks.

[164] Hereinafter, the case of using a gradient vector whose feature size is reduced is described for convenience of explanation. However, reducing the feature size as described above is an optional operation, and subsequent operations can thus be performed without reducing the size of the gradient vector feature.

[165] FIG. 5A and 5B are diagrams illustrating an exemplary filter search with an index matrix according to an exemplary embodiment of the present invention.

[166] The memory 110 may store an index matrix a × c', as illustrated in FIG. 5A. The index matrix may include a 1 × c' index vectors. Here, the index vectors represent the characteristics of an image block and may also be referred to as classes or groups. Fig. 5A illustrates an index matrix including index vectors indicating characteristics for each of the classes or groups into which the characteristics of an image block are divided.

[167] The processor 120 may perform the multiplication between the index matrix a×c' and the gradient vector c'×1 whose feature size is reduced, and calculate the a×1 match vector, as illustrated in FIG. 5b. Each line of the match vector indicates the match of the image block to each class or group.

[168] The higher the number in the match vector, the higher the match. For example, in the case where 0.9 is the largest number in FIG. 5B, processor 120 may determine that the image block has class or group characteristics corresponding to 0.9.

[169] The index matrix may be created by an external server, and the electronic device 100 may receive the index matrix from the external server and store the index matrix in memory 110. The external server may compute a index vectors, which may represent a plurality of discrete gradient vectors computed from set of first discrete image blocks using the K-SVD or K-means algorithm, which is one of the unsupervised learning methods. The external server may create an index matrix including a index vectors.

[170] However, the present invention is not limited to this, and processor 120 may also calculate a index vectors, which may represent a plurality of discrete gradient vectors computed from a plurality of first discrete image blocks.

[171] In addition, although the case in which the characteristics of the image block are divided into a classes or groups is described above in this document, the characteristics of the image block are not limited to this, but may also be divided into a different number of classes or groups. However, hereinafter, a case in which the characteristics of an image block are divided into a classes or groups is described for the convenience of explanation. In addition, index vectors, classes, and groups are used together above in this document, but the case in which classes are used as a concept indicating characteristics of an image block and index vectors indicate each class is described later in this document.

[172] Meanwhile, the numbers of classes or groups can be changed depending on the kind of filters. For example, in the case of a resolution-enhancing filter, the characteristics of the image block may be divided into a classes or groups, but in the case of a sharpening filter, the characteristics of the image block may be divided into classes or filters whose number is different from a.

[173] FIG. 6A, 6B, and 6C are diagrams illustrating an exemplary filter search computational reduction method according to an exemplary embodiment of the present invention.

[174] FIG. 6A is a diagram illustrating that a plurality of classes are grouped into four groups, and classes 620 representing each group can be assigned in each group 610, 611, 612, and 613. FIG. 6B illustrates a first Index Matrix 1 including a plurality of index vectors corresponding to classes 620 representing each group. Fig. 6C illustrates a second Index Matrix 2 including a plurality of index vectors corresponding to classes included in one of the four groups. That is, the number of second index matrices is the same as the number of groups.

[175] The first index matrix and the second index matrix may be created by the external server, and the electronic device 100 may receive the first index matrix and the second index matrix from the external server and store the first index matrix and the second index matrix in the storage device 110. The external server may execute grouping by a index vectors by k-means clustering, or the like, and determining index vectors representing each group. However, the present invention is not limited to this, and the external server may also calculate representing index vectors representing each group. In this case, the representing index vectors may not match all of the index vectors in the corresponding group.

[176] However, the present invention is not limited to this, and the processor 120 may also calculate the first index matrix and the second index matrix in the manner described above, and store the first index matrix and the second index matrix in the memory 110.

[177] The processor 120 may perform the multiplication between the first 4 × c' index matrix and the c' × 1 gradient vector whose feature size is reduced, and compute matches to each group. Processor 120 may derive one group based on the highest match of the matches for each group. The processor 120 may perform multiplication between the second index matrix b × c' corresponding to the resulting group and the gradient vector c' × 1 whose feature size is reduced and compute matches to each of the classes included in the resulting group.

[178] In this case, the calculation of correspondences to classes except for the three representative classes among b × 3 classes included in other groups except for the obtained group is not performed, and the accuracy may be reduced compared to FIG. 5A and 5B, but the computational speed can be improved.

[179] If the number of groups is reduced, the accuracy can be further improved. The calculation speed can be further improved by increasing the number of groups.

[180] FIG. 7A and 7B are diagrams illustrating an exemplary filter search computational reduction method according to another exemplary embodiment of the present invention. For example, the method in FIG. 7A and 7B is a method for selecting a class having a higher correspondence by additional calculation after the method in FIG. 6A, 6B and 6C is fully completed.

[181] The processor 120 may obtain the class A 710 of the third group as the class whose correspondence to the image block is the highest as the result of the calculations of the correspondences to the second index matrix, as illustrated in FIG. 7A.

[182] Memory 110 may store a third Index Matrix 3 for class A 710, as illustrated in FIG. 7B. The third index matrix may include a plurality of index vectors corresponding to classes A' 720-1, 720-2, and 720-3 included in groups other than class A 710 and having similar characteristics to class A 710. In addition, , memory 110 may store three index matrices for each of the other classes that are not class A 710. For example, the number of third index matrices may be the same as the number of classes. In addition, the sizes of the plurality of third index matrices may differ from each other.

[183] The plurality of third index matrices may be created by an external server or created by processor 120.

[184] The processor 120 may perform the multiplication between the 3×c' third index matrix and the c'×1 gradient vector c'×1 whose feature size is reduced, and compute correspondences to the A' classes 720-1, 720-2, and 720-3 having characteristics similar to class A 710.

[185] A process for calculating matches to a third index matrix can be added to the method of FIG. 7A and 7B compared to the method in FIG. 6A, 6B and 6C, but the accuracy could be improved.

[186] FIG. 8A and 8B are diagrams illustrating an exemplary method for applying a filter to an image block according to an exemplary embodiment of the present invention. In FIG. 8A, a 5×5 image block is illustrated for the convenience of explanation.

[187] Processor 120 may apply at least one of a plurality of filters to the image block based on the matches computed by the methods that are illustrated in FIG. 5A, 5B, 6A, 6B, 6C, 7A and 7B. For example, when all class matches are computed, as illustrated in FIG. 5B, processor 120 may obtain a first filter corresponding to class 0.9, a second filter corresponding to class 0.2, and a third filter corresponding to class 0.1 in a high-match sequence. Each of the plurality of filters may correspond to each of the plurality of classes. For example, in a case in which a classes are created depending on the characteristics of the image block, one filter corresponding to each of the a classes is created, and a total of a filters can be created. A detailed description for the filter creation method is described below.

[188] The processor 120 may convert a plurality of filters into a single filter through the following Equation 2:

[189] [Equation 2]

[190]

[191] Here, H(1), H(2), and H(3) indicate the first filter, second filter, and third filter, respectively, and C(1), C(2), and C(3) indicate the image block correspondences the first filter, the second filter, and the third filter, respectively.

[192] However, the present invention is not limited to this, and the processor 120 may also convert a plurality of filters into a single filter without considering the matches of each filter. In this case, processor 120 may convert multiple filters into a single filter by replacing C(1), C(2), and C(3) with 1 in Equation 2.

[193] Processor 120 may obtain final image block 820 for target pixel 810 by applying a filter to the 5×5 image block as illustrated in FIG. 8A. The right side in Fig. 8A illustrates a final image block in which a total of six target pixels are processed in an image, and is a view in which an upscaling filter is applied. That is, one target pixel can be scaled up to a total of nine pixels.

[194] Processor 120 may vectorize the image block and apply a filter to the image block. For example, processor 120 may vectorize a 5×5 image block and apply a filter to a 5×5 image block. That is, processor 120 may obtain a final 3×3 image block as illustrated on the right side of FIG. 8A by performing the multiplication between the filter and the vector. The method for vectorizing an image block and obtaining a final image block from the vector is the same or similar to the method described in FIG. 3E, and the duplicate description is thus omitted.

[195] Although the case in which the resolution enhancement filter is used is described above in this document, the present invention is not limited thereto. For example, a sharpening filter may be used. In this case, the resolution is kept as is, and the target pixel can be filtered for sharpening based on the plurality of surrounding pixels. The sharpening filter may also be created by an artificial intelligence algorithm, and the discrete image used in the training process may be different from the discrete image in the resolution upscaling filter. A detailed description for this is provided below.

[196] Meanwhile, the processor 120 may store information about whether or not the image block is transposed depending on the directionality of the original image block in the memory 110. When the final image block is obtained from the transposed image block, the processor 120 may again transpose the final image block based on the information stored in the storage device 110, as illustrated in FIG. 8B.

[197] Meanwhile, the processor 120 can obtain an additional image block by applying a non-learning filter to the image block and update the final image block based on the best match of the set of calculated matches and the additional image block.

[198] In detail, processor 120 may update the final image block as represented by the following Equation 3:

[199] [Equation 3]

[200]

[201]

[202] Here, Y1' indicates the updated end image block, Y1 indicates the end image block, Y2 indicates an additional image block, w indicates a weighting factor, c indicates the highest match of the set of calculated matches, and k indicates a constant.

[203] A filter without training may show a better performance compared to a filter performance based on training, depending on the kind of filter and the kind of image characteristics. For example, a learning-based resolution enhancement filter shows excellent performance relative to edge components whose image features are clear, but exhibits a noise component relative to a flat area whose image feature is unclear, or exhibits poor performance on the detail surface. Therefore, in the case in which the final image block by the learning-based resolution enhancement filter is updated with an additional image block by the non-training-based resolution enhancement filter as represented by the following Equation 3, excellent filtering performance can be provided with respect to a flat area whose image feature is unclear.

[204] In addition, since the correspondence between the image block and the training-based resolution enhancement filter is taken into account as a weighting factor in the update process, the performance can be further improved. For example, in a case in which the image block is a flat area whose image distinguishing feature is not clear, the correspondences of the image block to a plurality of classes may be low. A discrete image block including a flat area whose image feature is not clear may be excluded from training in the training process.

[205] Meanwhile, the processor 120 can also perform resolution enhancement relative to the unlearned magnification by connecting the untrained resolution enhancement filter to the training-based resolution enhancement filter in series.

[206] For example, the training-based resolution enhancement filter is activated depending on the trained resolution enhancement only, and may not be activated relative to the untrained resolution enhancement. However, processor 120 may upsample with the training-based upsampling filter and then resample with the untrained upsampling filter to produce a result depending on the desired upsampling.

[207] Meanwhile, although the case in which three filters are weighted and summed in Equation 2 has been described, the present invention is not limited thereto. For example, processor 120 may use only one filter, or use multiple filters and apply different weights to the multiple filters.

[208] In addition, the processor 120 may provide a leaf image block with a single filter and a plurality of leaf image blocks with a plurality of filters to which different weights are applied, and may modify Equation 2 depending on the user's choice. For example, the processor 120 may provide an end image block with one filter and a plurality of end image blocks with a plurality of filters depending on Equation 2, and in the case where the user selects an end image block with one filter, the processor 120 may use one filter. in the subsequent process of applying the filter.

[209] FIG. 9A and 9B are diagrams illustrating an exemplary training method for a filter according to an exemplary embodiment of the present invention. In FIG. 9A and 9B, a resolution enhancement filter training method and a case in which training is performed in an external server are described for convenience of explanation.

[210] An external server may store a high resolution image as illustrated in FIG. 9A. The external server may read the plurality of second discrete picture blocks from the high-resolution image and obtain the plurality of first discrete pictures by reducing the resolution of each of the plurality of second discrete picture blocks, as illustrated in FIG. 9b. The external server may store a plurality of second discrete image blocks having a high resolution and a plurality of first discrete image blocks corresponding to a plurality of second discrete image blocks having a low resolution. Here, high resolution and low resolution should indicate relative states, and the low resolution image block may refer to, for example, an image block having a resolution lower than that of the high resolution image block.

[211] The external server may obtain a plurality of discrete gradient vectors from a plurality of first discrete images. Here, the external server may perform the transposition considering the directionality of the plurality of first discrete images, but this is an optional operation.

[212] The external server may derive an eigenvector from the set of discrete gradient vectors and reduce the size of the features of the set of discrete gradient vectors. However, the external server may also perform training without feature size reduction.

[213] The external server may divide the plurality of discrete gradient vectors whose feature sizes are reduced into a predetermined number of groups for each of the characteristics of the image block, and obtain index vectors representing each group. Here, the index vectors representing each group may or may not be the same as one of the plurality of discrete gradient vectors corresponding to the corresponding group. The external server may receive an index matrix including a plurality of index vectors.

[214] For convenience of explanation, a case is described in which an external server divides the characteristics of an image block into two groups having a vertical orientation and a horizontal orientation. In this case, the external server may obtain two index vectors representing each group and obtain an index matrix including the two index vectors.

[215] In addition, the external server may classify the first discrete image blocks into one of two groups. For example, the external server may compute the first discrete gradient vectors for the first discrete image blocks, compute the correspondences of the first discrete image blocks to two groups by multiplying between the index matrix and the first discrete gradient vectors, and classify the first discrete image blocks into a group having a higher match. The external server may perform the process as described above on all of the first discrete image blocks. As a result, for example, 354,000 out of 1,000,000 first discrete image blocks can be classified into a group having a horizontal directionality, and 646,000 out of 1,000,000 first discrete image blocks can be classified into a group having a vertical directionality.

[216] The external server may create a first filter by learning the relationship between the 354,000 first discrete image blocks having horizontal directionality and the 354,000 second discrete image blocks corresponding to the 354,000 first discrete image blocks through an artificial intelligence algorithm. In addition, the external server can create a second filter by learning the relationship between the 646,000 first discrete image blocks having a vertical directionality and the 646,000 second discrete image blocks corresponding to the first 646,000 discrete image blocks through the artificial intelligence algorithm.

[217] An index vector may be considered as an index indicating the characteristics of one image block, and a filter corresponding to the index vector may be obtained by examining the first discrete image blocks and the second discrete image blocks corresponding to the first discrete image blocks based on the characteristics of the image block, corresponding to the corresponding index vector.

[218] The external server may perform training by the L2 target error minimization and/or reduction method and the L1 target error minimization and/or reduction method. For example, the external server may compute an image enlargement filter that minimizes and/or reduces the L2 target error through the following Equation 4:

[219] [Equation 4]

[220]

[221] Here, K means a class, X indicates an image value for a high-resolution discrete image block, and Y indicates an image value for a low-resolution discrete image block.

[222] However, this is only an example, and various teaching methods can be used.

[223] In addition, although the case of learning a high resolution image block and a low resolution image block is described in FIG. 9A and 9B, the present invention is not limited thereto. For example, the external server may also obtain a sharpening filter or a blur filter by learning a blurry image block and a sharp image block. In this case, the resolution of the image block may not be changed. Here, blurry image block means an image block having a sharpness relatively lower than that of a sharp image block.

[224] In addition, the electronic device 100 can directly perform training instead of an external server. In this case, the memory 110 may store the first discrete image blocks and the second discrete image blocks, and the processor 120 may obtain an index matrix and a plurality of filters by performing learning based on the stored information. In addition, when an image block is input, processor 120 may create a final image block by applying at least one filter to the input image block.

[225] Meanwhile, it is described above that transposing the image block depending on the directivity estimation result, reducing the size of the gradient vector feature, and applying the filter without training are optional operations. For example, the index matrix and filters can be changed depending on whether optional operations are performed or not.

[226] For example, when the transposition of the image block depending on the result of the directionality estimation is performed, an a × c index matrix can be created (in the case in which the feature size reduction is not performed). On the other hand, when the transposition of the image block depending on the directivity estimation result is not performed, a 2 × a × c index matrix may be created (in the case in which feature size reduction is not performed). This is because in a case in which image characteristics are divided by a, when no transposition is performed, the two cases of horizontal directionality and vertical directionality are separated, and image characteristics are thus duplicated.

[227] When reducing the size of the feature of the gradient vectors is performed, an index matrix a × c' (in the case in which the transposition is performed) can be created. On the other hand, when the feature size reduction of the gradient vectors is not performed, an a × c index matrix (in the case in which the transposition is performed) can be created.

[228] FIG. 10 is a flowchart illustrating an exemplary image transmission operation according to an exemplary embodiment of the present invention.

[229] The external server 200 may, for example and without limitation, be a device that provides an image to the electronic device 100. In addition, the external server 200 may, for example and without limitation, be a desktop PC, laptop PC, tablet, smartphone, or the like. as well as a server, and can be any device that can perform AI learning and encoding.

[230] The external server 200 may obtain a low-resolution image from a high-resolution image (S1010). For example, external server 200 may receive 4K video from 8K video. The way to reduce video resolution can be any way.

[231] In addition, the external server 200 can generate filter information by examining a high-resolution image and a low-resolution image (S1020). For example, the external server 200 may divide the corresponding high-resolution image frames into image blocks having a predetermined size, and may also divide the low-resolution image in the same manner. In this case, the number of image blocks created from the high resolution image may be the same as the number of image blocks created from the low resolution image. In addition, each of the image blocks created from the high resolution image may correspond to one of the image blocks created from the low resolution image, and may only differ in resolution from one of the image blocks created from the low resolution image. Hereinafter, image blocks created from a high-resolution image and image blocks created from a low-resolution image are described as high-resolution image blocks and low-resolution image blocks, respectively.

[232] The external server 200 may create a plurality of filters by performing training to obtain corresponding high-resolution image blocks from low-resolution image blocks. Here, the method for creating a plurality of filters is the same as the method described in FIG. 1A-9B, and its description is thus not repeated here.

[233] In addition, the external server 200 may encode and compress the low resolution image (S1030). The encoding method is not particularly limited.

[234] The external server 200 may transmit the low-resolution encoded image and filter information to the electronic device 100 (S1040). Here, the filter information may include an index matrix and a plurality of filters. In addition, according to an exemplary embodiment, the filter information may further include an eigenvector. In addition, the filter information may include a method for dividing the corresponding low-resolution image frame into image blocks having a predetermined size.

[235] The electronic device 100 may decode the low resolution encoded image to recreate the low image image (S1050). The decoding method is not particularly limited as long as it matches the encoding method of the external server 200.

[236] The electronic device 100 can recreate a high resolution image by applying the filter information to the low resolution image (S1060). For example, the electronic device 100 may divide a low-resolution image into a plurality of image blocks based on a method for dividing corresponding low-resolution image frames into image blocks having a predetermined size included in the filter information. In addition, the electronic device 100 can obtain high-resolution image blocks from the corresponding image blocks based on an index matrix and a plurality of filters. The method for obtaining high-resolution image blocks is the same as that described in FIG. 1A-9B, and its description is thus not repeated here. The electronic device 100 may acquire a high resolution image from a plurality of high resolution image blocks.

[237] It is difficult to transmit an 8K image only by the current coding technology, but an 8K image can be transmitted by the method described above. For example, the external server 200 may encode a 4K image and transmit the encoded 4K image to the electronic device 100. In addition, the electronic device 100 may decode the received information to recreate a 4K image, and may create an 8K image from the 4K image using the training data obtained by the algorithm artificial intelligence to get 8K image from 4K image.

[238] The filter information transmitted from the external server 200 to the electronic device 100 is only a tool for obtaining an 8K image from a 4K image, and can be considered different from the image information. For example, the filter information capacity may be much smaller than the capacity when an 8K image is directly transmitted, and in the case of using the filter information as described above, 8K image streaming may be possible.

[239] FIG. 11 is a flowchart illustrating an exemplary method for controlling an electronic device according to an exemplary embodiment of the present invention.

[240] An image block including a target pixel and a plurality of surrounding pixels is classified into one of the plurality of image templates based on the relationship between pixels in the image block (S1110). In addition, the final image block in which the target pixel is processed in the image is obtained by applying at least one filter corresponding to the classified image template among the plurality of filters each corresponding to the plurality of image templates to the image block (S1120). The plurality of filters can be obtained by learning the relationship between the plurality of first discrete image blocks and the plurality of second discrete image blocks corresponding to the plurality of first discrete image blocks by an artificial intelligence algorithm based on each of the plurality of image templates.

[241] Classifying (S1110) may include calculating a gradient vector from the image block, calculating correspondences of the image block to each of the plurality of index vectors based on an index matrix including the plurality of index vectors and the gradient vector, and classifying the image block into one of the image pattern sets based on the calculated matches, and the index vector set may be obtained based on the set of discrete gradient vectors calculated from the set of first discrete image blocks and may correspond to the set of image patterns, respectively.

[242] In addition, when calculating the gradient vector, the gradient vector can be calculated from the image block when the intensity of the horizontal directivity of the image block is greater than the intensity of the vertical directivity of the image block, and the image block can be transposed, and the gradient vector can be calculated from the transposed image block when the intensity of the horizontal directivity of the image block is less than the intensity of the vertical directivity of the image block.

[243] Meanwhile, the method for controlling the electronic device according to the exemplary embodiment of the present invention may further include reducing the size of the gradient vector with an eigenvector, wherein in computing the correspondences of the image block to each of the plurality of index vectors, based on the gradient vector, the size which is reduced, and the eigenvector is created by applying principal component analysis from a set of discrete gradient vectors.

[244] Further, calculating the matches may include calculating a first match of the image block to each of the plurality of index vector groups based on the first index matrix and the gradient vector, deriving one of the plurality of index vector groups based on the first match, deriving a second index matrix corresponding to obtained index vector group, among the plurality of second index matrices, and calculating a second image block correspondence to each of the plurality of index vectors included in the obtained index vector group, based on the obtained second index matrix and the gradient vector, and upon obtaining (S1120) the end image block, the end block image may be obtained by applying at least one of the plurality of filters to the image block based on the second match, the plurality of index vector groups may be obtained by dividing the plurality of index vectors by a predetermined number group layer, the first index matrix may include index vectors representing each of the plurality of index vector groups, and each of the plurality of second index matrices may include a plurality of index vectors corresponding to each of the plurality of index vector groups.

[245] Calculating matches may include deriving one of the plurality of index vectors included in the obtained index vector group based on the second match, deriving at least one additional index vector included in others of the plurality of index vector groups and corresponding to the obtained to the index vector, based on the same information of the index vector, and calculating a third image block correspondence based on the obtained index vector, the obtained additional index vector, and the gradient vector, and when obtaining (S1120) the end image block, the end image block can be obtained by applying at least at least one of the plurality of filters to the image block based on the third match.

[246] Meanwhile, obtaining (S1120) a final image block may include obtaining at least two of the plurality of filters based on the plurality of calculated matches, calculating the final filter based on at least two filters and the correspondences corresponding to each of at least two filters, and obtaining a final image block by applying the computed final filter to the image block.

[247] In addition, the plurality of first discrete image blocks may be images in which the resolutions of the respective second discrete image blocks are reduced, respectively, and when obtaining (S1120) the target image block, the target image block in which the resolution of the target pixel is increased may be obtained by applying at least one of the plurality of filters to the image block.

[248] The method for controlling an electronic device according to an exemplary embodiment of the present invention may further include obtaining an additional image block in which the resolution of the target pixel is increased by applying a learning-less resolution upscaling technique to the image block, and updating the final image block based on the best match of a set of computed matches and an additional image block.

[249] Meanwhile, when calculating the gradient vector, horizontal gradients and vertical gradients for the target pixel and each of the plurality of surrounding pixels can be calculated, and the gradient vector can be calculated based on the horizontal gradients and vertical gradients.

[250] In addition, each of the plurality of filters can be obtained by obtaining a plurality of first subsample image blocks corresponding to one of the plurality of index vectors, among the plurality of first discrete image blocks, obtaining a plurality of second subsample image blocks corresponding to the plurality of first subsample image blocks, among the plurality of second sub-sample image blocks, and learning the relationship between the plurality of first sub-sample image blocks and the plurality of second sub-sample image blocks by an artificial intelligence algorithm.

[251] According to various exemplary embodiments of the present invention, as described above, an electronic device can perform real-time image processing and thus be used in various applications, and can improve image processing quality compared to image processing without training by performing processing images based learning.

[252] Meanwhile, the processor may apply the filter to each pixel of the image block. The processor may apply the filter to each subpixel of each pixel in the image block. The processor may only apply the filter to some of the subpixels. The processor may distinguish the chrominance component and the luma component of each pixel in the image block from each other and apply a filter to at least one of the chrominance component and the luminance component.

[253] Meanwhile, according to an exemplary embodiment of the present invention, the various exemplary embodiments described above can be implemented by software including instructions stored on a computer-readable storage medium (eg, a computer-readable storage medium). The machine may be a device that recalls a stored instruction from a storage medium and may operate depending on the recalled instruction, and may include an electronic device according to the disclosed exemplary embodiments. In the example in which the instruction is executed by the processor, the processor may directly perform the function corresponding to the instruction, or other components may perform the function corresponding to the instruction under the control of the processor. The command may include codes generated or executed by a compiler or interpreter. The computer-readable storage medium may be provided in the form of a durable storage medium. Here, the term 'durable' may refer to, for example, a storage medium that is tangible and does not distinguish whether data is stored almost permanently or temporarily on the storage medium.

[254] In addition, according to an exemplary embodiment of the present invention, the methods according to the various exemplary embodiments described above may be included and provided in a computer program product. The computer program product may be traded as a product between the seller and the buyer. The computer program product may be distributed in the form of a storage medium (eg, compact disc read only memory (CD-ROM)) that can be read by a machine, or online through an application store (eg, PlayStore™). In the case of online distribution, at least portions of the computer program product may be at least temporarily stored on a storage medium, such as the memory of a manufacturer's server, an application store server, or a relay server, or be temporarily created.

[255] In addition, according to an exemplary embodiment of the present invention, the various exemplary embodiments described above may be implemented in a computer or computer-readable storage medium using software, hardware, or any combination of software and hardware. In some cases, the exemplary embodiments described in the present invention may be implemented by the processor itself. According to software implementation, exemplary embodiments such as the procedures and functions described in the present invention may be implemented by separate software modules. Each of the software modules may perform one or more of the functions and operations described in the present invention.

[256] Meanwhile, computer instructions for performing the processing operations of the machines according to the various exemplary embodiment of the present invention described above can be stored in a non-volatile computer-readable medium. Computer instructions stored on a non-volatile computer-readable medium enable a particular machine to perform processing operations on the machine according to the various exemplary embodiments described above when they are executed by the processor of the particular machine. A durable computer-readable medium may refer to, for example, a medium that stores data almost permanently in itself and is machine-readable. Examples of non-volatile computer-readable media may include a compact disc (CD), a digital versatile disc (DVD), a hard drive, a Blu-ray disc, a universal serial bus (USB), a memory card, a read only memory (ROM), or the like. .P.

[257] In addition, each of the components (e.g., modules or programs) according to the various exemplary embodiments described above may include a single object or a plurality of objects, and some of the respective subcomponents described above may be omitted, or others subcomponents may be further included in various exemplary embodiments. Alternatively or additionally, some of the components (eg, modules or programs) may be combined into one entity and may perform the functions performed by the respective respective components before being combined in the same or similar manner. The operations performed by modules, programs, or other components according to various exemplary embodiments may be performed in a serial manner, in a parallel manner, in an iterative manner, or in a heuristic manner, at least some of the operations may be performed in a different order or omitted, or other operations may be added.

[258] Although various exemplary embodiments of the present invention have been illustrated and described above herein, the present invention is not limited to the above specific exemplary embodiments, but can be modified in various ways by those skilled in the art to which the present invention belongs without departing from the true spirit and full scope of the present invention as described in the accompanying claims. These modifications are also to be understood as falling within the scope and spirit of the present invention.

Claims (40)

1. An electronic device that performs image processing, comprising: a storage device configured to store a plurality of filters, each of the plurality of filters corresponding to a plurality of image templates; and a processor configured to classify an image block including a target pixel and a plurality of surrounding pixels into one of the plurality of image templates based on a relationship between pixels in the image block, and obtain a final image block in which the target pixel is processed in the image by applying at least at least one filter corresponding to the classified image template, among the plurality of filters to the image block, wherein the plurality of filters are obtained by learning, through the artificial intelligence algorithm, the relationship between the plurality of first discrete image blocks and the plurality of second discrete image blocks corresponding to the plurality of first discrete image blocks based on each of the plurality of image templates. 2. The electronic device of claim 1, wherein the processor is configured to determine a gradient vector from the image block, determine image block mappings to each of the plurality of index vectors based on an index matrix stored in the memory including the plurality of index vectors, and the gradient vector, and classify the image block into one of a plurality of image templates based on certain matches, and a plurality of index vectors are obtained based on a plurality of discrete gradient vectors determined from a plurality of first discrete image blocks, wherein the plurality of index vectors correspond to the plurality of image templates. 3. The electronic device of claim 2, wherein the processor is configured to determine a gradient vector from the image block when the intensity of the horizontal directivity of the image block is greater than the intensity of the vertical directivity of the image block, and transpose the image block and determine the gradient vector from the transposed image block when the intensity the horizontal directivity of the image block is less than the intensity of the vertical directivity of the image block. 4. The electronic device of claim 2, wherein the processor is configured to reduce the size of the gradient vector with an eigenvector stored in the memory and determine image block mappings to each of the plurality of index vectors based on the gradient vector whose size is reduced, and the eigenvector is created by applying principal component analysis from a set of discrete gradient vectors. 5. The electronic device of claim 2, wherein the processor is configured to determine a first image block correspondence to each of the plurality of index vector groups based on the first index matrix stored in the memory and the gradient vector, obtain one of the plurality of index vector groups based on of the first match, determine the second match of the image block to each of the plurality of index vectors included in the obtained index vector group, based on the second index matrix corresponding to the obtained index vector group, among the plurality of second index matrices stored in the memory and the gradient vector, and obtain final image block, applying at least one of the plurality of filters to the image block based on the second match, wherein: the set of index vector groups are obtained by dividing the set of index vectors by a predetermined number of groups, the first index matrix includes index vectors representing each of the plurality of index vector groups, and each of the plurality of second index matrices includes a plurality of index vectors corresponding to each of the plurality of index vector groups. 6. The electronic device according to claim 5, wherein the processor is configured to obtain one of the plurality of index vectors included in the obtained index vector group, based on the second match, to obtain at least one additional index vector included in the plurality of index vector groups, different from the received one of the plurality of index vectors and corresponding to the received index vector, based on the similar information of the index vector stored in the storage device, determine the third image block matching based on the received index vector, the obtained additional index vector and the gradient vector, and obtain the final image block, applying at least one of the plurality of filters to the image block based on the third match. 7. The electronic device of claim. 2, wherein the processor is configured to obtain at least two of the plurality of filters based on the plurality of defined matches, determine the final filter based on at least two filters and the matches corresponding to each, at least two filters, and obtain a final image block by applying the determined final filter to the image block. 8. The electronic device of claim 2, wherein the plurality of first discrete image blocks comprise images in which the resolutions of the respective second discrete image blocks are reduced, and the processor is configured to obtain a final image block in which the resolution of the target pixel is increased by applying at least one of the plurality of filters to the image block. 9. The electronic device of claim 8, wherein the processor is configured to obtain an additional image block in which the resolution of the target pixel is increased by applying a learningless resolution upscaling technique to the image block, and to update the final image block based on the best match among the set of determined matches. and an additional image block. 10. The electronic device of claim 2, wherein the electronic device is configured to obtain each of the plurality of filters, obtaining a plurality of first subsample image blocks corresponding to one of the plurality of index vectors among the plurality of first discrete image blocks, obtaining a plurality of second subsample image blocks corresponding to a plurality of first sub-sampled image blocks, among the plurality of second sub-sampled image blocks, and learning, by means of an artificial intelligence algorithm, a relationship between the plurality of first sub-sampled image blocks and the plurality of second sub-sampled image blocks. 11. A method for controlling an electronic device that performs image processing, comprising the steps of: classifying the image block including the target pixel and the plurality of surrounding pixels into one of the plurality of image templates based on a relationship between pixels in the image block; and obtaining a final image block in which the target pixel is processed in the image by applying at least one filter corresponding to the classified image template, among a plurality of filters, each corresponding to a plurality of image templates, to the image block, wherein the plurality of filters are obtained by learning, through the artificial intelligence algorithm, the relationship between the plurality of first discrete image blocks and the plurality of second discrete image blocks corresponding to the plurality of first discrete image blocks based on each of the plurality of image templates. 12. The method of claim 11, wherein the classification comprises the steps of: determining a gradient vector from the image block; determining the correspondence of the image block to each of the plurality of index vectors based on an index matrix including the plurality of index vectors and the gradient vector; and classifying the image block into one of the plurality of image templates based on certain matches, and a plurality of index vectors are obtained based on a plurality of discrete gradient vectors calculated from a plurality of first discrete image blocks, wherein the plurality of index vectors correspond to the plurality of image templates. 13. The method of claim 12, wherein, when determining the gradient vector, the gradient vector is calculated from the image block when the horizontal directivity intensity of the image block is greater than the vertical directivity intensity of the image block, and the image block is transposed, and the gradient vector is calculated from the transposed image block, when the intensity of the horizontal directivity of the image block is less than the intensity of the vertical directivity of the image block. 14. The method of claim 12, further comprising reducing the size of the gradient vector with an eigenvector, wherein, when determining the correspondences, the correspondences of the image block to each of the plurality of index vectors are calculated based on the gradient vector, the size of which is reduced, and the eigenvector is created by applying principal component analysis from a set of discrete gradient vectors. 15. The method of claim 12, wherein determining the matches includes the steps of: determining a first correspondence of the image block to each of the plurality of index vector groups based on the first index matrix and the gradient vector; obtaining one of the plurality of index vector groups based on the first match; determining a second correspondence of the image block to each of the plurality of index vectors included in the obtained index vector group, based on the second index matrix corresponding to the obtained index vector group, among the plurality of second index matrices and the gradient vector, and wherein, upon obtaining the final image block, the final image block is obtained by applying at least one of the plurality of filters to the image block based on the second match, wherein: the set of index vector groups are obtained by dividing the set of index vectors by a predetermined number of groups, the first index matrix includes index vectors representing each of the plurality of index vector groups, and each of the plurality of second index matrices includes a plurality of index vectors corresponding to each of the plurality of index vector groups.

Images