KR20190044761A - Apparatus Processing Image and Method thereof - Google Patents

Abstract

The present invention relates to an image processing apparatus, which extracts a feature point of an input image, converts pixel values of the image into color space component values of a colorimetric system different from a colorimetric system representing the pixel values to automatically analyze a composition of the image. The image processing apparatus of the present invention comprises: a feature point extracting unit for extracting feature points in consideration of pixel values for each pixel from an input image; a feature information calculation unit for converting the pixel values of the image into color space component values in accordance with a chromaticity system different from a colorimetric system representing the pixel values, and calculating feature information on a composition of the input image on the basis of the converted color space component values and the extracted feature points; and a composition analysis unit for analyzing the composition of the input image using the calculated feature information.

Description

[0001] Apparatus Processing Image and Method [0002]

The present invention relates to an apparatus and method for processing an image. More particularly, the present invention relates to an image processing apparatus and method for analyzing a composition of an input image.

As the technology of the digital camera has recently developed, anyone can easily take a picture by using a portable device including a camera function. Unlike the past where a separate camera was required for photographing, a portable device with a camera function The population that has become universalized and taking pictures is growing rapidly.

However, it is difficult to master photographic techniques, so a variety of techniques are being developed for non-specialists, such as ordinary people, to obtain high-quality photographs without the help of specialists. The composition of the picture as a factor for high quality photography is very important, but the background of the picture composition of the general public is often insufficient compared to its importance. Especially, there is not much research on the composition according to the golden ratio among the composition of the photograph.

It is scientifically proven that golden rain can be easily found in nature, and that most people feel the harmony when they see structures or natural objects with a golden ratio structure. Also, in photographing, a photograph having a composition of golden ratio makes the most harmonious appearance.

However, it is difficult for the general user to frame the golden ratio, and it is not easy for the general user to capture the composition according to the golden ratio in the photographs containing various subjects.

Therefore, it is required that the general user develops a technique for easily taking good quality photographs without the help of experts.

Korean Patent Laid-Open No. 10-2015-0090456 (published on Aug. 06, 2015)

Disclosure of Invention Technical Problem [8] The present invention has been made to solve the above problems, and it is an object of the present invention to provide an image processing apparatus for automatically analyzing a composition of an input image. In particular, the present invention discloses an image processing apparatus that extracts feature points of an input image and converts pixel values of the image into color space component values of a colorimetric system different from a colorimetric system that represents the pixel values, thereby automatically analyzing a composition of an image. Also disclosed is an image processing method for analyzing a composition of an image.

According to an aspect of the present invention, there is provided an image processing apparatus including a feature point extracting unit for extracting feature points by considering pixel values for each pixel from an input image; The pixel values of the image are converted into color space component values according to a colorimetric system different from that of the colorimetric system that represents the pixel values and the color space component values of the color space component values corresponding to the composition of the input image A feature information calculation unit for calculating feature information; And a composition analyzer for analyzing a composition of the input image using the calculated feature information; .

The image processing apparatus according to the present invention includes: an image division unit which repeatedly divides an input image from which the minutiae points are extracted into a predetermined divided region to generate divided images; And the feature information calculating unit may calculate the feature information based on the minutiae for each of the divided regions.

In the present invention, the feature point extracting unit may designate an arbitrary pixel in the input image, and may extract the designated pixel as a feature point in consideration of a pixel value change amount of pixels adjacent to the designated pixel.

The feature information calculating unit may include a spiral generating unit formed within the divided images, the spiral generating unit generating spirals converging on at least one divided image, the divided images passing through at least one diagonal side edge of the divided images; And the feature information can be calculated using the coordinates of the pixel where the generated helix converges.

 In the present invention, the feature information calculation unit may include a first feature value calculation unit for calculating a first feature value in consideration of the degree of feature points of the divided regions in all feature points extracted from the input image; And the feature information may be calculated using the calculated first characteristic value.

In the present invention, the feature information calculation unit may include a second feature value calculation unit for calculating a second feature value in consideration of the mean coordinates of the feature points for each of the divided regions, the coordinates of the pixels where the helix converges, and the corner coordinates of the input image; And the feature information may be calculated using the calculated second characteristic value.

Wherein the feature point calculating unit calculates a third feature value by taking into consideration the average coordinates of pixels having component values of a predetermined threshold value or more among the color space component values and the coordinates of the pixel where the helix converges; And the feature information may be calculated using the calculated third characteristic value.

Wherein the step of repeatedly dividing the image into the predetermined divided region includes calculating a width or a length of the input image and initially dividing the calculated longer length according to a golden ratio, May be re-divided in the same manner as the initial division to determine the divided region, and may be repeatedly divided into the determined divided region.

The generated helix may start at at least one corner of the input image and be arranged to have different helix directions depending on the position of the edge and the coordinates of the pixel where the helix converges.

The composition analyzing unit may include a directional prediction model generating unit that learns the calculated feature information using a machine learning technique to generate a directional prediction model for the direction of the composition; And the composition of the image can be analyzed using the generated directional prediction model.

The composition analyzing unit may include a score prediction model generation unit for learning the calculated feature information using the machine learning technique to generate a score prediction model for the score of the composition; And the composition of the image can be analyzed using the generated score prediction model.

According to another aspect of the present invention, there is provided an image processing method comprising the steps of: extracting feature points by taking pixel values of an input image into consideration; The pixel values of the image are converted into color space component values according to a colorimetric system different from that of the colorimetric system that represents the pixel values and the color space component values of the color space component values corresponding to the composition of the input image Calculating feature information; Analyzing a composition of the input image using the calculated feature information; .

The image processing method according to the present invention may further include the steps of: repeatedly dividing the input image from the extracted feature points into a predetermined divided region; And the feature information may be calculated based on the minutiae of the divided regions.

In the present invention, the extracting step may designate an arbitrary pixel in the input image, and may extract the designated pixel as a feature point in consideration of a pixel value change amount of pixels adjacent to the designated pixel.

In the present invention, the calculating step includes the steps of: generating a spiral formed inside the divided images, converging on at least one divided image, passing through at least one diagonal corner of the divided images; And the feature information can be calculated using the coordinates of the pixel where the generated helix converges.

Calculating the first characteristic value in consideration of the degree of the feature points of the divided regions in all the feature points extracted from the input image; And the feature information may be calculated using the calculated first characteristic value.

Calculating the second characteristic value in consideration of the average coordinates of the minutiae of the divided regions, the coordinates of the pixel where the spiral converges, and the corner coordinates of the input image; And the feature information may be calculated using the calculated second characteristic value.

Calculating the third characteristic value taking into consideration the average coordinates of the pixels having component values of a predetermined threshold value or more among the color space component values and the coordinates of the pixel where the spiral converges; And the feature information may be calculated using the calculated third characteristic value.

Wherein the step of repeatedly dividing the image into the predetermined divided region includes calculating a width or a length of the input image and initially dividing the calculated longer length according to a golden ratio, May be re-divided in the same manner as the initial division to determine the divided region, and may be repeatedly divided into the determined divided region.

Wherein the analyzing step comprises: learning the calculated feature information using a machine learning technique to generate a directional prediction model for the direction of the composition; And the composition of the image can be analyzed using the generated directional prediction model.

According to an embodiment of the present invention, the analyzing step may include: learning the calculated feature information using a machine learning technique to generate a score prediction model for a score of the composition; And the composition of the image can be analyzed using the generated score prediction model.

The present invention also discloses a computer program stored in a computer-readable recording medium for executing the above-described image processing method in a computer.

According to the present invention, it is possible to automatically analyze the composition of an input image.

In particular, it has the advantage of automatically analyzing the direction and score of the golden ratio composition.

1 is a block diagram of an image processing apparatus according to an embodiment of the present invention.
2 is an exemplary view of an input image from which feature points are extracted according to an embodiment of the present invention.
3 is an illustration of a spiral generated on a segmented input image according to one embodiment.
4 is an illustration of four different helical directions according to the position of the edge of the input image at which the helix starts and the coordinates of the pixel where the helix converges.
5 is an enlarged block diagram of the feature information calculation unit in the embodiment of FIG.
6 is an exemplary diagram illustrating a process of calculating a first characteristic value and a second characteristic value for calculating characteristic information.
7 shows a result of converting the pixel values of the input image into color space component values according to a color system different from the color system representing the pixel values in the feature information calculation unit.
8 is an exemplary diagram illustrating a process of calculating a third characteristic value for calculating characteristic information.
9 is an enlarged block diagram of the composition analyzing unit in the embodiment of FIG.
FIG. 10 shows a composition direction and a composition score of an image analyzed by a composition analysis unit.
11 is a flowchart of an image processing method according to an embodiment of the present invention.
12 is an enlarged flow chart of steps calculated in the embodiment of Fig.
13 is an enlarged flow chart of the step of analyzing in the embodiment of FIG.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In the following description with reference to the accompanying drawings, the same or corresponding components are denoted by the same reference numerals, and a duplicate description thereof will be omitted.

In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope. The singular expressions include plural expressions unless the context clearly dictates otherwise.

Each of the steps described below may be implemented by one or a plurality of software modules, or hardware that is responsible for each function, or a combination of software and hardware. Throughout the specification, when an element is referred to as " including " an element, it does not exclude other elements unless specifically stated to the contrary. The terms "part", "unit", "module", "block", and the like described in the specification mean units for processing at least one function or operation, And a combination of software. Specific meanings and examples of the terms will be described below in accordance with the order of each drawing. Hereinafter, the configuration of an image processing apparatus according to an embodiment of the present invention will be described in detail with reference to related drawings.

1 is a block diagram of an image processing apparatus 10 according to an embodiment of the present invention.

The image processing apparatus 10 includes a feature point extracting unit 100, an image dividing unit 200, a feature information calculating unit 300, and a composition analyzing unit 400.

The image processing apparatus 10 can automatically analyze the composition of an image by receiving an image. In the present invention, the composition of the image includes information on the orientation and composition score of the composition. In addition, in the present invention, the composition of the image means information on the arrangement of objects included in the image, and includes a circle composition, a triangle composition, a diagonal composition, a wedge composition composition, a radiation composition, an S composition, a parallel composition, A tile composition, a vertical and horizontal composition, and other compositions set by the user.

The image processing apparatus 10 receives an image, extracts feature points, repeatedly divides the input image from the extracted feature points into predetermined divided regions to generate divided images, and uses the divided images to extract feature information And analyze the image. The image processing apparatus 10 may be implemented in a portable device or the like to analyze a composition of an input image, but at least one of the feature information calculating unit 300 and the composition analyzing unit 400 may be implemented in a separate server or the like, Or the like.

The feature point extraction unit 100 extracts feature points by considering pixel values for each pixel from an input image. Will be described with reference to FIG.

For example, the feature point extracting unit 100 may designate an arbitrary pixel in the input image, and may extract the designated pixel as a feature point in consideration of a pixel value change amount of pixels adjacent to the designated pixel. In the present invention, the feature point extracting unit 100 may use an FAST feature point detector and an ORB (Oriented Fast and Rotated BRIEF) detector based on a Visual Descriptor BRIEF (Binary Robust Independent Elementary Feature). The feature point extraction unit 100 extracts feature points by considering pixel values for each pixel on the input image, and FIG. 2 shows feature points extracted by focusing on the boundary region of the subject on the input image.

Specifically, in order to determine which arbitrary pixel p on the input image corresponds to the feature point, the feature point extraction unit 100 uses a pixel value of 16 pixels on a circle having a radius of 3 centered at an arbitrary pixel p It is possible to determine whether the arbitrary pixel is a minutiae point. The feature point extraction unit 100 determines that the feature point is a case where n or more pixel values that are brighter than the pixel value of any pixel p are continuous or n or more pixel values that are darker than the pixel value of any pixel p are continuous. In the FAST algorithm, the n value may be set differently according to the performance requirements of the image processing apparatus 10.

The image divider 200 repeatedly divides the extracted input image into predetermined divided regions to generate divided images. Will be described with reference to FIG.

For example, the image dividing unit 200 repeatedly divides the image into a predetermined dividing region, calculates the width or length of the input image, and initially divides the calculated longer portion according to the golden ratio, The image divided into narrow regions among the input images is re-divided in the same manner as the initial division.

For example, as shown in FIG. 3, the image divider 200 may initially divide the horizontal axis of the input image having a longer horizontal length by a ratio of 1.618: 1, and divide P1 and Q1 Branch images). Among the generated divided images, the divided image Q1 means a divided image divided into an area narrower than the divided image P1. The image divider 200 can re-divide the vertical axis of the segmented image Q1 having a longer vertical length to a ratio of 1.618: 1 to generate a segmented image (having a divided region of P2 and Q2) marked P2 and Q2 have. Among the generated divided images, the divided image Q2 means a divided image divided into an area narrower than the divided image P2.

In the present invention, at least one of the horizontal or vertical lengths included in the predetermined divided areas forms a golden ratio with at least one horizontal or vertical length among the different divided areas. Accordingly, the image divider 200 can divide the input image into golden images to generate divided images. When the image dividing unit 200 repeatedly divides an input image into a predetermined dividing region, the dividing image approaches infinity, which can be represented by a point, which can correspond to the center of the spiral.

In another embodiment, a spiral may be formed that passes through at least one diagonal edge of at least one of the split images divided by the image divider 200 and converges on at least one of the split images, If there is a relationship, the helix can be defined as a golden helix. The pixel on the segmented image where the golden helix converges can be defined as the center of the helix and the image processing apparatus 10 can measure the composition score of the input image where the subject is located at the center of the golden helix. Will be described with reference to FIG.

For example, the generated helix may start at at least one corner of the input image, and may have four different helical directions depending on the position of the edge and the coordinates of the pixel where the helix converges. If the spiral starts at the upper left corner of the corner of the input image and the coordinates of the pixel where the spiral converges are located in the split image located at the upper right of the input image, Can be indicated.

In another embodiment, if the helix starts at the bottom left corner of the edge of the input image and the coordinates of the pixel where the helix converges are located at the bottom right of the input image, the direction of the helix is represented by the helix direction 4 . As will be described later, the composition analyzing unit 400 can analyze the composition of the input image, and the composition analysis of the image includes predicting the direction of the spiral. The matters related to the golden helix will be described later in the feature information calculation unit 300. [

The feature information calculating unit 300 includes a spiral generating unit 320, a first characteristic value calculating unit 340, a second characteristic value calculating unit 360, and a third characteristic value calculating unit 380. Will be described with reference to Figs.

The feature information calculation unit 300 converts the pixel values of the input image into color space component values according to a color system different from the color system that represents the pixel values and outputs the converted color space component values and the extracted feature points The feature information on the composition of the input image is calculated.

For example, the pixel value of each pixel of the image input to the image processing apparatus 10 may be represented by an RGB value. In addition, the color system different from the RGB color system may be a color system that transmits only the information of luminosity, and may mean a gray scale that expresses the pixel values as one single sample. In the present invention, The conversion into component values means that the gray scale, which is another color system, is arranged in the one-dimensional color space and converted into one color space component value among the 0 to 255 color space component values listed in the one-dimensional color space . In the present invention, a color space means that a color system, which is a system for expressing colors, is imaged in space.

For example, when the RGB colorimetric system is imaged in space, it can be imaged in a three-dimensional RGB color space, and when the grayscale is imaged in space, it can be imaged in a one-dimensional color space. The conversion of the pixel values of the input image into the color space component values according to the colorimetric system different from the colorimetric system that represents the pixel values is performed by converting the input image represented by the RGB pixel values into the saliency map Map).

In the present invention, the feature information calculating unit 300 may use at least any one of color, brightness, motion, harmony, and orientation of images in forming the salient map.

For example, the feature information calculator 300 may generate a saliency map by expressing a pixel value of each input pixel in the input image represented by RGB pixel values using a color of the image, . The feature information calculating unit 300 may calculate a saliency in context (SALICON) based on a depth learning based on an input image, or use a BMS (A Boolean Map Approach) algorithm for fast processing, The threshold value can be obtained, and the feature information can be calculated using the threshold value.

In another embodiment, the feature information calculation unit 300 may convert the RGB pixel values of each pixel of the input image into color space component values having a level between 0 and 255 according to the gray scale, And feature information on the composition of the input image based on the feature points.

The spiral generator 320 is formed within the split images, and generates spirals that converge on at least one split image, passing through at least one diagonal edge on both sides of the split images. Preferably, the helix means a golden helix, and the type of helix and the method of generating the helix are as described above.

The first characteristic value calculation unit 340 calculates a first characteristic value in consideration of the degree to which the minutiae points of the respective subregions occupy all of the minutiae extracted from the input image. The first characteristic value can be obtained by the following equation (1).

Here, f1 is a first characteristic value, p1 is the number of feature points 122 existing in a wide divided image among the divided images generated when the input image is initially divided, q1 is the number of feature points 122 I is an integer from 1 to 4, and qi is the number of feature points existing on the divided image divided into narrow regions among the divided images generated in the i-th divided region, it means.

For example, if Equation 1 is applied in the chart shown in Fig. 6, p1 can be set to 3 and q1 to 5. According to Equation (1), the first characteristic value f1 may be defined as a ratio of the number of minutiae in the entire region to the number of minutiae in the divided region, and the characteristic information calculator 300 uses the calculated first characteristic value To thereby calculate the feature information.

The second characteristic value calculation unit 360 calculates the second characteristic value in consideration of the average coordinates of the minutiae points of the divided regions, the coordinates of the pixel where the spiral converges, and the corner coordinates of the input image. The second characteristic value can be obtained by the following equation (2).

Here, f2 is a second characteristic value, i is an integer from 1 to 4, and distance_pqi is a distance between the average coordinate 362 of the minutiae points of the i-th divided region and the coordinates of the pixel where the spiral converges (the center of the helix 322) The mean, max_distance, is the maximum distance between the corner of the input image and the coordinates of the pixel where the spiral converges (the center of the spiral, 322).

6, the second characteristic value calculation unit 360 calculates the center coordinates 322 of the spirals marked with triangles and the average coordinates of the minutiae for each of the divided regions (for example, 362, and the feature information calculating unit 300 may calculate the feature information using the calculated second feature value.

The third characteristic value calculation unit 380 calculates a third characteristic value in consideration of the average coordinates of pixels having component values of a predetermined threshold value or more among the color space component values and the coordinates of the pixel where the spirals converge. The third characteristic value can be obtained by the following equation (3).

Here, f3 is the third characteristic value, and distance_s is the distance between the average coordinate 382 of the pixels whose color values exceed the threshold on the Salliance Map and the coordinates of the pixel where the spiral converges (the center of the spiral 322) The distance and max_distance mean the maximum distance between the corner of the input image and the coordinates of the pixel where the spiral converges.

For example, when the SALICON algorithm is used, the third characteristic value calculation unit 380 uses the average coordinates of pixels having a threshold value larger than 70, and when the BMS algorithm is used, The third characteristic value can be calculated using the average coordinates. Will be described with reference to Figs. 9 and 10. Fig.

The composition analyzing unit 400 includes a directional prediction model generation unit 420 and a scored prediction model generation unit 400.

For example, the composition analyzing unit 400 may analyze the composition of the input image using the feature information calculated by the feature calculating unit 300. The composition of the image includes information as to whether or not it matches the direction and composition of the composition. The composition analyzing unit 400 receives an image as shown in FIG. 9, and converts the direction of the golden helix on the input image to a score to convert the degree of golden division into a score.

For example, the composition analyzing unit 400 generates a learning model by learning the feature information calculated by the feature information calculating unit 300 using a machine learning technique, The composition can be analyzed. In the present invention, the composition analyzing unit 400 can generate a learning model using algorithms such as Logistic Classification, Support Vector Machine (SVM), and Random Forest (RF).

The directional prediction model generation unit 420 learns the calculated feature information using a machine learning technique to generate a directional prediction model for the direction of the composition.

Table 1 shows the accuracy of the directional prediction model. The degree by which the composition analyzing unit 400 matches the direction of the spiral predicted using the directional prediction model with the results of the survey conducted can be expressed by the accuracy of the directional prediction model. In Table 1, f1 + f2 + f3 (by SALICON) indicates the accuracy of the directional prediction model according to the salicone algorithm, and f1 + f2 + f3 (by BMS) indicates the accuracy of the directional prediction model according to the BMS algorithm. In the present invention, it can be seen that the performance can be evaluated through the 5-fold crossover verification, and the accuracy of the directional prediction model according to the salicone algorithm is as high as 85.9%.

More specifically, the directional prediction model generation unit 420 downloads 207 photographs from an arbitrary database in order to generate a directional prediction model, generates a predictive model, and evaluates the accuracy of the generated predictive model 207 A preliminary questionnaire is conducted on the photographs of the chapters, and the results of questionnaires on the direction of the spirals can be obtained in advance and compared with the results of the predictive model, and the accuracy can be evaluated.

Score Prediction Model Generation Unit 400 The feature information calculated by the feature information calculation unit 300 is learned by a machine learning technique to generate a score prediction model for the score of the composition. For example, the score prediction model generation unit 400 may learn the feature information using a linear regression algorithm and generate a score prediction model. The score prediction model generation unit 400 generates the score prediction model by learning the score of the composition predicted using the score prediction model and the score of the score obtained from the results of the preliminary survey conducted on the 207 photographs in a direction that minimizes the error of the score can do.

Table 2 shows the scores obtained by the Score Prediction Model and the mean scores of the questionnaire surveys for four spiral directions in total. In Table 2, f1 + f2 + f3 (by SALICON) represents the error of the score prediction model according to the salicone algorithm, and f1 + f2 + f3 (by BMS) represents the error of the score prediction model according to the BMS algorithm. It can be seen that the score prediction model based on the salicone algorithm shows better prediction results.

2 is an exemplary view of an input image from which feature points are extracted according to an embodiment of the present invention.

The feature point extracting unit 100 may designate an arbitrary pixel in the input image and may extract the designated pixel as a feature point in consideration of a pixel value change amount of pixels adjacent to the designated pixel. The number of feature points extracted by the feature point pivoting unit 100 is not limited and the number of pixels adjacent to any pixel for determining whether the feature point is a feature point may vary according to the requirements of the image processing apparatus 10. [

3 is an illustration of a spiral generated on a segmented input image according to one embodiment.

The image divider 200 repeatedly divides the extracted input image into predetermined divided regions to generate divided images. The image divider 200 can repeatedly divide the inputted image according to the golden ratio, and splits the long axis on the horizontal axis or the vertical axis first.

For example, since the horizontal axis of the input image shown in FIG. 3 is longer, the image dividing unit 200 can divide the input image into a length ratio of 1.618: 1, And a method of widely dividing the left image. The image dividing unit 200 divides the divided image into a narrow region among the divided images generated after the initial division. After the initial partitioning, the second partitioning process is as follows.

Since the length of the vertical axis of the right divided image, which is the divided image divided into the narrow area, is long, the image dividing unit 200 can divide the vertical axis into the length ratio of 1.618: 1, (1.618: 1) and a method of broadly dividing the bottom partition image (1: 1.618). The divided images generated by repeatedly dividing the input image by the image division unit 200 may converge to a specific point at the time of division by infinite number of times.

In another embodiment, a helix may be formed that passes through at least one diagonal edge of at least one of the divided images divided by the image divider 200, starting at at least one edge of the input image, In the case of a golden split relation, the above helix can be defined as a golden helix. In the present invention, the golden helix can have four golden helix directions according to the position of the corner starting from the input image and the coordinates of the pixel where the golden helix converges.

The image processing apparatus 10 can evaluate the composition score of the input image high if the subject on the input image is arranged along the direction of the golden helix. The golden spirals and the golden partitions are the same as described above, so they are omitted.

4 is an illustration of four different helical directions according to the position of the edge of the input image at which the helix starts and the coordinates of the pixel where the helix converges.

The spiral generated by the spiral generating unit 320 may be divided into four spiral directions according to the positions of the edges starting from the input image and the coordinates of the pixels where the spiral converges. For example, if the helix starts at the upper left corner of the input image, and the coordinates of the pixel where the helix converges are in the upper right corner of the input image, the helix direction 1 can be defined.

5 is an enlarged block diagram of the feature information calculation unit in the embodiment of FIG.

The feature information calculating unit 300 includes a spiral generating unit 320, a first characteristic value calculating unit 340, a second characteristic value calculating unit 360, and a third characteristic value calculating unit 380. The above-described points related to the feature information calculating unit 300 will be omitted.

For example, the feature information calculation unit 300 may convert the pixel values of the input image into color space component values according to a color system different from the color system that represents the pixel values, And the feature information on the composition of the input image is calculated based on the extracted feature points. In the present invention, the pixel values of the input image are represented by RGB values in the RGB colorimetric system, and the colorimetric system other than the RGB colorimetric system may be provided in a gray scale including only luminous intensity information of the color.

The feature information calculating unit 300 converts the input image into a salient map according to a gray scale, and uses the coordinates of a pixel corresponding to a threshold value or more of a predetermined threshold value in the converted salient map, And calculates feature information by using this value.

6 is an exemplary diagram illustrating a process of calculating a first characteristic value and a second characteristic value for calculating characteristic information.

The first characteristic value calculation unit 340 calculates a first characteristic value in consideration of the degree to which the minutiae points of the respective subregions occupy all of the minutiae extracted from the input image.

The process of calculating the first characteristic value is the same as described above, and thus will not be described. In the present invention, the degree to which the minutiae of each divided region occupies the extracted minutiae points may mean the ratio of the number of minutiae occupied by the minutiae points of the minutiae in the total number of minutiae points.

The second characteristic value calculation unit 360 calculates a second characteristic value in consideration of the average coordinates of the minutiae of each of the divided regions, the pixel coordinates of the convergence of the helix, and the corner coordinates of the input image. The process of calculating the second characteristic value is the same as described above, and thus will not be described.

7 shows a result of converting the pixel values of the input image into color space component values according to a color system different from the color system representing the pixel values in the feature information calculation unit.

In the present invention, converting the pixel values of the input image into color space component values according to a colorimetric system different from the colorimetric system that represents the pixel values may be expressed as forming a salient map. For example, the feature information calculating unit 300 may form a salience map (importance map) using any one of color, brightness, motion, harmony, or direction of an image.

Specifically, the process of forming a salient map includes: generating a low-level feature map using the one characteristic; calculating a region of interest map using the low-level characteristic map; And setting a conditional random field (CRF) using information on any one of a scale contrast, a center-surround histogram (CSH), and a spatial distribution of colors. And setting a region of interest that is a core region from the calculation.

FIG. 8 is a diagram illustrating a process of calculating a third characteristic value for calculating characteristic information; FIG.

The third characteristic value calculation unit 380 calculates the third characteristic value in consideration of the average coordinates of the pixels having the component values equal to or greater than the predetermined threshold value among the color space component values and the coordinates of the pixel where the spiral converges. The method for calculating the third characteristic value according to Equation (3) is as described above.

9 is an enlarged block diagram of the composition analyzing unit in the embodiment of FIG.

The composition analyzing unit 400 includes a directional prediction model generation unit 420 and a scored prediction model generation unit 400. For example, the composition analyzing unit 400 may analyze the composition of the input image using the feature information calculated by the feature calculating unit 300. A specific method of analyzing the composition of the image in the composition analyzing unit 400 is as described above, and thus will be omitted.

FIG. 10 shows a composition direction and a composition score of an image analyzed by a composition analysis unit.

The image processing apparatus 10 receives an image and analyzes the composition of the input image. The image processing apparatus 10 analyzes the composition of the image, including the prediction of the golden spiral direction and the estimation of the composition score. The specific method of analyzing the image by the composition analyzing unit 400 is the same as described above, and thus will not be described.

11 is a flowchart of an image processing method according to an embodiment of the present invention.

The image processing method includes the following steps performed in the image processing apparatus 10 in a time-series manner.

In step S100, the feature point extraction unit 100 extracts feature points by considering pixel values for each pixel from the input image. The feature point extracting unit 100 may designate an arbitrary pixel in the input image and extract a designated pixel as a feature point in consideration of a pixel value change amount of pixels adjacent to the designated pixel. The detailed method for extracting the minutiae points by the minutiae point extracting unit 100 is the same as described above, and thus will not be described.

In S200, the image dividing unit 200 repeatedly divides the extracted input image into predetermined divided regions to generate divided images. The method of dividing the input image by the image divider 200 is the same as described above, and thus will not be described.

In S300, the feature information calculation unit 300 converts the pixel values of the input image into color space component values according to a color system different from the color system representing the pixel values, and outputs the converted color space component values and the extraction And the feature information on the composition of the input image is calculated based on the feature points. A specific method of calculating the first characteristic value, the second characteristic value, and the third characteristic value by the characteristic information calculation unit 300 and calculating the characteristic information using the calculated characteristic values is as described above, and thus will not be described.

In operation S400, the composition analyzing unit 400 learns the feature information calculated by the feature information calculating unit 300 using a machine learning technique to generate a learning model, and generates a learning model by using the generated learning model The composition can be analyzed. The details of the machine learning algorithm used by the composition analyzer are as described above, and thus will be omitted.

12 is an enlarged flow chart of steps calculated in the embodiment of Fig.

The calculating step includes the following steps performed in the feature information calculating unit 300 in a time series manner.

At S320, the spiral generator 320 is formed inside the split images, and generates spirals that converge on at least one split image, passing through at least one diagonal edge of at least one of the split images. The spirals generated by the spiral generating unit 320 and the golden spirals are the same as described above, and thus will not be described.

In step S340, the first characteristic value calculation unit 340 calculates a first characteristic value in consideration of the degree to which the minutiae points of the respective subregions occupy all of the minutiae extracted from the input image. The process of calculating the first characteristic value is as described above in Equation (1) and is omitted.

In step S360, the second characteristic value calculation unit 360 calculates a second characteristic value in consideration of the average coordinates of the minutiae points of the divided areas, the coordinates of the pixels where the spiral converges, and the corner coordinates of the input image. The process of calculating the second characteristic value is as described above in Equation (2) and is omitted.

In S380, the third characteristic value calculation unit 380 calculates a third characteristic value in consideration of the average coordinates of the pixels having component values of a predetermined threshold value or more among the color space component values and the coordinates of the pixel where the spiral converges . The process of calculating the third characteristic value is as described above in Equation (3) and is omitted.

13 is an enlarged flow chart of the step of analyzing in the embodiment of FIG.

The analyzing step includes the following steps performed in the composition analyzing unit 400 in a time-series manner.

In S420, the directional prediction model generation unit 420 generates a directional prediction model for the direction of the composition by machine-learning the calculated feature information. The machine learning algorithm used by the directional prediction model generation unit 420 and the direction of the golden helix are the same as described above and are therefore omitted.

In S440, the score prediction model generation unit 440 performs a machine learning on the calculated feature information to generate a prediction model for the score of the composition. The machine learning algorithm and the composition score used by the score prediction model generation unit 400 are the same as described above, and thus will not be described.

All or part of the method of an embodiment of the present invention described above can be implemented in the form of a computer-executable recording medium (or a computer program product) such as a program module executed by a computer. Here, the computer-readable medium may include computer storage media (e.g., memory, hard disk, magnetic / optical media or solid-state drives). Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media.

Also, all or part of the method according to an embodiment of the present invention may include instructions executable by a computer, the computer program comprising programmable machine instructions to be processed by a processor, Language, an object-oriented programming language, an assembly language, or a machine language.

Means or module in the present specification may mean hardware capable of performing the functions and operations according to the respective names described herein and may be implemented by computer program code , Or may refer to an electronic recording medium, e.g., a processor or a microprocessor, having computer program code embodied thereon to perform particular functions and operations. In other words, a means or module may mean a functional and / or structural combination of hardware for carrying out the technical idea of the present invention and / or software for driving the hardware.

Thus, a method according to an embodiment of the present invention may be implemented by a computer program as described above being executed by a computing device. The computing device may include a processor, a memory, a storage device, a high-speed interface connected to the memory and a high-speed expansion port, and a low-speed interface connected to the low-speed bus and the storage device. Each of these components is connected to each other using a variety of buses and can be mounted on a common motherboard or mounted in any other suitable manner.

It will be apparent to those skilled in the art that various modifications, substitutions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims. will be. Therefore, the embodiments disclosed in the present invention and the accompanying drawings are intended to illustrate and not to limit the technical spirit of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments and the accompanying drawings . The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

Claims (20)

Extracting feature points from the input image in consideration of pixel-by-pixel pixel values;
The pixel values are converted into color space component values according to a colorimetric system different from the colorimetric system that represents the pixel values, and the feature relating to the composition of the input image based on the converted color space component values and the extracted feature points Calculating information; And
Analyzing a composition of the input image using the calculated feature information; / RTI > The method of claim 1, further comprising:
Repeatedly dividing the extracted input image into predetermined divided regions; Further comprising:
And the feature information is calculated based on the minutiae of each divided region. 2. The method of claim 1, wherein the extracting comprises:
Designating an arbitrary pixel in the input image and extracting the designated pixel as a feature point in consideration of a pixel value change amount of pixels adjacent to the designated pixel. 3. The method of claim 2, wherein the calculating step
Generating a spiral that is formed within the split images, passes through at least one diagonal edge of the split images, and converges in at least one split image; Further comprising:
And the feature information is calculated using coordinates of a pixel where the generated helix converges. 5. The method of claim 4, wherein the calculating step
Calculating a first characteristic value in consideration of the degree of each of the minutiae of each of the divided regions occupying the entire minutiae extracted from the input image; Further comprising:
And the feature information is calculated using the calculated first characteristic value. 6. The method of claim 5, wherein the calculating step
Calculating a second characteristic value in consideration of the average coordinates of the minutiae of the divided regions, the coordinates of the pixel where the spiral converges, and the corner coordinates of the input image; Further comprising:
And the feature information is calculated using the calculated second characteristic value. 7. The method of claim 6, wherein the calculating step
Calculating a third characteristic value in consideration of the average coordinates of pixels having component values of a predetermined threshold value or more among the color space component values and the coordinates of the pixel where the spiral converges; Further comprising:
And the feature information is calculated using the calculated third characteristic value. 3. The method as claimed in claim 2, wherein the repeated division into the predetermined dividing area
The method includes the steps of: calculating a width or a length of the input image; dividing the calculated length into longer portions according to a golden ratio; and dividing the divided image into narrow regions in the same manner as the initial division Determining the divided region, and repeatedly dividing the divided region into the determined divided region. 9. The method of claim 8,
Starting at at least one corner of the input image and having different spiral directions depending on the position of the edge and the coordinates of the pixel where the spiral converges. 3. The method of claim 2,
Learning the calculated feature information using a machine learning technique to generate a directional prediction model for the direction of the composition; Further comprising:
And analyzing the composition of the image using the generated directional prediction model. 11. The method of claim 10, wherein analyzing
Learning the calculated feature information using the machine learning technique to generate a score prediction model for the score of the composition; Further comprising:
And analyzing a composition of the image using the generated score prediction model. A program stored in a computer-readable recording medium for realizing the image processing method according to any one of claims 1 to 11 through being executed by a processor. A feature point extracting unit for extracting feature points by considering pixel values for each pixel from an input image;
The pixel values are converted into color space component values according to a colorimetric system different from the colorimetric system that represents the pixel values, and the feature relating to the composition of the input image based on the converted color space component values and the extracted feature points A feature information calculation unit for calculating information; And
A composition analyzing unit for analyzing a composition of the input image using the calculated feature information; And the image processing apparatus. 14. The method of claim 13,
An image dividing unit for dividing an input image from the minutiae points into a predetermined divided region to generate divided images; Further comprising:
Wherein the feature information calculating unit calculates the feature information based on the feature points for each of the divided regions. 14. The apparatus of claim 13, wherein the feature information calculation unit
A spiral generating unit formed in the divided images, the spiral generating unit generating spirals that converge in at least one divided image, passing through at least one diagonal side edge of the divided images; Further comprising:
And the feature information is calculated using the coordinates of the pixel where the generated helix converges. 16. The apparatus of claim 15, wherein the feature information calculation unit
A first characteristic value calculation unit for calculating a first characteristic value in consideration of the degree of each of the minutiae of each of the divided regions occupying all the minutiae extracted from the input image; Further comprising:
And calculates the feature information by using the calculated first characteristic value. 17. The apparatus of claim 16, wherein the feature information calculation unit
A second characteristic value calculation unit for calculating a second characteristic value in consideration of the average coordinates of the minutiae of each of the divided regions, the coordinates of the pixel where the helix converges, and the corner coordinates of the input image; Further comprising:
And calculates the feature information using the calculated second characteristic value. 18. The apparatus of claim 17, wherein the feature point calculating unit
A third characteristic value calculation unit for calculating a third characteristic value in consideration of an average coordinate of pixels having component values of a predetermined threshold value or more among the color space component values and a coordinate of a pixel where the spiral converges; Further comprising:
And calculates the feature information by using the calculated third characteristic value. 15. The method of claim 14,
Wherein the image processing apparatus starts at at least one corner of the input image and has different spiral directions depending on the position of the edge and the coordinates of a pixel where the spiral converges. 15. The apparatus of claim 14, wherein the composition analyzing unit
A direction prediction model generation unit that learns the calculated feature information using a machine learning technique to generate a directional prediction model for the direction of the composition; And
A score prediction model generation unit that learns the calculated feature information using the machine learning technique to generate a score prediction model for the score of the composition; Further comprising:
And analyzes the composition of the image using the generated direction prediction model and the score prediction model.

Images