CN111950565B - Abstract painting image orientation recognition method based on feature fusion and naive Bayes - Google Patents

Abstract

The invention belongs to the technical field of image processing and computer vision, and particularly relates to an abstract picture image direction identification method based on feature fusion and naive Bayes, which comprises the following steps: s1, rotating the abstract picture images to obtain four abstract picture images with different directions; simultaneously, dividing the abstract picture image to obtain four subblocks; s2, extracting low-level features of the abstract picture image; s3, extracting the high-level features of the abstract picture image by adopting a Convolutional Neural Network (CNN); s4, linearly combining the image low-level characteristic value and the image high-level characteristic value to obtain a final characteristic value of the abstract picture image; and S5, inputting the final characteristic value of the abstract image into a naive Bayes classifier for training and prediction. The method acquires the characteristic value of the image by fusing the low-level and high-level characteristics, and then puts the characteristic value of the image into a naive Bayes classifier (NB) for training and prediction, thereby realizing the automatic prediction of the direction of the abstract image and improving the prediction precision.

Description

Abstract painting image orientation recognition method based on feature fusion and naive Bayes

technical field

The invention belongs to the technical field of image processing and computer vision, in particular to an abstract painting image orientation recognition method based on feature fusion and Naive Bayes.

Background technique

Abstract art is a visual language that uses shape, color and line to compose images, and is independent of the world to a certain extent. Among them, paintings created for emotional expression are called "hot abstractions", while paintings that describe the world in an abstract way are called "cold abstractions". Usually when creating abstract paintings, the artist decides the correct hanging direction of the work according to his own aesthetic concept. Although the correct orientation is usually specified on the back of the canvas, this is not obvious to other lay viewers. Moreover, some studies in psychology in recent years have addressed the orientation of abstract painting, with most studies agreeing that correctly positioned paintings will receive higher aesthetic evaluations. Experiments with participants showed that about half of preference orientation decisions were in line with the artist's expected orientation, which is well above chance but below perfect performance. These provide evidence for the relationship between painting direction and aesthetic quality. The study of orientation recognition can reveal the objective laws of visual aesthetic evaluation.

With the trend of digitization of information, digital images of paintings can be easily found on the Internet. This enables computer-aided painting analysis. People have studied various aesthetic evaluation methods by directly exploring the relationship between human aesthetic perception and computational visual features, but none of them can solve the problem of aesthetic evaluation through computer-aided orientation judgment. The research status of image orientation in recent years is as follows: (1) The research on image orientation recognition is mainly aimed at photographic pictures, such as natural or scene images, and the recognition rate is relatively satisfactory. However, for abstract painting images, the content and semantics are more subtle and less obvious than photographic images, so it is more difficult to identify the direction of abstract paintings, and there is less related work in recent years. (2) Humans generally recognize the direction by understanding the content of the image, so some methods use high-level semantic features to recognize the direction of the image, and the accuracy is obviously higher. But its accuracy will largely depend on being able to bridge the semantic gap between high-level cues and low-level features.

The current extensive research on orientation in natural images motivates us to explore the problem of orientation judgment in abstract paintings. The purpose of the present invention is to better understand the sense of direction of abstract painting, especially to establish the relationship between the visual content of the image and the correct direction under the framework of machine learning when there is no substantive content.

SUMMARY OF THE INVENTION

The invention overcomes the deficiencies in the prior art, and provides a method for recognizing the direction of an abstract painting image based on feature fusion and Naive Bayes, which can realize automatic prediction of the direction of the abstract painting image through computer operation.

In order to solve the above-mentioned technical problems, the technical solution adopted in the present invention is: a method for recognizing the direction of an abstract painting image based on feature fusion and Naive Bayes, comprising the following steps:

S1. Rotate the abstract painting image by 0°, 90°, 180°, and 270° to obtain four abstract painting images with different directions. The abstract painting images are equally divided up and down, and left and right are equally divided; thus, each abstract painting image It is divided into four sub-blocks: upper, lower, left and right;

S2. Extract the low-level features of the abstract painting image, calculate the low-level feature descriptions of each sub-block respectively, and use the comparison result of the low-level feature descriptions of each word block as the image low-level feature value. If the comparison result is true, it is represented as 1, otherwise it is 0;

S3, using the convolutional neural network CNN to extract the high-level features of the abstract painting image, the specific steps are as follows:

S301. Adjust the four sub-blocks of the abstract painting into RGB color images of 128×128;

S302, respectively input the four sub-blocks into the convolutional neural network CNN. The convolutional neural network CNN includes three convolutional layers with a stride of 1, three 2×2 maximum pooling layers and two fully connected layers. The activation function in the convolutional layer adopts ReLU, the dimensions of the two fully connected layers are 1024 and 521 respectively, and finally a 512-dimensional vector is obtained as the neural network feature vector;

S303, judging the comparison result of the feature vectors of the upper and lower sub-blocks and the left and right sub-blocks as the high-level feature values f14 and f15 of the image, and the calculation formula is as follows:

f14=f_cnn _A ≥ f_cnn _B ; f15=f_cnn _L ≥ f_cnn _R ;

Where f_cnn _A , f_cnn _B , f_cnn _L , f_cnn _R represent the neural network eigenvalues of the upper, lower, left and right sub-blocks, respectively;

S4, linearly combine the low-level eigenvalues of the image obtained in step S2 and the high-level eigenvalues of the image obtained in step S3 to obtain the final eigenvalues of the abstract painting image;

S5. Perform the operations of the above steps S1 to S4 on all the abstract paintings in the image library to obtain the final eigenvalues of the abstract paintings images, input the naive Bayes classifier for training and prediction, and finally classify the abstract paintings into "up", "" Down", "Left" or "Right", which enables automatic prediction of the orientation of the abstract painting image.

The step S2 specifically includes the following steps:

S201, converting the four sub-blocks in step S1 into an HSV model from the RGB color space, dividing the H-S space into 16 hues and 8 saturations, and counting the number of pixels of 128 colors as the color histogram vector of the abstract painting; The comparison results of the histogram vectors of the upper and lower sub-blocks and the left and right sub-blocks are judged as the image feature values f1 and f2. The specific formula is as follows:

f1=Hist _A ≥Hist _B ; f2=Hist _L ≥Hist _R ;

Among them, Hist _A , Hist _B , Hist _L , Hist _R are the histogram vectors of the upper, lower, left and right sub-blocks respectively;

S202, representing the maximum gradient of the image as the complexity of the image, and calculating the complexity of the four sub-blocks; judging the comparison result of the complexity of the upper and lower sub-blocks and the left and right sub-blocks as the image feature values f3 and f4, the formula as follows:

f3 = Comp _A ≥ Comp _B ; f4 = Comp _L ≥ Comp _R ;

Among them, Comp _A , Comp _B , Comp _L , and Comp _R represent the complexity of the upper, lower, left and right sub-blocks, respectively;

S203, calculate the similarity between every two sub-blocks in the four sub-blocks; and use the comparison result of the similarity between the sub-blocks as the image feature values f5, f6 and f7; the formula is as follows:

f5=Sim(A,L)≥Sim(A,R); f6=Sim(B,L)≥Sim(B,R); f7=Sim(A,B)≥Sim(L,R);

S204. Use Hough transform to detect the significant straight lines of the four sub-blocks, determine whether it is a static line or a dynamic line according to the inclination angle α of the straight line, calculate the number of static lines, dynamic lines and the average length of all lines as image features, and use two of the sub-blocks as image features. The comparison results of the attribute values of the straight lines between the blocks are taken as the image feature values f8, f9, f10, f11, f12 and f13, respectively, and the formulas are as follows:

f8=Len_S _A ≥Len_S _B ; f9=Len_D _A ≥Len_D _B ; f10=Ave_Len _A ≥Ave_Len _B ;

f11=Len_S _L ≥Len_S _R ; f12=Len_D _L ≥Len_D _R ; f13=Ave_Len _L ≥Ave_Len _R ;

Among them, Len_S _A , Len_S _B , Len_S _L , and Len_S _R represent the number of static lines in the upper, lower, left, and right sub-blocks, respectively, and Len_D _A , Len_D _B , Len_D _L , and Len_D _R represent upper, lower, left, respectively , the number of dynamic lines in the four right sub-blocks, Ave_Len _A , Ave_Len _B , Ave_Len _L , and Ave_Len _R represent the average length of all lines in the upper, lower, left and right sub-blocks, respectively.

In the step S202, the calculation formula of the image complexity is as follows:

分别表示图像中(x,y)点的R、G、B的梯度值，Pixelnum(G)表示图像G的总像素点个数，Comp_G表示图像G的复杂度。Among them, G _max (x, y) represents the maximum gradient of the pixel (x, y) in the image in the RGB color space,

Represent the gradient values of R, G, and B at the (x, y) point in the image, respectively, Pixelnum(G) represents the total number of pixels in the image G, and Comp _G represents the complexity of the image G.

In the step S201, the formula for converting the image from the RGB color space to the HSV model is as follows:

kmax=max(r',g',b'); kmin=min(r',g',b'); Δ=kmax-kmin;

v = kmax;

where r, g, b represent the RGB values of image pixels in the RGB color space, respectively, r', g', and b' are intermediate variables, kmax represents the maximum value among r', g', and b', and kmin represents r' The minimum value among , g' and b', h, s, v represent the hue value, saturation and lightness of the image pixel in the HSV model.

The network structure of the convolutional neural network CNN is: the first convolutional layer consists of 16 convolution kernels of 3×3; the second convolutional layer consists of 8 convolutional kernels of 3×3; the third convolutional layer consists of 8 convolution kernels of 3×3. Each convolutional layer consists of four 3×3 convolution kernels. The feature map obtained after each convolution is filled with 0 to the edge part, keeping the size unchanged; after each convolutional layer, 2 is used ×2 max pooling to reduce feature resolution; finally, four 16×16 2D matrices are converted into 1024-dimensional feature vectors using fully connected layers, and 1024 is reduced to 512-dimensional.

In the step S4, the vector dimension of the final feature value of the linearly combined abstract painting image is 1291.

In the step S5, when the Naive Bayes classifier predicts the direction of the abstract painting image, the specific methods for performing four classifications of "upward", "downward", "leftward" and "rightward" are:

Divide these four cases into four groups: select one direction as one category for each group, and the other three directions as the other category, and calculate the ratio of the posterior probabilities of the two categories in each group. The calculation formula is as follows:

是每组中两类的后验概率比；P(C_θ|F)表示其中选择的方向的后验概率，

表示其余三个方向的后验概率；比较四组中的两类的后验概率比值

选取

值最大的方向作为抽象画图像的正确方向。in,

is the posterior probability ratio of the two classes in each group; P(C _θ |F) represents the posterior probability of the direction chosen among them,

Represents the posterior probability of the remaining three directions; compares the posterior probability ratio of the two categories in the four groups

select

The direction with the largest value is used as the correct direction for the abstract painting image.

Compared with the prior art, the present invention has the following beneficial effects:

The invention provides a method for recognizing the direction of an abstract painting image based on feature fusion and Naive Bayes. (1) The abstract painting image is equally divided up and down, and left and right equally divided. Thus, each abstract painting is divided into four sub-blocks (top, bottom, left and right). The image features are based on the comparison results of the four directional feature descriptions, which can more specifically reflect the directional structure of the image. (2) According to the basic principles of abstract painting theory, extract the low-level features of all abstract painting images, including color, complexity, similarity and straight line attributes. From the perspective of painting principles, these features can not only better express the basic characteristics of abstract paintings, but also reflect the directionality of images. (3) Convolutional neural network CNN is used to extract high-level features of abstract painting images. (4) Linearly combine low-level features and high-level features, and the combined vector is the final feature value of the abstract painting image. In this way, the local and global features of the image can be better fused, and the image orientation can be detected more accurately.

Description of drawings

1 is a schematic diagram of the rotation of an abstract painting in an embodiment of the present invention;

FIG. 2 is a schematic diagram of a summary abstract painting segmentation according to an embodiment of the present invention;

3 is a schematic structural diagram of a CNN model adopted in an embodiment of the present invention;

FIG. 4 is a framework for recognizing the direction of an abstract painting image according to an embodiment of the present invention.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not All the embodiments; based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work, all belong to the protection scope of the present invention.

The invention provides a method for recognizing the direction of an abstract painting image based on feature fusion and Naive Bayes. The method is selected to disclose the painting of a website, and an experiment is carried out. The specific implementation steps are as follows:

S1: Select 500 abstract paintings in the WikiArt (http://www.wikiart.org) dataset. Rotate all abstract painting images clockwise in four directions (0°, 90°, 180°, 270°), refer to Figure 1. Finally, four abstract paintings in different directions were obtained, totaling 2,000. The abstract painting image is equally divided up and down, and the left and right are equally divided. Thus, each abstract painting is divided into four sub-blocks (upper (A), lower (B), left (L) and right (R)), refer to FIG. 2 .

S2: According to the basic principles of abstract painting theory, extract the low-level features of all abstract painting images. Calculate the low-level feature description of each sub-block separately, and take the comparison result of these feature descriptions as the final image feature value. If the comparison result is true, it is represented as 1, otherwise it is 0. Specific steps are as follows:

S201: Convert the four sub-blocks in S1 into an HSV model (hue (h), saturation (s), lightness (v)) from the RGB color space. Calculated as follows:

v = kmax; (3)

Among them, r, g, b respectively represent the RGB value of the image pixel in the RGB color space, r', g' and b' are intermediate variables, kmax represents the maximum value among r', g' and b', and kmin represents r The minimum values of ', g' and b', h, s, v represent the hue, saturation and lightness of the image pixels in the HSV model.

kmax=max(r',g',b'); kmin=min(r',g',b'); Δ=kmax-kmin; (5)

In the direction recognition, the influence factor of lightness is very small, so the H-S space is divided into 16 hues and 8 saturations, and the number of pixels of 128 colors is counted as the color histogram vector of the painting. The image feature values f1 and f2 are the comparison results of the two sub-block histogram vectors, and the formula is as follows:

f1=Hist _A ≥Hist _B ; f2=Hist _L ≥Hist _R ; (6)

Among them, Hist _A , Hist _B , Hist _L , and Hist _R are the histogram vectors of the four sub-blocks, respectively. The dimensions of f1 and f2 are 128 dimensions.

S202: Denote the maximum gradient map of the image as the complexity of the image, and then calculate the complexity of the four sub-block images in step S1. Let the image be G, and in the RGB color space, calculate the maximum gradient of the pixel (x, y) in the image as G _max (x, y). Then the average value of G _max of all pixels in the image is taken as the complexity of the image. Calculated as follows:

是(x,y)点的梯度值，Pixelnum(G)是图像G的总像素点个数，Comp_G是图像G的复杂度。图像特征值f3和f4为两个子块的复杂度的比较结果，公式如下：Where (x, y) represents the coordinates of the pixel in the image,

is the gradient value of the (x, y) point, Pixelnum(G) is the total number of pixels in the image G, and Comp _G is the complexity of the image G. The image feature values f3 and f4 are the comparison results of the complexity of the two sub-blocks, and the formula is as follows:

f3 = Comp _A ≥ Comp _B ; f4 = Comp _L ≥ Comp _R ; (9)

Among them, Comp _A , Comp _B , Comp _L , and Comp _R represent the complexity of the upper, lower, left, and right sub-blocks, respectively.

S203: Calculate the similarity between every two sub-blocks in the four sub-blocks.

Assuming two images G ₁ and G ₂ , a Histogram Pyramid (HOG) is used to calculate the similarity between the two images. Taking the image as a unit with 8 directions, the HOG features of 3 channels are computed in RGB mode. The calculation formula of the similarity Sim(G ₁ , G ₂ ) between two images is as follows:

where G ₁ , G ₂ ∈ RGB, H ₁ and H ₂ are the corresponding normalized histograms of images G ₁ and G ₂ , respectively, and m is the number of cells present in the HOG feature. The image feature values f5, f6 and f7 are the comparison results of the similarity between sub-blocks, and the formula is as follows: f5=Sim(A,L)≥Sim(A,R); f6=Sim(B,L)≥Sim( B,R); f7=Sim(A,B)≥Sim(L,R); (11)

S204: Use Hough transform to detect the salient straight lines of the four sub-blocks. According to the inclination angle α of the line, if the inclination angle is (-15°<α<15°) or (75°<α<105°), the line is a static line, otherwise it is a dynamic line. Calculate the number of static lines, dynamic lines and the average length of all lines as image features. The image feature values f8, f9, f10, f11, f12 and f13 are the comparison results of the attribute values of the straight lines between the two sub-blocks, and the formula is as follows:

f8=Len_S _A ≥Len_S _B ; f9=Len_D _A ≥Len_D _B ; f10=Ave_Len _A ≥Ave_Len _B ; (12)

f11=Len_S _L ≥Len_S _R ; f12=Len_D _L ≥Len_D _R ; f13=Ave_Len _L ≥Ave_Len _R ; (13)

Among them, Len_S _A , Len_S _B , Len_S _L , and Len_S _R represent the number of static lines in the upper, lower, left, and right sub-blocks, respectively, and Len_D _A , Len_D _B , Len_D _L , and Len_D _R represent upper, lower, left, respectively , the number of dynamic lines in the four right sub-blocks, Ave_Len _A , Ave_Len _B , Ave_Len _L , and Ave_Len _R represent the average length of all lines in the upper, lower, left and right sub-blocks, respectively.

S3: Convolutional neural network CNN (Convolutional Neural Networks) is used to extract the high-level features of all abstract painting images. Refer to Figure 3 for the model. Specific steps are as follows:

S301. Adjust the four sub-blocks of the abstract painting into RGB color images of 128×128;

S302, respectively input the four sub-blocks into the convolutional neural network CNN. The convolutional neural network CNN includes three convolutional layers with a stride of 1, three 2×2 maximum pooling layers and two fully connected layers. The activation function in the convolutional layer adopts ReLU, the dimensions of the two fully connected layers are 1024 and 521 respectively, and finally a 512-dimensional vector is obtained as the neural network feature vector;

S303, judge the comparison result of the feature vectors of the upper and lower sub-blocks and the left and right sub-blocks, if it is true, it is represented as 1, otherwise it is 0, and the comparison result is used as the high-level feature values f14 and f15 of the image, and the calculation formula is as follows:

f14=f_cnn _A ≥f_cnn _B ; f15=f_cnn _L ≥f_cnn _R ; (14)

where f_cnn _A , f_cnn _B , f_cnn _L , and f_cnn _R represent the neural network eigenvalues of the upper, lower, left and right sub-blocks, respectively. The dimensions of f14 and f15 are 512 dimensions.

In this embodiment, the CNN includes three convolutional layers with a stride of 1, the activation function adopts ReLU, and the convolutional layer uses filters to convolve the input samples to obtain feature maps. The first convolutional layer consists of 16 3×3 convolution kernels; the second convolutional layer consists of 8 3×3 convolutional kernels; the third convolutional layer consists of 4 3×3 convolutional layers The feature map obtained after each convolution is filled with 0 to the edge part, keeping the size unchanged. The CNN contains 3 2×2 max pooling layers to reduce the resolution. The pooling layer samples the input data in order to reduce parameters and avoid overfitting; the CNN contains 2 fully connected layers to connect all neurons. The dimensions of the two fully connected are 1024 and 521 respectively, and finally the 512-dimensional vector is used as the eigenvalue of the neural network, which is represented by f_cnn. Other parameter settings of the CNN network: batch_size is 8, epochs is 10, learning rate is 1e-4, cost function selects "cross entropy loss function", and optimizer is "Adam".

S4: Linearly combine the image features f1-f15 of S2 and S3, and the combined vector is the final feature value of the abstract painting image. The combined feature vector dimension is 1291.

S5: 400 paintings are randomly selected as the original images of the training set, and 100 paintings are used as the test set. Therefore, the final training set samples are 1600 and the test set samples are 400 after the original images are rotated. To get more accurate classification results, 10-fold cross-validation was used to evaluate the classification model. Put the eigenvalues of the abstract paintings obtained in step S4 into Naive Bayes (NB) for training and prediction, and finally divide the abstract paintings into four categories: "up", "down", "left" and "right" , so as to realize the automatic prediction of the image orientation of abstract paintings. Refer to Figure 4 for the framework of image orientation recognition in abstract paintings.

When using the Naive Bayes classifier for binary classification ("up" and "non-up"), the ratio of the posterior probability is:

Among them, F=[f1, f2..., f15] represents the feature direction of the abstract painting image G, _C1 represents the upward category, and C2 represents the non-upward direction category. P(C ₁ ) and P(C ₂ ) are the prior probabilities of the two classes, respectively, P(C ₁ |F) and P(C ₂ |F) are the posterior probabilities of the two classes, respectively, P(F| C ₁ ) and P(F|C ₂ ) represent the conditional probabilities of all features, respectively, and P(f _i |C ₁ ) and P(f _i |C ₂ ) represent the conditional probability of the i-th feature state, respectively.

All features are discrete, P(f _i |C _j ) (i=1, 2, . . . , 1291, j=1, 2) is consistent with a 0-1 distribution. The conditional probability P(f _i |C _j ) of each feature state can be calculated during the training phase. In the prediction stage, the probability of which category the abstract painting G should be classified into is determined according to the posterior probability ratio. The formula is as follows:

Wherein, T is a threshold, and in this embodiment of the present invention, the threshold T takes a value of 0.5.

作为另一类。然后计算每组中两类的后验概率的比值，公式如下：In this embodiment, the naive Bayes classifier can also be used to classify the abstract paintings into four directions, and the abstract paintings images are identified as four directions of "upward", "downward", "leftward" and "rightward". The specific method is: divide these four situations into four groups: choose one of the directions θ as one category, and the other three directions

as another category. The ratio of the posterior probabilities of the two classes in each group is then calculated using the following formula:

是每组中两类的后验概率比。比较各组的

值，选取

值最大的方向作为抽象画图像的正确方向。in,

is the ratio of the posterior probabilities of the two classes in each group. Compare the groups

value, select

The direction with the largest value is used as the correct direction for the abstract painting image.

In order to fully verify the effectiveness and applicability of the method of the present invention, the low-level features, high-level features, and low-level and high-level features are used to test the classification model, and the classification accuracy is shown in Table 1. The experimental results show that whether the abstract painting images are divided into two categories or four categories, the method of fusion of low-level features and high-level features has the highest classification accuracy.

Table 1: Comparison of classification accuracy under different features

In addition, the fused features are classified and tested on commonly used classifiers, and the test results are shown in Table 2. The results show that, since in the embodiment of the present invention, the feature values are all 1 or 0, the classification accuracy obtained by using the naive Bayesian multi-classification model of the present invention is higher.

Table 2: Comparison of classification accuracy under different classifiers

To sum up, the present invention provides a method for recognizing the direction of an abstract painting image based on feature fusion and Naive Bayes. The Naive Bayes classifier (NB) is trained and predicted, which realizes the automatic prediction of the image direction of abstract paintings, and can effectively identify the direction of the image, that is, it can establish the relationship between the visual content of the image and the correct direction under the framework of machine learning. , and improve the prediction accuracy.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions described in the foregoing embodiments can still be modified, or some or all of the technical features thereof can be equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present invention. scope.

Claims (4)

1. An abstract picture image direction identification method based on feature fusion and naive Bayes is characterized by comprising the following steps:

s1, rotating the abstract drawing image by 0 degrees, 90 degrees, 180 degrees and 270 degrees to obtain four abstract drawing images with different directions, and performing upper-lower average segmentation and left-right average segmentation on the abstract drawing images; therefore, each abstract picture image is divided into an upper sub-block, a lower sub-block, a left sub-block and a right sub-block;

s2, extracting low-level features of the abstract picture image, respectively calculating low-level feature descriptions of each sub-block, taking a comparison result of the low-level feature descriptions of each block as an image low-level feature value, if the comparison result is true, representing as 1, otherwise, representing as 0; the step S2 specifically includes the following steps:

s201, converting the four subblocks in the step S1 from an RGB color space into HSV models, dividing the H-S space into 16 hues and 8 saturations, and counting the number of pixels of 128 colors to be used as a color histogram vector of an abstract picture; judging the comparison result of the histogram vectors of the upper sub-block, the lower sub-block and the left sub-block and the right sub-block as image characteristic values f1 and f2, wherein the specific formula is as follows:

f1＝Hist_A≥Hist_B；f2＝Hist_L≥Hist_R；

wherein, Hist_A，Hist_B，Hist_L，Hist_RHistogram vectors of an upper sub-block, a lower sub-block, a left sub-block and a right sub-block are respectively;

the formula for converting the image from the RGB color space to the HSV model is as follows:

kmax＝max(r′，g′，b′)；kmin＝min(r′，g′，b′)；Δ＝kmax-kmin；

v＝kmax；

the method comprises the following steps that r, g and b respectively represent RGB values of image pixels in an RGB color space, r ', g ' and b ' are intermediate variables, kmax represents the maximum value of r ', g ' and b ', kmin represents the minimum value of r ', g ' and b ', and h, s and v represent hue values, saturation and brightness of the image pixels in an HSV model;

s202, representing the maximum gradient of the image as the complexity of the image, and calculating the complexity of the four sub-blocks; judging the comparison result of the complexity of the upper sub-block, the lower sub-block and the left sub-block and the right sub-block as image characteristic values f3 and f4, wherein the following formula is adopted:

f3＝Comp_A≥Comp_B；f4＝Comp_L≥Comp_R；

therein, Comp_A、Comp_B、Comp_L、Comp_RRespectively representing the complexity of an upper sub block, a lower sub block, a left sub block and a right sub block;

the calculation formula of the complexity is as follows:

wherein, G_max(x, y) represents the maximum gradient of a pixel point (x, y) in the image in the RGB color space,

representing R, G, B points of (x, y) in the image respectivelyGradient value, Pixelnum (G), represents the total number of pixels of the image G, Comp_GRepresents the complexity of image G;

s203, calculating the similarity between every two sub-blocks in the four sub-blocks; and the comparison result of the similarity between the sub-blocks is taken as the image feature values f5, and f6 and f 7; the formula is as follows:

f5＝Sim(A，L)≥Sim(A，R)；f6＝Sim(B，L)≥Sim(B，R)；f7＝Sim(A，B)≥Sim(L，R)；

the similarity calculation formula is as follows:

wherein Sim (G)₁，G₂) Indicating the degree of acquaintance, G, of the images G1 and G2₁，G₂∈RGB，H(i)₁And H (i)₂Are respectively an image G₁And G₂M is the number of cells present in the HOG feature;

s204, detecting the significant straight lines of the four sub-blocks by using Hough transformation, judging whether the straight lines are static lines or dynamic lines according to the inclination angles alpha of the straight lines, calculating the number of the static lines and the dynamic lines and the average length of all the lines as image characteristics, and respectively taking the comparison results of the straight line attribute values between the two sub-blocks as image characteristic values f8, f9, f10, f11, f12 and f13, wherein the formula is as follows:

f8＝Len_S_A≥Len_S_B；f9＝Len_D_A≥Len_D_B；f10＝Ave_Len_A≥Ave_Len_B；

f11＝Len_S_L≥Len_S_R；f12＝Len_D_L≥Len_D_R；f13＝Ave_Len_L≥Ave_Len_R；

wherein Len _ S_A、Len_S_B、Len_S_L、Len_S_RRespectively representing the number of static lines in the upper, lower, left and right sub-blocks, Len _ D_A、Len_D_B、Len_D_L、Len_D_RRespectively represents the number of dynamic lines in the upper sub-block, the lower sub-block, the left sub-block and the right sub-block, Ave_Len_A、Ave_Len_B、Ave_Len_L、Ave_Len_RRespectively representing the average length of all lines in the upper sub-block, the lower sub-block, the left sub-block and the right sub-block;

s3, extracting high-level features of the abstract picture image by adopting a Convolutional Neural Network (CNN), which comprises the following concrete steps:

s301, adjusting four sub-blocks of the abstract picture into a 128 multiplied by 128 RGB color image;

s302, respectively inputting the four subblocks into a convolutional neural network CNN, wherein the convolutional neural network CNN comprises 3 convolutional layers with the step length of 1, 3 maximum pooling layers of 2 multiplied by 2 and 2 full-connection layers, a ReLU is adopted as an activation function in each convolutional layer, the dimensionalities of the two full-connection layers are respectively 1024 and 521, and finally, 512-dimensional vectors are respectively obtained and used as neural network characteristic vectors;

s303, judging the comparison result of the feature vectors of the upper sub-block, the lower sub-block and the left sub-block and the right sub-block as image high-level feature values f14 and f15, wherein the calculation formula is as follows:

f14＝f_cnn_A≥f_cnn_B；f15＝f_cnn_L≥f_cnn_R；

wherein f _ cnn_A、f_cnn_B、f_cnn_L、f_cnn_RRespectively representing the characteristic values of the neural network of the upper sub-block, the lower sub-block, the left sub-block and the right sub-block;

s4, linearly combining the image low-level characteristic value obtained in the step S2 and the image high-level characteristic value obtained in the step S3 to obtain a final characteristic value of the abstract picture image;

and S5, performing the operations of S1-S4 on all abstract pictures in the image library to obtain the final characteristic value of the abstract picture, inputting the final characteristic value into a naive Bayes classifier for training and prediction, and finally dividing the abstract picture into 'upward', 'downward', 'left' or 'right', thereby realizing the automatic prediction of the direction of the abstract picture image.

2. The method for identifying the direction of the abstract picture image based on the feature fusion and naive Bayes as claimed in claim 1, wherein the network structure of the convolutional neural network CNN is as follows: the first convolutional layer consists of 16 convolution kernels of 3 × 3; the second convolutional layer consists of 8 convolution kernels of 3 × 3; the third convolution layer consists of 4 convolution kernels of 3 multiplied by 3, a feature graph obtained after each convolution fills the edge part with 0, and the size is kept unchanged; after each convolutional layer, reducing feature resolution with maximum pooling of 2 × 2; finally, the 4 16 × 16 two-dimensional matrices are converted into 1024-dimensional eigenvectors using the fully-connected layer, and 1024 is reduced to 512 dimensions.

3. The method for identifying the direction of an abstract drawing image based on feature fusion and naive Bayes as claimed in claim 1, wherein in said step S4, the vector dimension of the final feature value of the linearly combined abstract drawing image is 1291.

4. The method for identifying the direction of the abstract drawing image based on the feature fusion and naive Bayes as claimed in claim 1, wherein in said step S5, the concrete method for performing four classifications of "upward", "downward", "leftward" and "rightward" when the naive Bayes classifier predicts the direction of the abstract drawing image is:

these four cases are divided into four groups: one direction is selected as one type in each group, the other three directions are used as the other types, the ratio of the posterior probabilities of the two types in each group is calculated, and the calculation formula is as follows:

wherein,

is the posterior probability ratio of the two classes in each group; p (C)_θIf) represents the posterior probability of the direction selected therein,

representing the posterior probabilities of the remaining three directions; comparing the posterior probability ratios of two of the four groups

Selecting

The direction with the largest value is taken as the correct direction for the abstract picture image.

Images