WO2023015610A1 - Artificial intelligence-based method and system for authenticating ancient and modern artwork - Google Patents

Abstract

A method and system for authenticating ancient and modern artwork that uses artificial intelligence. The method comprises: inputting image information of a genuine work of art and image information of artwork to be authenticated (101); detecting, by means of a detector, in-distribution samples and out-of-distribution samples from the image information of the artwork to be authenticated and the image information of the genuine work of art (102); classifying the in-distribution samples (103); performing fine-grained classification on the classified in-distribution samples and class image information that is similar to the artwork to be authenticated (104); and outputting the classified in-distribution samples or the fine-grained classified samples (105, 105'), and obtaining the confidence of the image information of the artwork to be authenticated compared with the image information of the genuine work of art as the authentication conclusion. Thus, the accuracy of authentication is improved, and the amount of computation during model training is reduced.

Description

Artificial intelligence-based identification method and system for ancient and modern works of art technical field

The invention belongs to the field of identification of ancient and modern works of art, in particular to a method and system for identifying ancient and modern works of art by using artificial intelligence.

Background technique

With the development of ancient and modern art trade, the authenticity of the traded art has always been the core issue in this field. In order to ensure the accuracy of identification conclusions, more and more artificial intelligence technology research results have been applied to this field.

CN107341461A discloses a method and system for identifying the authenticity of artworks with intelligent identification and analysis technology, classifying artworks into dead artworks and living artworks; establishing a database for artworks, storing all information of artworks, and analyzing the artist's The picture system performs intelligent analysis and storage; enters the information of artworks, and the system performs self-learning evolution, and performs target source matching for artworks that need to be identified; if it exists, it compares the authenticity; if it does not exist Then compare the style of the artwork deduced by self-learning, and get the final result of authenticity identification; this solves the current situation that the art identification field cannot be systematized and standardized; through intelligent identification and analysis technology, and based on A self-learning evolution system developed on the basis of the database established by the artwork image information.

CN109191145A discloses a method for establishing a database for judging the age of artworks and a method for judging the age of artworks. The method for establishing a database for judging the age of artworks includes the following steps: (1) selecting the same age, same type, At least two artwork specimens of the same style; (2) extracting the total field of view image; (3) establishing a database; a method for identifying the age of artworks which includes the following steps: Ⅰ. using the database established above; Ⅱ. extracting the art to be determined Ⅲ. Analyze the extracted image on the artwork to be determined and the image saved in the database using image recognition technology; Ⅳ. Judgment.

CN111339974A discloses a method for identifying modern ceramics and ancient ceramics, by constructing positive samples corresponding to ancient ceramics and negative samples corresponding to antique porcelain, converting RGB images to HSV color space to obtain HSV images, and obtaining feature descriptors of HSV images , input the feature descriptor into the support vector machine for training, obtain the training parameters of the support vector machine, input the RGB image into the deep convolutional neural network architecture for training to obtain the network parameters of the convolutional neural network, according to the training parameters of the support vector machine and The network parameters of the convolutional neural network determine the deep learning model, input the grayscale image of the positive sample and the grayscale image of the negative sample into the deep learning model for training, obtain the identification model, obtain the image of the porcelain to be identified, and convert the image to be identified The picture is input into the identification model, and according to the output result of the identification model, it is determined whether the porcelain to be identified is modern ceramics or ancient ceramics, so as to improve the efficiency of corresponding ceramic identification.

It can be seen that the technical means of identifying ancient and modern artworks has experienced the development from database to model training. However, in order to improve the accuracy of identification and reduce the complexity of identification methods, simpler and more accurate model training methods are needed.

Contents of the invention

The object of the present invention is to provide a method and system for appraising ancient and modern artworks by using artificial intelligence.

One aspect of the present invention provides a method for identifying ancient and modern works of art, including: inputting authentic image information; and inputting image information of artworks to be identified; combining the image information of artworks to be identified with the Authentic image information detects the samples in the distribution and the samples out of the distribution through the detector; classifies the samples in the distribution; classifies the samples in the distribution after classification and class image information similar to the artwork to be identified Granular classification: output classified samples in the distribution or samples after fine-grained classification, and obtain the confidence degree of the image information of the artwork to be identified compared with the authentic image information as the identification conclusion.

Another aspect of the identification method of the present invention, wherein the step of detecting the in-distribution samples and out-of-distribution samples through the detector to detect the image information of the artwork to be identified and the image information of the authentic works further includes: using a pre-trained model ( The maximum normalized index probability output by the pre-trained model) is used for statistical analysis; the distribution of the normalized index probability of the OOD sample and the ID sample is found statistically; the distribution gap between the two is increased; an appropriate threshold is selected to judge a sample Whether it is an out-of-distribution sample or a sample in-distribution.

In another aspect of the identification method of the present invention, the step of detecting the samples in the distribution and the samples out of the distribution by using the image information of the artwork to be identified and the image information of the authentic works through a detector further includes: using a model to learn a pair The uncertainty attribute of the input sample; to judge the test data, if the model input is a sample in the distribution, the uncertainty is low; on the contrary, if the model input is an out-of-distribution sample, the uncertainty is high.

Another aspect of the identification method of the present invention, wherein the step of detecting the samples in the distribution and the samples out of the distribution by using the image information of the artwork to be identified and the image information of the authentic works through a detector further includes: using variational automatic coding Variational Autoencoder (Variational Autoencoder) reconstruction error (reconstruction error) or other measurement methods to determine whether a sample belongs to the distribution or out-of-distribution samples; the latent space of the encoder can learn the obviousness of the data in the distribution feature (silent vector), but not for out-of-distribution samples, so out-of-distribution samples will produce higher reconstruction errors.

According to another aspect of the identification method of the present invention, the step of detecting the samples in the distribution and the samples out of the distribution by using the image information of the artwork to be identified and the image information of the authentic works through the detector further includes: using a classifier to extract Classify the features of the distribution to determine whether it is an out-of-distribution sample; some modify the network structure to an n+1 classifier, n is the number of categories of the original classification task, and the n+1th class is an out-of-distribution class; some directly take Extract features for classification without modifying the structure of the network.

In another aspect of the identification method of the present invention, the step of fine-grained classification of the classified samples in the distribution and image-like information similar to the artwork to be identified further includes: finding the image to be tested The feature area of the data; the feature area is input into the convolutional neural network; a part of the information of the feature area of the convolutional neural network enters the fully connected layer and the normalized exponential logistic regression layer for classification; through the volume Another part of the information of the feature region of the product neural network passes through the attention suggestion sub-network (APN) to obtain the candidate region; repeat the above-mentioned classification steps and APN steps, so that the feature region selected by the APN is the most discriminative region; introduce A loss function to obtain higher accuracy in identifying the image information of the class.

In another aspect of the identification method of the present invention, the step of fine-grained classifying the classified samples in the distribution and the image-like information similar to the artwork to be identified further includes: The local images (312) and (313) selected in the information (311) are input into two convolutional neural networks (314, A) and (315, B); the output of the convolutional neural network streams (A) and (B) is in Each location of the image is multiplied (318) using the outer product and combined to obtain a bilinear vector (316); the prediction is obtained through the classification layer (317).

The identification method according to another aspect of the present invention, wherein the classification layer (317) is a logistic regression or support vector machine classifier.

In another aspect of the identification method of the present invention, the step of fine-grained classifying the classified samples in the distribution and the image-like information similar to the artwork to be identified further includes: The information generates multiple candidate boxes on different scale feature maps, and the coordinates of each candidate box correspond to the pre-designed anchors; the "information content" of each candidate area is scored, and the area with a large amount of information has a high score; The above feature map is followed by the feature extraction step, the fully connected layer (FC) and the normalized index step; the probability that the input area belongs to the target label is judged; the unnormalized probability extracted from each local area and the whole map is merged together Generates a long vector outputting the unnormalized probabilities for the 200 classes.

The present invention also provides a system for identifying ancient and modern works of art, including: an input module for inputting authentic image information; and inputting image information for artworks to be identified; The information and the authentic image information are detected by the detector to detect the samples in the distribution and the samples out of the distribution; the sample classification module classifies the samples in the distribution; the fine-grained classification module combines the classified samples in the distribution with the simulated Fine-grained classification of similar image information of the identified works of art; the output module outputs the samples in the distribution after classification or the samples after fine-grained classification, and obtains the image information of the artwork to be identified and the authentic The confidence level compared with the image information is used as the identification conclusion.

Using the above-mentioned method and system for using artificial intelligence to identify ancient and modern artworks, because of the multi-layer classification method for data, the accuracy of identification has been improved, and the amount of computation in the model training process has been reduced. .

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present invention more clearly, the following will briefly introduce the drawings that are used in the embodiments. Apparently, the drawings in the following description are only some examples of the present invention, and those skilled in the art can also obtain other drawings according to these drawings without any innovative work.

Fig. 1 is the flow chart of the ancient and modern works of art appraisal method of the present invention.

Figure 2(a)-(d) is a flow chart of the steps of detecting samples in distribution (in distribution) and out of distribution (out of distribution (OOD)) in the steps of the ancient and modern art identification method of the present invention.

Figure 2(a) is a flowchart of a normalized index based embodiment.

Fig. 2(b) is a flowchart of an embodiment of uncertainty.

Figure 2(c) is a flowchart of an embodiment of a probabilistic generative model.

Figure 2(d) is a flowchart of an embodiment of a classification model.

Fig. 3 (a) is a flow chart of the implementation of the attention convolutional neural network in the fine-grained classification step in the steps of the ancient and modern art identification method of the present invention.

Fig. 3(b) is a schematic diagram of the framework of a recurrent attention convolutional neural network ("RA-CNN") for an implementation of the fine-grained classification step in the identification method of the present invention.

Fig. 3(c) is a schematic diagram of the bilinear vector network structure of another embodiment of the fine-grained classification step in the identification method of the present invention.

Fig. 3(d) is a flow chart of the implementation mode of bilinear vector network in the step of fine-grained classification in the steps of the ancient and modern art identification method of the present invention.

Fig. 3(e) is a flow chart of an embodiment in which the fine-grained classification step adopts the navigation-teaching-examination network (NTS-Net) classification in the steps of the ancient and modern artwork appraisal method of the present invention.

Fig. 4 is a structural diagram of the ancient and modern art identification system of the present invention.

Fig. 5 is a computer product diagram of the portable or fixed storage unit of the ancient and modern art identification system of the present invention.

Fig. 6(1) is an example of authentic image information involved in an embodiment of the ancient and modern artwork identification method of the present invention.

Fig. 6(2) is an example of authentic image information used to train the model involved in an embodiment of the ancient and modern artwork identification method of the present invention.

Fig. 6(3) is an example of the image information of the artwork to be authenticated involved in one embodiment of the ancient and modern artwork authentication method of the present invention.

Fig. 6 (4) is an example of the classification involved in an implementation of the ancient and modern artwork identification method of the present invention.

Detailed ways

Specific embodiments of the present invention will now be described in conjunction with the corresponding drawings. However, this invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. These embodiments are provided only so that the present invention will be thorough and complete so that those skilled in the art can fully describe the scope of the present invention. Wording used in the detailed description of the embodiments illustrated in the drawings should not limit the invention.

Fig. 1 is the flow chart of the ancient and modern works of art appraisal method of the present invention. In step 101, the image information and authentic image information of the artwork to be identified are input; in step 102, the image information of the artwork to be identified and the authentic image information are detected by a detector in distribution ) samples and distribution (out of distribution, OOD) samples; in step 103, the samples in the distribution are classified; in step 104, the samples in the distribution after classification are similar to the artwork to be identified Fine-grained classifier is performed on the class image information; in step 105, the classified samples in the distribution or samples after fine-grained classification are output to obtain the identification conclusion.

Figure 2(a)-(d) is a flow chart of the steps of detecting samples in distribution (in distribution) and out of distribution (out of distribution (OOD)) in the steps of the ancient and modern art identification method of the present invention. Wherein Fig. 2 (a) is the flow chart of the embodiment based on normalized index; Fig. 2 (b) is the flow chart of the embodiment of uncertainty; Fig. 2 (c) is the flow chart of the embodiment of probability generation model ; Figure 2(d) is a flowchart of an embodiment of a classification model.

When the image-like data for model training and testing are independent and identically distributed (IID, Independent Identical Distribution), the image-like data for training and testing are called In Distribution (ID) samples. In addition to ID samples, in practical applications, the data obtained after the model is deployed and launched is often not fully controlled, that is to say, the data received by the model may be out-of-distribution (OOD) samples, also known as outlier samples (outlier, abnormal). The depth model will consider an out-of-distribution (OOD) sample as a certain class in the distribution (ID) sample, and give a high degree of confidence. The confidence degree described here is a normalized value of 0-1. Find out-of-distribution samples, but their settings may be different. For example, out-of-distribution detection (OOD detection) is modified on the model task, which requires not only to be able to effectively detect out-of-distribution (OOD) samples, but also to ensure that the performance of the model is not affected.

In the present invention, the detection distribution and out-of-distribution steps for the image-like data of ancient and modern artworks can be based on normalization index (Softmax-based), uncertainty (Uncertainty), probability generation model (Generative model), classification model (Classifier) method to detect in-distribution and out-of-distribution sample methods.

Wherein, in the embodiment based on the normalized index method, in step 201, statistical analysis is performed using the maximum normalized index probability output by the pre-trained model (pre-trained model), and in step 202, statistically found OOD samples and For the distribution of the normalized index probability of the ID sample, in step 203, the distribution gap between the two is increased, and in step 204, an appropriate threshold is selected to determine whether a sample belongs to an out-of-distribution sample or an in-distribution sample. This type of method is simple and effective, without modifying the structure of the classification model, and without training an out-of-distribution sample classifier.

In the embodiment using the uncertainty method, since the probability output of the model cannot directly represent the confidence of the model. In step 211, the model is used to learn an uncertainty attribute for the input samples. In step 212, the test data is judged. If the model input is a sample in the distribution, the uncertainty is low; on the contrary, if the model input is an out-of-distribution sample, the uncertainty is high. Such methods need to modify the network structure of the model to learn the uncertainty property.

In the embodiment of using the probability generation model method, in step 221, use the reconstruction error (reconstruction error) of the variational autoencoder (Variational Autoencoder) or other measurement methods to judge whether a sample belongs to the sample in the distribution or out of the distribution; In step 222, the hidden space (latent space) of the encoder can learn the obvious features (silent vector) of the data in the distribution, but not for the out-of-distribution samples, so the out-of-distribution samples will generate higher reconstruction errors . This method only focuses on out-of-distribution detection performance, and does not focus on the original task of the data in the distribution.

In the implementation of the classification model method, in step 231, a classifier is used to classify the extracted features to determine whether it is an out-of-distribution sample; in step 232, some modify the network structure to be an n+1 classifier, n is the number of categories of the original classification task, and the n+1th category is an out-of-distribution category; in step 233, some features are directly extracted for classification without modifying the structure of the network.

Fig. 3 (a) is a flow chart of the implementation of the attention convolutional neural network in the fine-grained classification step in the steps of the ancient and modern art identification method of the present invention. In step 321, the characteristic area of the image data to be tested is searched, and in step 322, the characteristic area is input into the convolutional neural network; in step 323, a part of the information of the characteristic area through the convolutional neural network is entered Fully connected layer and normalized exponential logistic regression layer are classified; in step 324, another part of the information of the feature region through the convolutional neural network is passed through the attention suggestion subnetwork (APN), to obtain the candidate region; in step 325, repeating the step 323 and the step 324, so that the feature region selected by the APN is the most discriminative region; in step 326, introducing a loss function to obtain a higher accuracy of identifying the type of image information.

The "fine-grained" classification step is under the ordinary classification. For a more fine-grained division, it is necessary to explicitly find the most "discriminative" features in the picture. For ancient and modern works of art, it is necessary to find the characteristics of details, such as the degree of upturning of petals, the nuances of patterns, etc.

Fig. 3(b) is a schematic diagram of the framework of a recurrent attention convolutional neural network ("RA-CNN") for an implementation of the fine-grained classification step in the identification method of the present invention. where the symbol

It means to cut a part of the characteristic area of the identified image-like information and enlarge it. Each row 301, 302, 303 represents a common CNN network respectively. As shown in Figure 3(b), the input ranges from coarse full-scale images to finer region attention (from top to bottom). The picture (a ₁ ) in the first row 301 is the roughest, and the picture (a ₃ ) in the third row is finer. After the image information a ₁ enters b ₁ (several convolutional layers), it is divided into two paths, all the way to c ₁ and connected to fully connected layers (fully connected layers, FC) and softmax logistic regression layer for simple classification, and the other path enters d ₁ is the attention proposal sub-network ("Attention Proposal Network", APN), get a candidate area. Using the candidate area proposed by d ₁ on the original image, a more discriminative small area is cropped on the original image, and a ₂ is obtained after interpolation, and a ₃ is obtained in the same way.

The feature area is continuously enlarged and refined after two APNs. In order to make the feature area selected by APN the most discriminative area in the image, a loss function (Ranking loss) is introduced: that is, the forced area a ₁ , a ₂ , a ₃ The classification confidence level (confidence score) is getting higher and higher (that is, the corresponding P _t probability of the last column of the picture is getting higher and higher), which means that the accuracy of identifying image information is getting higher and higher. In this way, combined with ordinary classification loss, the network continuously refines the discriminative attention region.

Fig. 3(c) is a schematic diagram of the bilinear vector network structure of another embodiment of the fine-grained classification step in the identification method of the present invention.

Partial images 312 and 313 selected from the identified class image information 311 are input into two convolutional neural networks 314(A) and 315(B). The outputs of the convolutional neural network streams A and B are multiplied 318 by outer product at each position of the image and combined to obtain a bilinear vector 316 , which is then passed through a classification layer 317 to obtain a prediction result. The bilinear model M consists of a quadruple: M = (f _A , f _B ; P; C). Among them, f _A and f _B represent feature extraction functions, that is, convolutional network A and convolutional network B in Figure 3(c), P is a pooling function (Pooling function), and C is a classification function.

The feature extraction function f( ) (i.e., the convolutional neural network stream CNN stream) consists of convolutional layers, pooling layers, and activation functions. This part of the network structure can be regarded as a function map:

f: L×I→RK×D (1)

Map the input identified class image information and location area into a one-dimensional feature, where K is the number of channels of the convolutional network output feature map, and D is the number of one-dimensional feature vectors expanded from the two-dimensional feature map in each channel. size. The convolution features output by the two feature extraction functions are converged through bilinear operations to obtain bilinear features 316: bilinear(l; T; f _A , f _B ) = f _A (L; T) ^T f _B (L;T) ^T . The function of the pooling function P is to aggregate the bilinear features of all positions into one feature. The pooling function adopted is to accumulate the bilinear features of all positions to obtain the global feature representation of the image Φ'(I):

Φ`(I)=Σ _l∈L bilinear(l; T, f _A ; f _B )=Σ _l∈L f _A (l; I) ^T f _B (l; I) (2)

In the case that the feature dimensions extracted by the two feature functions f _A and f _B are K×M and K×N respectively, the output of the pooling function P is an M×N matrix. Before classifying it, the feature matrix Stretched into a list of MN-sized feature vectors. Finally, a classification function is used to classify the extracted features, and the classification layer 317 is implemented using a logistic regression or a support vector machine (support vector machine, SVM) classifier.

The CNN network can achieve high-level semantic feature acquisition for fine-grained images, and filter irrelevant background information in the image by iteratively training the convolution parameters in the network model. On the other hand, the convolutional neural network flow A and the convolutional neural network flow B play complementary roles in the image recognition task, that is, the network A can locate the object in the image, and the network B can complete the positioning of the network A to Feature extraction of the object position. In this way, the two networks can cooperate to complete the class detection and target feature removal process of the input fine-grained image, and better complete the fine-grained image recognition task.

Fig. 3(d) is a flow chart of the implementation mode of bilinear vector network in the step of fine-grained classification in the steps of the ancient and modern art identification method of the present invention. In step 331, the partial images 312 and 313 selected in the identified class image information 311 are input into two convolutional neural networks 314 (A) and 315 (B); in step 332, the convolutional neural network streams A and B The output is multiplied 318 using an outer product at each location in the image and combined to obtain a bilinear vector 316 , and at step 333 the prediction is obtained through the classification layer 317 .

Fig. 3(e) is a flow chart of an embodiment in which the fine-grained classification step adopts the navigation-teaching-examination network (NTS-Net) classification in the steps of the ancient and modern artwork appraisal method of the present invention. In step 341, generate a plurality of candidate boxes on different scale feature maps (Feature maps) with the identified class image information, and the coordinates of each candidate box correspond to pre-designed anchors (Anchors); in step 342, give The "information content" of each candidate area is scored, and the area with a large amount of information has a high score; in step 343, a feature extraction step (Feature Extractor), a fully connected layer (FC) and a normalization index ( softmax) step; in step 344, determine the probability that the input region belongs to the target label (target label); in step 345, merge (concat) together the unnormalized probability (logits) extracted from each local area and the whole picture Generates a long vector outputting unnormalized probabilities (logits) corresponding to 200 categories.

The fine-grained classification step in the identification method of the present invention can also be adopted, and the navigation-teaching-examination network (NTS-Net) classification method of dividing the network subject into three components of navigation (Navigator), teaching (Teacher), and examination (Scrutinizer), In the navigation step, multiple candidate boxes are generated on feature maps of different scales, and the coordinates of each candidate box correspond to the pre-designed anchors (Anchors). The Navigator scores the "information content" of each candidate area, and the area with a large amount of information has a higher score. The teaching step is the commonly used feature extraction step (Feature Extractor), fully connected layer (FC) and normalized index (softmax) step, to judge the probability that the input area belongs to the target label (target label); the review step is a fully connected layer, The input is to combine (concat) the unnormalized probability (logits) extracted from each local area and the whole image together to generate a long vector, and output the unnormalized probability (logits) corresponding to 200 categories.

The specific steps of using this NTS method are: 1) The original image of size (448, 448, 3) enters the network, and after entering the Resnet-50 to extract features, it becomes a (14, 14, 2048) feature map, a A 2048-dimensional feature vector after the global pooling layer and a 200-dimensional unnormalized probability after the global pooling layer and the fully connected layer. 2) The preset network (RPN) for generating candidate regions generates correspondences according to different sizes (Size) and aspect ratios on the three scales of (14, 14) (7, 7) (4, 4) There are 1614 anchors in total. 3) Use the feature map in step 1 to score in the navigation, and use Non-Maximum Suppression (NMS) to retain only N local candidate boxes with the most information according to the scoring results. 4) Bilinearly interpolate the N local regions to (224, 224), input them into the teacher network, and obtain the feature vectors and unnormalized probabilities (logits) of these local regions. 5) Merge (concat) the full-image feature vector feature vector and local feature vector in steps 1 and 4, and then connect to the FC layer to obtain the joint classification logits for final decision-making.

Fig. 4 is a structural diagram of the ancient and modern art identification system of the present invention. For example, the server 401 of the ancient and modern artwork appraisal system. The server of this ancient and modern art identification system includes a processor 410, where the processor can be a general-purpose or special-purpose chip (ASIC/eASIC) or FPGA or NPU, etc., and a computer program product in the form of a memory 420 or a computer-programmable Read media. Memory 420 may be electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM. The memory 420 has a storage space 430 for program codes for performing any method steps in the methods described above. For example, the storage space 430 for program codes may include respective program codes 431 for respectively implementing various steps in the above methods. These program codes can be read or written into the processor 410 . These computer program products comprise program code carriers such as hard disks, compact disks (CDs), memory cards or floppy disks. Such a computer program product is typically a portable or fixed storage unit as described with reference to FIG. 5 . Fig. 5 is a computer product diagram of the portable or fixed storage unit of the ancient and modern art identification system of the present invention. The storage unit may have storage segments, storage spaces, etc. arranged similarly to the memory 420 in the server of FIG. 4 . The program code can eg be compressed in a suitable form. Typically, the storage unit includes computer readable code 431', i.e. code readable by, for example, a processor such as 410, which code, when executed by the server, causes the server to perform the various steps in the methods described above. These codes, when executed by the server, cause the server to perform the steps of the methods described above.

Fig. 6(1) is an example of authentic image information involved in an embodiment of the ancient and modern artwork identification method of the present invention. Figure 6(2) is an example of authentic image information used to train the model. Figure 6(3) is an example of the image information of the artwork to be identified. Figure 6(4) is an example of classification. Among them, Fig. 6(1) shows an example of a certain image information of an authentic artwork. Taking the multiple image information obtained from 360 degrees of the authentic artwork given in Fig. 6 (2) as a standard, Fig. 6 (3 ) in the artwork image information to be identified, that is, image-like information containing different characteristic regions, to identify the artwork model, obtain a confidence level of 0 to 1, and evaluate the similarity between the artwork to be identified and the authentic work, the more The closer to 1, the more similar, when the result is 1, the two are consistent. Figure 6(4) shows the classification examples of different confidence values based on different artwork image information to be identified.

Reference herein to "one embodiment," "an embodiment," or "one or more embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Additionally, please note that examples of the word "in one embodiment" herein do not necessarily all refer to the same embodiment.

The above description is only used to illustrate the technical solutions of the present invention, and any person skilled in the art can modify and change the above embodiments without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention should be determined by the scope of the claims. The present invention has been described above with reference to examples. However, other embodiments than those described above are equally possible within the disclosed scope of the present invention. The different features and steps of the invention may be combined in other ways than described. The scope of the present invention is limited only by the appended claims. More generally, one of ordinary skill in the art can readily understand that all parameters, dimensions, materials and configurations described herein are for exemplary purposes and actual parameters, dimensions, materials and/or configurations will depend on the particular application or invention Teach the application for which it is used.

Claims (18)

1. A method for identifying ancient and modern works of art, comprising:

   Input the image information of the authentic work; and input the image information of the artwork to be identified;

   Detecting the in-distribution sample and the out-of-distribution sample through the detector to detect the image information of the artwork to be identified and the image information of the authentic work;

   Classify the samples in the distribution;

   Carrying out fine-grained classification of the classified samples in the distribution and image-like information similar to the artwork to be identified;

   Output the samples in the classified distribution or the samples after fine-grained classification, and obtain the confidence degree of the image information of the artwork to be identified compared with the authentic image information as the identification conclusion.
2. The identification method according to claim 1, wherein the step of detecting the in-distribution sample and the out-of-distribution sample through the detector to detect the image information of the artwork to be identified and the authentic image information further comprises:

   Statistical analysis using the maximum normalized exponential probability output by the pre-trained model;

   Statistically discover the distribution of normalized index probability of OOD samples and ID samples;

   Increase the distribution gap between the two;

   Select an appropriate threshold to judge whether a sample belongs to an out-of-distribution sample or an in-distribution sample.
3. The identification method according to claim 1, wherein the step of detecting the in-distribution sample and the out-of-distribution sample through the detector to detect the image information of the artwork to be identified and the authentic image information further comprises:

   Use the model to learn an uncertainty attribute for the input sample;

   Judging the test data, if the model input is a sample in the distribution, the uncertainty is low. On the contrary, if the model input is an out-of-distribution sample, the uncertainty is high.
4. The identification method according to claim 1, wherein the step of detecting the in-distribution sample and the out-of-distribution sample through the detector to detect the image information of the artwork to be identified and the authentic image information further comprises:

   Use the reconstruction error of the Variational Autoencoder (Variational Autoencoder) or other metrics to determine whether a sample belongs to a sample in the distribution or out of the distribution;

   The latent space (latent space) of the encoder can learn the obvious features (silent vector) of the data in the distribution, but not for the out-of-distribution samples, so the out-of-distribution samples will generate higher reconstruction errors.
5. The identification method according to claim 1, wherein the step of detecting the in-distribution sample and the out-of-distribution sample through the detector to detect the image information of the artwork to be identified and the authentic image information further comprises:

   Use a classifier to classify the extracted features to determine whether it is an out-of-distribution sample;

   Some modify the network structure into an n+1 classifier, n is the number of categories of the original classification task, and the n+1th class is an out-of-distribution class;

   Some directly take the extracted features for classification without modifying the structure of the network.
6. The identification method according to any one of claims 1-5, wherein the step of fine-grained classification of the classified samples in the distribution and image-like information similar to the artwork to be identified further includes:

   Find the characteristic area of the image data to be tested;

   inputting the feature region into a convolutional neural network;

   Part of the information of the feature region through the convolutional neural network enters the fully connected layer and the normalized exponential logistic regression layer for classification;

   Another part of the information of the feature region through the convolutional neural network passes through the attention suggestion sub-network (APN) to obtain a candidate region;

   Repeat the above classification steps and APN steps, so that the feature region selected by APN is the most discriminative region;

   A loss function is introduced to obtain higher accuracy in identifying the image information.
7. The identification method according to any one of claims 1-5, wherein the step of fine-grained classification of the classified samples in the distribution and image-like information similar to the artwork to be identified further includes:

   Input two convolutional neural networks (314, A) and (315, B) into partial images (312) and (313) selected in the identified class image information (311);

   The outputs of the convolutional neural network streams (A) and (B) are multiplied (318) using an outer product at each location in the image and combined to obtain a bilinear vector (316);

   Predictions are obtained through the classification layer (317).
8. The authentication method according to claim 7, wherein the classification layer (317) is a logistic regression or a support vector machine classifier.
9. The identification method according to any one of claims 1-5, wherein the step of fine-grained classification of the classified samples in the distribution and image-like information similar to the artwork to be identified further includes:

   Generate multiple candidate boxes on feature maps of different scales from the identified class image information, and the coordinates of each candidate box correspond to the pre-designed anchors;

   Score the "information content" of each candidate area, and the area with a large amount of information has a high score;

   Implement feature extraction step, fully connected layer (FC) and normalized index step to described feature map successively;

   Determine the probability that the input area belongs to the target label;

   The unnormalized probabilities extracted from each local area and the whole image are combined to generate a long vector, and the unnormalized probabilities corresponding to 200 categories are output.
10. An ancient and modern art identification system, including:

    Input module, input authentic image information; and input the image information of the artwork to be identified;

    The detection module detects the samples in the distribution and the samples out of the distribution by using the detector to detect the image information of the artwork to be identified and the image information of the authentic work;

    A sample classification module, classifying the samples in the distribution;

    The fine-grained classification module performs fine-grained classification on the classified samples in the distribution and similar image information similar to the artwork to be identified;

    The output module outputs the classified samples in the distribution or the fine-grained classified samples, and obtains the confidence degree of the image information of the artwork to be identified compared with the authentic image information as the identification conclusion.
11. The identification system according to claim 10, wherein said detection module further comprises:

    The analysis module utilizes the maximum normalized index probability output by the pre-trained model (pre-trained model) to carry out statistical analysis;

    Statistically discover the distribution of normalized index probability of OOD samples and ID samples;

    Increase the distribution gap between the two;

    Select the module and select an appropriate threshold to judge whether a sample belongs to an out-of-distribution sample or an in-distribution sample.
12. The identification system according to claim 10, wherein said detection module further comprises:

    The learning module uses the model to learn an uncertainty attribute of the input sample;

    The judgment module judges the test data. If the model input is a sample in the distribution, the uncertainty is low. On the contrary, if the model input is an out-of-distribution sample, the uncertainty is high.
13. The identification system according to claim 10, wherein said detection module further comprises:

    The judging module uses the reconstruction error (reconstruction error) of the variational autoencoder (Variational Autoencoder) or other measurement methods to judge whether a sample belongs to the sample in the distribution or out of the distribution;

    The latent space of the encoder can learn the obvious features (silent vector) of the data in the distribution, but not for the out-of-distribution samples, so the out-of-distribution samples will generate higher reconstruction errors.
14. The identification system according to claim 10, wherein said detection module further comprises:

    Use a classifier to classify the extracted features to determine whether it is an out-of-distribution sample;

    Some modify the network structure into an n+1 classifier, n is the number of categories of the original classification task, and the n+1th class is an out-of-distribution class;

    Some directly take the extracted features for classification without modifying the structure of the network.
15. The identification system according to any one of claims 10-14, wherein the fine-grained classification module further includes:

    The feature finding module is used to find the feature area of the image data to be tested;

    A feature training module, which inputs the feature region into a convolutional neural network;

    Part of the information classification module, through a part of the information of the feature area of the convolutional neural network, enters the fully connected layer and the normalized exponential logistic regression layer for classification;

    Candidate region obtaining module, another part of information of the feature region through the convolutional neural network passes through the attention suggestion sub-network (APN), to obtain the candidate region;

    Repeat the above classification steps and APN steps, so that the feature region selected by APN is the most discriminative region;

    The identification module introduces a loss function to obtain higher accuracy in identifying the type of image information.
16. The identification system according to any one of claims 10-14, wherein the fine-grained classification module further includes:

    Partial image training module, input two convolutional neural networks (314, A) and (315, B) into the partial images (312) and (313) selected in the class image information (311) to be identified;

    The outputs of the convolutional neural network streams (A) and (B) are multiplied (318) using an outer product at each location in the image and combined to obtain a bilinear vector (316);

    The prediction module obtains the prediction result through the classification layer (317).
17. The authentication system according to claim 16, wherein said classification layer (317) is a logistic regression or support vector machine classifier.
18. The identification system according to any one of claims 10-14, wherein the fine-grained classification module further includes:

    Candidate box anchoring module generates multiple candidate boxes on feature maps of different scales from the identified class image information, and the coordinates of each candidate box correspond to the pre-designed anchors;

    The scoring module scores the "information volume" of each candidate area, and the area with a large amount of information has a high score;

    The feature map processing module implements feature extraction, fully connected layer (FC) and normalized index to the feature map in sequence;

    The probability judgment module judges the probability that the input area belongs to the target label;

    The probability merge output module combines the unnormalized probabilities extracted from each local area and the whole image to generate a long vector, and outputs the unnormalized probabilities corresponding to 200 categories.

Images