CN115705688A

CN115705688A - Ancient and modern artwork identification method and system based on artificial intelligence

Info

Publication number: CN115705688A
Application number: CN202110916187.XA
Authority: CN
Inventors: 李應樵; 马志雄
Original assignee: Marvel Digital Ai Ltd
Current assignee: Marvel Digital Ai Ltd
Priority date: 2021-08-10
Filing date: 2021-08-10
Publication date: 2023-02-17
Also published as: WO2023015610A1

Abstract

The invention relates to a method and a system for identifying ancient and modern artworks by artificial intelligence. The method comprises the following steps: inputting authentic work image information; inputting image information of the artwork to be authenticated; detecting a sample in distribution and a sample out of distribution by the detector according to the artwork image information and the authentic work image information to be authenticated; classifying samples in the distribution; classifying the classified samples in the distribution and class image information similar to the artwork to be identified in a fine-grained manner; and outputting the samples in the classified distribution or the samples after fine-grained classification, and obtaining the confidence coefficient of the image information of the artwork to be identified compared with the authentic image information as an identification conclusion. The accuracy of identification by adopting the method is improved, and the operand in the model training process is reduced.

Description

Ancient and modern artwork identification method and system based on artificial intelligence

Technical Field

The invention belongs to the field of ancient and near modern artwork identification, and particularly relates to a method and a system for identifying ancient and near modern artwork by using artificial intelligence.

Background

With the development of ancient and modern artworks trading, the authenticity judgment of traded artworks is always the core problem of discussion in the field. To ensure the accuracy of the identification conclusion, more and more artificial intelligence technical research efforts are applied to this field.

CN107341461A discloses a method and a system for identifying the authenticity of artworks by intelligent identification and analysis technology, which classifies the artworks into the ancient artworks and the ancient artworks; establishing a database of the artwork, storing all information of the artwork, and intelligently analyzing and storing a picture system of the artist; inputting the information of the artwork, and performing self-learning evolution by the system to the artwork needing to be identified; carrying out target source matching on the artwork to be identified, and if the artwork exists, carrying out true-false comparison; if not, comparing the styles of the works of art deduced according to self-learning to obtain the final result of true and false identification; the current situation that the existing art identification field cannot be systematized and standardized is solved; the self-learning evolution system is developed on the basis of an intelligent identification and analysis technology and a database established on the basis of artwork image information.

CN109191145A discloses a database establishment method for dating of artworks and an artworks dating method, the database establishment method for dating of artworks comprises the following steps: (1) Selecting at least two artwork specimens of the same age, the same type and the same style; (2) extracting a total field of view image; (3) establishing a database; the method for judging the age of the artwork comprises the following steps: I. using the established database; II, extracting an image on the artwork to be determined; analyzing the extracted image on the artwork to be determined and the image stored in the database by using an image recognition technology; and IV, judging.

CN111339974A discloses a method for identifying modern ceramics and ancient ceramics, which comprises the steps of constructing a positive sample corresponding to the ancient ceramics and a negative sample corresponding to the ancient ceramics, converting an RGB image into an HSV color space to obtain an HSV image, obtaining a feature descriptor of the HSV image, inputting the feature descriptor into a support vector machine for training to obtain training parameters of the support vector machine, inputting the RGB image into a deep convolutional neural network architecture for training to obtain network parameters of a convolutional neural network, determining a deep learning model according to the training parameters of the support vector machine and the network parameters of the convolutional neural network, respectively inputting a gray scale map of the positive sample and a gray scale map of the negative sample into the deep learning model for training to obtain an identification model, obtaining a to-be-identified picture of the to-be-identified porcelain, inputting the to-be-identified picture into the identification model, and determining the to-be-identified porcelain or the ancient ceramics according to an output result of the identification model so as to improve the efficiency of corresponding ceramic identification.

It can be seen that the technological means of identifying ancient and modern works of art has undergone a progression from databases to model training. However, in order to improve the accuracy of identification and reduce the complexity of the identification method, a simpler and more accurate model training method is required.

Disclosure of Invention

The invention aims to provide a method and a system for identifying ancient and modern artworks by using artificial intelligence.

One aspect of the invention provides a method for identifying ancient and modern artworks, which comprises the following steps: inputting authentic work image information; inputting image information of the artwork to be authenticated; detecting a sample in distribution and a sample out of distribution by a detector according to the artwork image information and the authentic work image information to be identified; classifying samples in the distribution; classifying the classified samples in the distribution and class image information similar to the artwork to be identified in a fine-grained manner; and outputting the samples in the classified distribution or the samples after fine-grained classification, and obtaining the confidence coefficient of the image information of the artwork to be identified compared with the authentic image information as an identification conclusion.

The method for authenticating according to another aspect of the present invention, wherein the step of detecting the in-distribution sample and the out-of-distribution sample by the detector using the image information of the work of art and the image information of the authentic work to be authenticated further comprises: performing statistical analysis by using the maximum normalized exponential probability output by a pre-trained model (pre-trained model); counting the distribution condition of the normalized exponential probability of the OOD sample and the ID sample; the distribution difference between the two is enlarged; and selecting a proper threshold value to judge whether one sample belongs to the sample outside the distribution or the sample in the distribution.

The authentication method of still another aspect of the present invention, wherein the step of detecting the in-distribution sample and the out-of-distribution sample by the detector using the image information of the work of art to be authenticated and the image information of the authentic work, further comprises: learning an uncertainty attribute for the input sample using the model; and judging the test data, wherein if the model input is a sample in distribution, the uncertainty is low, and if the model input is a sample out of distribution, the uncertainty is high.

The authentication method of still another aspect of the present invention, wherein the step of detecting the in-distribution sample and the out-of-distribution sample by the detector using the image information of the work of art to be authenticated and the image information of the authentic work, further comprises: judging whether a sample belongs to a sample in distribution or a sample out of distribution by utilizing a reconstruction error (reconstruction error) of a Variational automatic encoder (variable automatic encoder) or other measurement modes; the implicit space (late space) of the encoder can learn the obvious features (silence vector) of the data in the distribution, but not for the samples outside the distribution, so the samples outside the distribution can generate higher reconstruction errors.

The authentication method of still another aspect of the present invention, wherein the step of detecting the in-distribution sample and the out-of-distribution sample by the detector using the image information of the work of art to be authenticated and the image information of the authentic work, further comprises: classifying the extracted features by using a classifier to judge whether the extracted features are out-of-distribution samples; some modified network structures are n +1 classes of classifiers, n is the number of classes of the original classification task, and the n +1 th class is an outer class; some directly take and extract the characteristic to classify, do not need to modify the structure of the network.

The identification method of a further aspect of the present invention, wherein the step of classifying the classified samples in the distribution and the class image information similar to the artwork to be identified, at a fine granularity, further comprises: searching a characteristic area of the image data to be tested; inputting the characteristic region into a convolutional neural network; entering a full connection layer and a normalized index logistic regression layer for classification by part of information of the characteristic region of the convolutional neural network; passing another part of information of the characteristic region of the convolutional neural network through an attention suggestion sub-network (APN) to obtain a candidate region; repeating the classification step and the APN step to enable the feature region selected through the APN to be the region with the most discriminability; and a loss function is introduced, so that higher accuracy of identifying the class of image information is obtained.

The method for identifying in a further aspect of the present invention, wherein the step of classifying the classified samples in the distribution and the class image information similar to the art to be identified, with fine granularity, further comprises: inputting selected partial images (312) and (313) of the identified class image information (311) into two convolutional neural networks (314, A) and (315, B); the outputs of the convolutional neural network streams (a) and (B) are multiplied (318) at each position of the image using an outer product and combined to obtain a bilinear vector (316); the prediction result is obtained by the classification layer (317).

The authentication method of yet another aspect of the present invention, wherein said classification layer (317) is a logistic regression or support vector machine classifier.

The identification method of a further aspect of the present invention, wherein the step of classifying the classified samples in the distribution and the class image information similar to the artwork to be identified, at a fine granularity, further comprises: generating a plurality of candidate frames on the identified class image information on different scale characteristic graphs, wherein the coordinate of each candidate frame corresponds to a pre-designed anchor; the information amount of each candidate area is scored, and the area with large information amount is high in score; sequentially performing a feature extraction step, a full connection layer (FC) step and a normalization index step on the feature map; judging the probability that the input area belongs to the target label; combining the non-normalized probabilities extracted from each local area and the whole graph to generate a long vector, and outputting the non-normalized probabilities corresponding to 200 categories.

The invention also provides an ancient and modern artwork identification system, which comprises: the input module is used for inputting the authentic work image information; inputting image information of an artwork to be authenticated; the detection module is used for detecting a sample in distribution and a sample outside the distribution by the detector according to the artwork image information and the authentic work image information to be identified; the sample classification module is used for classifying the samples in the distribution; the fine-grained classification module is used for performing fine-grained classification on the classified samples in the distribution and class image information similar to the artwork to be identified; and the output module is used for outputting the classified samples in the distribution or the classified samples with fine granularity, and obtaining the confidence coefficient of the image information of the artwork to be identified compared with the authentic work image information as an identification conclusion.

By adopting the method and the system for identifying the ancient and modern artworks by applying the artificial intelligence, the accuracy of identification is improved and the operand in the model training process is reduced because a method of carrying out multi-layer classification on data is adopted.

Drawings

In order to more clearly illustrate the technical solution in the embodiments of the present invention, the drawings required to be used in the embodiments will be briefly described below. It is obvious that the drawings in the following description are only examples of the invention, and that for a person skilled in the art, other drawings can be derived from them without making an inventive step.

FIG. 1 is a flow chart of the method of identifying ancient and modern art works of the present invention.

FIGS. 2 (a) - (d) are flow charts of the steps of detecting both in-distribution and out-of-distribution (OOD) samples in the ancient and near-modern art identification methods of the present invention.

Fig. 2 (a) is a flow chart of an embodiment based on normalized indices.

Fig. 2 (b) is a flow chart of an embodiment of uncertainty.

Fig. 2 (c) is a flowchart of an embodiment of a probability generation model.

FIG. 2 (d) is a flow diagram of an embodiment of a classification model.

FIG. 3 (a) is a flow chart of an embodiment of the fine-grained classification step of the ancient and modern art identification method of the present invention using an attention-convolutional neural network.

Fig. 3 (b) is a schematic diagram of a framework of a cyclic attention convolutional neural network ("RA-CNN") according to an embodiment of the fine-grained classification step in the identification method of the present invention.

Fig. 3 (c) is a schematic diagram of a bilinear vector network structure of another embodiment of the fine-grained classification step in the identification method of the present invention.

FIG. 3 (d) is a flow chart of an embodiment of the fine granularity classification step in the ancient and modern artwork identification method of the present invention using bilinear vector network.

Fig. 3 (e) is a flow chart of an embodiment of the fine grained classification step in the ancient and modern art identification method steps of the present invention using navigation-teaching-review network (NTS-Net) classification.

FIG. 4 is a block diagram of the ancient and near modern art identification system of the present invention.

Fig. 5 is a computer product diagram of a portable or fixed storage unit of the ancient and recent modern art identification system of the present invention.

Fig. 6 (1) is an example of authentic work image information involved in one embodiment of the ancient and recent modern art work identification method of the present invention.

Fig. 6 (2) is an example of image information of a genuine product used for training a model, which is involved in one embodiment of the ancient and recent modern art identification method of the present invention.

Fig. 6 (3) is an example of image information of an art to be authenticated, which is involved in one embodiment of the ancient and recent modern art authentication method of the present invention.

Fig. 6 (4) is an example of classification involved in one embodiment of the ancient and recent modern art identification method of the present invention.

Detailed Description

Specific embodiments of the present invention will now be described with reference to the accompanying drawings. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. These embodiments are provided only for the purpose of exhaustive and comprehensive description of the invention so that those skilled in the art can fully describe the scope of the invention. The terminology used in the detailed description of the embodiments illustrated in the accompanying drawings is not intended to be limiting of the invention.

FIG. 1 is a flow chart of the method of identifying ancient and modern art works of the present invention. Inputting image information and authentic image information of an artwork to be authenticated in step 101; detecting an in-distribution (in distribution) sample and an out-of-distribution (OOD) sample of the artwork image information and the authentic image information to be authenticated by a detector in step 102; in step 103, classifying the samples in the distribution; in step 104, performing Fine-grained classification (Fine-grained classification) on the classified samples in the distribution and class image information similar to the artwork to be identified; in step 105, the classified samples in the distribution or the classified samples with fine granularity are output to obtain an identification conclusion.

FIGS. 2 (a) - (d) are flow charts of the steps of detecting both in-distribution and out-of-distribution (OOD) samples in the ancient and near-modern art identification methods of the present invention. Wherein FIG. 2 (a) is a flow chart of an embodiment based on normalized indices; FIG. 2 (b) is a flow chart of an embodiment of uncertainty; FIG. 2 (c) is a flow diagram of an embodiment of a probabilistic generative model; FIG. 2 (d) is a flow diagram of an embodiment of a classification model.

In the case where the class image data of model training and testing is Independent and Identically Distributed (IID), the class image data of training and testing is referred to as In Distribution (ID) sample. In addition to the ID samples, in practical applications, data obtained after the model is deployed online often cannot be completely controlled, that is, data received by the model may be out-of-distribution (OOD) samples, which are also called abnormal samples (outlier, innormal). The depth model will consider an out-of-distribution (OOD) sample as one of the classes of in-distribution (ID) samples and give a high confidence, here a normalized value of 0-1, to find out-of-distribution samples, but their settings may be different. For example, out-of-distribution detection (OOD detection) is modified on a model task, and is required to not only effectively detect out-of-distribution (OOD) samples, but also ensure that the performance of the model is not affected.

The method for detecting the samples in the distribution and outside the distribution can adopt a method based on a normalized index (Softmax-based), uncertainty (Uncertainty), a probability generation model (Generative model) and a classification model (Classifier) to detect the steps in the distribution and outside the distribution of class image data of ancient and modern artworks.

In the embodiment using the normalization index based method, in step 201, statistical analysis is performed using the maximum normalization index probability output by a pre-trained model (pre-trained model), in step 202, the distribution of the normalized index probabilities of the OOD sample and the ID sample is statistically found, in step 203, the distribution difference between the OOD sample and the ID sample is increased, and in step 204, a proper threshold is selected to determine whether a sample belongs to an out-distribution sample or an in-distribution sample. The method is simple and effective, the structure of the classification model is not required to be modified, and an out-of-distribution sample classifier is not required to be trained.

In embodiments using the uncertainty method, the confidence (confidence) of the model cannot be directly expressed due to the probabilistic output of the model. In step 211, an uncertainty attribute for the input sample is learned using the model. At step 212, the test data is evaluated, and if the model input is in-distribution samples, the uncertainty is low, whereas if the model input is out-of-distribution samples, the uncertainty is high. Such methods require modifying the network structure of the model to learn the uncertainty attributes.

In the embodiment using the probabilistic generative model method, in step 221, whether a sample belongs to a sample in distribution or a sample out of distribution is determined by using a reconstruction error (reconstruction error) of a Variational automatic encoder (Variational automatic encoder) or other measurement methods; in step 222, the implicit space (1 atent space) of the encoder can learn the significant features (silent vector) of the data in the distribution, but not for the samples outside the distribution, so the samples outside the distribution will generate higher reconstruction errors. The method only focuses on the detection performance outside the distribution, and does not focus on the original task of the data in the distribution.

In embodiments utilizing a classification model approach, at step 231, the classifier is used to classify the extracted features to determine whether the samples are out-of-distribution samples; in step 232, some modified network structures are n +1 classes of classifiers, n is the number of classes of the original classification task, and the n +1 th class is an out-of-distribution class; in step 233, some directly take the extracted features for classification without modifying the structure of the network.

FIG. 3 (a) is a flow chart of an embodiment of the fine-grained classification step of the ancient and modern art identification method of the present invention using an attention-convolutional neural network. In step 321, searching a characteristic region of the image data to be tested, and in step 322, inputting the characteristic region into a convolutional neural network; in step 323, a part of information of the characteristic region passing through the convolutional neural network enters a full connection layer and a normalized index logistic regression layer for classification; in step 324, another part of the information of the feature region passing through the convolutional neural network passes through an attention suggestion sub-network (APN) to obtain a candidate region; in step 325, repeating the step 323 and the step 324, so that the feature region selected by the APN is the region with the most discriminability; at step 326, a loss function is introduced to achieve a higher accuracy in identifying the class of image information.

The "fine-grained" classification step is under the ordinary classification, and the more fine classification requires explicit finding of the most discriminative (discriminative) feature in the picture. For ancient and modern artworks, it is necessary to find out the characteristics of details, such as the upwarping degree of petals, the subtle difference of patterns and the like.

Fig. 3 (b) is a schematic diagram of a framework of a cyclic attention convolutional neural network ("RA-CNN") according to an embodiment of the fine-grained classification step in the identification method of the present invention. Wherein the symbols

The representation cuts a portion of the feature region of the identified class image information and enlarges it. Each

row

301, 302, 303 represents a common CNN network. As shown in fig. 3 (b), the input goes from a coarse full-size image to a finer area attention (top-down). Wherein the picture (a) of the first row 301 ₁ ) The coarsest, third row of pictures (a) ₃ ) And is more precise. Image information a ₁ Into b ₁ (several convolution layers) are divided into two paths, one path is led to the step c ₁ Simply classifying the paths into full connected layers (FC) and softmax logistic regression layer, and the other path into d ₁ I.e. the Attention suggestion subnetwork ("APN"), gets a candidate area. Using d on the original drawing ₁ The proposed candidate area is cut (crop) to a small area with more discriminant on the original image, and a is obtained after interpolation ₂ By the same token, a is obtained ₃ 。

Continuously amplifying and thinning the characteristic region after the APN twice, and introducing a loss function (Ranking loss) in order to ensure that the characteristic region selected by the APN is the most discriminant region in the image: i.e. forced area a ₁ 、a ₂ 、a ₃ The classification confidence level (confidence score) of (1) is higher and higher (i.e. the corresponding P of the last column of the picture _t With increasing probability), that is to say with increasing accuracy, of the recognition class image information. In this way, the network is continually refined to differentiate attention regions (discrete attention regions) in conjunction with the normal classification loss.

Selected

partial images

312 and 313 of the identified class image information 311 are input to two convolutional neural networks 314 (a) and 315 (B). The outputs of the convolutional neural network streams a and B are multiplied 318 at each location of the image using an outer product and combined to obtain a bilinear vector 316, which is then passed through a classification layer 317 to obtain the prediction. The bilinear model M consists of a quaternionComprises the following components: m = (f) _A ，f _B (ii) a P; c) In that respect Wherein f is _A ，f _B The representative feature extraction functions, namely convolution networks a and B in fig. 3 (C), P is a Pooling function (and C is a classification function).

The feature extraction function f (·) (i.e., the convolutional neural network stream CNN stream) is composed of a convolutional layer, a pooling layer, and an activation function. This part of the network structure can be seen as a functional mapping:

f：L×I→RK×D (1)

and mapping the input identified class image information and the position area into one-dimensional characteristic, wherein K is the number of channels of the characteristic graph output by the convolution network, and D is the size of a one-dimensional characteristic vector expanded by the two-dimensional characteristic graph in each channel. The convolution characteristics output by the two characteristic extraction functions are converged through bilinear operation to obtain bilinear characteristics 316: biliiner (l; T; f) _A ，f _B )＝f _A (L；T) ^T f _B (L；T) ^T . The pooling function P serves to aggregate bilinear features of all locations into one feature. The pooling function used is to accumulate bilinear features at all positions to obtain a global feature representation of the image Φ' (I):

Φ`(I)＝∑ _l∈L bilinear(l；T，f _A ；f _B )＝∑ _l∈L f _A (l；I) ^T f _B (l；I) (2)

in two characteristic functions f _A ，f _B When the extracted feature dimensions are K × M and K × N, respectively, the output of the pooling function P is an M × N matrix, and the feature matrix is stretched into a row of feature vectors of MN size before being classified. Finally, the extracted features are classified by using a classification function, and the classification layer 317 is implemented by using a logistic regression or Support Vector Machine (SVM) classifier.

The CNN network can realize high-level semantic feature acquisition of fine-grained images, and irrelevant background information in the images is filtered through convolution parameters in an iterative training network model. On the other hand, the convolutional neural network stream a and the convolutional neural network stream B play complementary roles in the image recognition task, that is, the network a can locate the object in the image, and the network B completes feature extraction on the object position located by the network a. By the mode, the two networks can cooperate to complete the processes of class detection and target feature removal of the input fine-grained image, and the fine-grained image recognition task is well completed.

FIG. 3 (d) is a flow chart of an embodiment of the fine granularity classification step of the ancient and modern artwork identification method of the present invention using bilinear vector network. In step 331, the selected

partial images

312 and 313 in the identified class image information 311 are input into two convolutional neural networks 314 (a) and 315 (B); the outputs of the convolutional neural network streams a and B are multiplied 318 at each position of the image using the outer product and combined to obtain bilinear vectors 316, step 332, and the prediction results are obtained by the classification layer 317.

Fig. 3 (e) is a flow chart of an embodiment of the fine grained classification step in the ancient and modern art identification method steps of the present invention using navigation-teaching-review network (NTS-Net) classification. Generating a plurality of candidate frames on different-scale Feature maps (Feature maps) by using the identified class image information, wherein the coordinates of each candidate frame correspond to the pre-designed Anchors (Anchors) in step 341; in step 342, the "information amount" of each candidate area is scored, and the area with large information amount is scored high; in step 343, a Feature extraction step (Feature Extractor), a full connectivity layer (FC) and a normalization index (softmax) are sequentially performed on the Feature map; in step 344, the probability that the input area belongs to the target label is judged; at step 345, the non-normalized probabilities (logits) extracted from the local regions and the global map are merged (concat) together to generate a long vector, and the non-normalized probabilities (logits) corresponding to the 200 classes are output.

The fine-grained classification step in the identification method of the invention can also adopt a navigation-teaching-inspection network (NTS-Net) classification method which divides a network main body into three components of navigation (Navigator), teaching (Teacher) and inspection (Scrutinizer), wherein in the navigation step, a plurality of candidate frames are generated on Feature maps (Feature maps) with different scales, and the coordinates of each candidate frame correspond to the pre-designed Anchors (Anchors). Navigation (Navigator) scores the "information amount" of each candidate area, and areas with large information amounts have high scores. The teaching step is a commonly used Feature extraction step (Feature Extractor), a full connection layer (FC) and normalization index (softmax) step, and is used for judging the probability that the input area belongs to a target label (target label); the examination step is a full connection layer, the input is to merge (concat) the non-normalized probabilities (logits) extracted from each local region and the full graph together to generate a long vector, and the non-normalized probabilities (logits) corresponding to 200 categories are output.

The NTS method comprises the following specific steps: 1) The original graph of size (448, 3) enters the network, and after going through the Resnet-50 to extract features, the original graph becomes a (14, 14, 2048) feature graph, a 2048-dimensional feature vector after passing through the global pooling layer and a 200-dimensional unnormalized probability after passing through the global pooling layer and the full connection layer. 2) The preset network (RPN) generating candidate areas generates a total of 1614 Anchors (Anchors) according to different sizes (Size) and aspect ratios (aspect) in the three scales (14, 14) (7, 7) (4, 4). 3) And (3) scoring the navigation by using the feature map in the step 1, and only N local candidate frames with the largest information amount are reserved by using Non-Maximum Suppression (NMS) according to a scoring result. 4) Those N local regions are bilinearly interpolated (224 ) and input into a teaching (Teacher) network to obtain the feature vectors and non-normalized probabilities (logits) of those local regions. 5) Merging (concat) the full-graph feature vector and the local feature vector in the

steps

1 and 4, and then connecting to an FC layer to obtain a joint classification locations for final decision making.

Fig. 4 is a block diagram of an ancient and near modern art identification system of the present invention. Such as the server 401 of ancient and near modern art identification systems. The server of the ancient and recent modern art evaluation system includes a processor 410, which here may be a general purpose or application specific chip (ASIC/ASIC) or FPGA or NPU or the like, and a computer program product or computer readable medium in the form of a memory 420. The memory 420 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. The memory 420 has a memory space 430 for program code for performing any of the method steps of the method described above. For example, the storage space 430 for the program code may include respective program codes 431 for respectively implementing various steps in the above method. These program codes may be read or written into the processor 410. These computer program products comprise a program code carrier such as a hard disk, a Compact Disc (CD), a memory card or a floppy disk. Such a computer program product is typically a portable or fixed storage unit as described with reference to fig. 5. Fig. 5 is a computer product diagram of a portable or fixed storage unit of the ancient and recent modern art identification system of this invention. The storage unit may have storage segments, storage spaces, etc. arranged similarly to the memory 420 in the server of fig. 4. The program code may be compressed, for example, in a suitable form. Typically, the storage unit comprises computer readable code 431', i.e. code that can be read by a processor, such as 410, for example, which when executed by a server, causes the server to perform the steps of the method described above. The codes, when executed by the server, cause the server to perform the steps of the above described method.

Fig. 6 (1) is an example of authentic work image information involved in one embodiment of the ancient and recent modern art work identification method of the present invention. Fig. 6 (2) is an example of the authentic image information used to train the model. Fig. 6 (3) is an example of image information of an art to be authenticated. Fig. 6 (4) is a classification example. Wherein, fig. 6 (1) shows an example of certain image information of an artwork genuineness, and with the multiple pieces of image information obtained from 360 degrees of the artwork genuineness given in fig. 6 (2) as a standard, the artwork image information to be authenticated in fig. 6 (3), that is, class image information containing different feature regions, is subjected to artwork model identification to obtain confidence coefficients of 0 to 1, and the similarity between the artwork to be identified and the genuineness is evaluated, and the closer to 1, the more similar, and when the result is 1, the two are consistent. Fig. 6 (4) shows examples of classification based on different confidence values derived from different artwork image information to be authenticated, and fig. 6 (4) shows a value of 1 for which both are close to unity.

Reference herein to "one embodiment," "an embodiment," or "one or more embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. Moreover, it is noted that instances of the word "in one embodiment" are not necessarily all referring to the same embodiment.

The above description is only for the purpose of illustrating the present invention, and any person skilled in the art can modify and change the above embodiments without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention should be subject to the claims. The invention has been elucidated above with reference to an example. However, other embodiments than the above described are equally possible within the scope of this disclosure. The different features and steps of the invention may be combined in other ways than those described. The scope of the invention is limited only by the appended claims. More generally, those of ordinary skill in the art will readily appreciate that all parameters, dimensions, materials, and configurations described herein are exemplary and that actual parameters, dimensions, materials, and/or configurations will depend upon the particular application or applications for which the teachings of the present invention is/are used.

Claims

1. An ancient and modern artwork identification method comprises the following steps:

inputting authentic work image information; inputting image information of the artwork to be authenticated;

detecting a sample in distribution and a sample out of distribution by the detector according to the artwork image information and the authentic work image information to be authenticated;

classifying the samples in the distribution;

classifying the classified samples in the distribution and class image information similar to the artwork to be identified in a fine-grained manner;

and outputting the samples in the classified distribution or the samples after fine-grained classification, and obtaining the confidence coefficient of the image information of the artwork to be identified compared with the authentic work image information as an identification conclusion.

2. The authentication method as claimed in claim 1, wherein the step of detecting the in-distribution sample and the out-of-distribution sample of the artwork image information and the authenticity image information to be authenticated by a detector further comprises:

performing statistical analysis by using the maximum normalized exponential probability output by a pre-trained model (pre-trained model);

counting the distribution condition of the normalized exponential probability of the OOD sample and the ID sample;

enlarging the distribution difference of the two;

and selecting a proper threshold value to judge whether one sample belongs to the sample outside the distribution or the sample in the distribution.

3. The authentication method as claimed in claim 1, wherein the step of detecting the in-distribution sample and the out-of-distribution sample by the detector with the image information of the art to be authenticated and the image information of the authentic work further comprises:

learning an uncertainty attribute for the input sample using the model;

and judging the test data, wherein if the model input is a sample in the distribution, the uncertainty is low, and conversely, if the model input is a sample out of the distribution, the uncertainty is high.

4. The authentication method as claimed in claim 1, wherein the step of detecting the in-distribution sample and the out-of-distribution sample by the detector with the image information of the art to be authenticated and the image information of the authentic work further comprises:

judging whether a sample belongs to a sample in distribution or a sample out of distribution by utilizing a reconstruction error (reconstruction error) of a Variational automatic encoder (variable automatic encoder) or other measurement modes;

the implicit space (late space) of the encoder can learn the obvious features (silence vector) of the data in the distribution, but the implicit space does not apply to the samples outside the distribution, so the samples outside the distribution can generate higher reconstruction errors.

5. The authentication method as claimed in claim 1, wherein the step of detecting the in-distribution sample and the out-of-distribution sample of the artwork image information and the authenticity image information to be authenticated by a detector further comprises:

classifying the extracted features by using a classifier to judge whether the extracted features are out-of-distribution samples;

some modified network structures are n +1 classes of classifiers, n is the number of classes of the original classification task, and the n +1 class is the distributed class;

some methods directly take the extracted features for classification without modifying the structure of the network.

6. The authentication method as recited in any one of claims 1 to 5, wherein the step of fine-grained classifying the classified samples in the distribution with class image information similar to the artwork to be authenticated further comprises:

searching a characteristic area of the image data to be detected;

inputting the characteristic region into a convolutional neural network;

entering a full connection layer and a normalized index logistic regression layer for classification by part of information of the characteristic region of the convolutional neural network;

passing another part of information of the characteristic region of the convolutional neural network through an attention suggestion sub-network (APN) to obtain a candidate region;

repeating the classification step and the APN step to enable the feature region selected by the APN to be the region with the highest discriminability;

and a loss function is introduced, so that higher accuracy of identifying the class of image information is obtained.

7. The authentication method as recited in any one of claims 1 to 5, wherein the step of fine-grained classifying the classified samples in the distribution with class image information similar to the artwork to be authenticated further comprises:

inputting selected local images (312) and (313) in the identified class image information (311) into two convolutional neural networks (314, A) and (315, B);

the outputs of the convolutional neural network streams (a) and (B) are multiplied (318) at each position of the image using an outer product and combined to obtain a bilinear vector (316);

the prediction result is obtained by the classification layer (317).

8. The authentication method of claim 7, wherein said classification layer (317) is a logistic regression or a support vector machine classifier.

9. The method of any one of claims 1 to 5, wherein the step of fine-grained classifying the classified samples in the distribution with class image information that approximates the art to be identified, further comprises:

generating a plurality of candidate frames on different scale characteristic graphs of the identified class image information, wherein the coordinates of each candidate frame correspond to a pre-designed anchor;

the information amount of each candidate area is scored, and the area with large information amount is high in score;

sequentially performing a feature extraction step, a full connection layer (FC) step and a normalization index step on the feature map;

judging the probability that the input area belongs to the target label;

and combining the non-normalized probabilities extracted from each local area and the full graph to generate a long vector, and outputting the non-normalized probabilities corresponding to 200 categories.

10. An ancient and near-modern artwork identification system comprising:

the input module is used for inputting the authentic work image information; inputting image information of an artwork to be authenticated;

the detection module is used for detecting a sample in distribution and a sample outside the distribution by the detector according to the artwork image information and the authentic work image information to be identified;

the sample classification module is used for classifying the samples in the distribution;

the fine-grained classification module is used for performing fine-grained classification on the classified samples in the distribution and class image information similar to the artwork to be identified;

and the output module is used for outputting the classified samples in the distribution or the classified samples with fine granularity, and obtaining the confidence coefficient of the image information of the artwork to be identified compared with the authentic image information as an identification conclusion.

11. The authentication system of claim 10, wherein the detection module further comprises:

the analysis module is used for carrying out statistical analysis by utilizing the maximum normalized exponential probability output by a pre-trained model (pre-trained model);

enlarging the distribution difference of the two;

and the selection module selects a proper threshold value to judge whether one sample belongs to the sample outside the distribution or the sample in the distribution.

12. The authentication system of claim 10, wherein the detection module further comprises:

a learning module for learning an uncertainty attribute for the input sample using the model;

and the judging module is used for judging the test data, wherein if the model input is a sample in distribution, the uncertainty is low, and if the model input is a sample out of distribution, the uncertainty is high.

13. The authentication system of claim 10, wherein the detection module further comprises:

the judging module judges whether a sample belongs to a sample in distribution or a sample out of distribution by using a reconstruction error (reconstruction error) of a Variational automatic encoder (Variational automatic encoder) or other measurement modes;

the implicit space (late space) of the encoder can learn the obvious features (silence vector) of the data in the distribution, but not for the samples outside the distribution, so the samples outside the distribution can generate higher reconstruction errors.

14. The authentication system of claim 10, wherein the detection module further comprises:

some modified network structures are n +1 classes of classifiers, n is the number of classes of the original classification task, and the n +1 th class is an outer class;

15. The authentication system of any of claims 10-14, wherein the fine-grained classification module further comprises:

the characteristic searching module is used for searching a characteristic area of the image data to be detected;

the characteristic training module inputs the characteristic region into a convolutional neural network;

the partial information classification module is used for entering a full connection layer and a normalization index logistic regression layer for classification through partial information of the characteristic region of the convolutional neural network;

a candidate region obtaining module, which obtains a candidate region through another part of information of the characteristic region of the convolutional neural network via an attention suggestion sub-network (APN);

repeating the classification step and the APN step to enable the feature region selected through the APN to be the region with the most discriminability;

and the identification module introduces a loss function to obtain higher accuracy of identifying the class of image information.

16. The authentication system of any of claims 10-14, wherein the fine-grained classification module further comprises:

a local image training module, which inputs the selected local images (312) and (313) in the identified class image information (311) into two convolution neural networks (314, A) and (315, B);

a prediction module obtains a prediction result through the classification layer (317).

17. The authentication system of claim 16, wherein the classification layer (317) is a logistic regression or a support vector machine classifier.

18. The authentication system of any of claims 10-14, wherein the fine-grained classification module further comprises:

the candidate frame anchoring module is used for generating a plurality of candidate frames from the identified class image information on different scale characteristic graphs, and the coordinates of each candidate frame correspond to the pre-designed anchor;

the scoring module is used for scoring the 'information amount' of each candidate area, and the area with large information amount is high in score;

a feature map processing module for sequentially performing feature extraction, a full connection layer (FC) and a normalization index on the feature map;

the probability judgment module is used for judging the probability that the input area belongs to the target label;

and the probability merging output module is used for merging the non-normalized probabilities extracted from each local area and the full graph together to generate a long vector and outputting the non-normalized probabilities corresponding to 200 categories.