CN111291675B - Deep learning-based hyperspectral ancient painting detection and identification method - Google Patents
Deep learning-based hyperspectral ancient painting detection and identification method Download PDFInfo
- Publication number
- CN111291675B CN111291675B CN202010080017.8A CN202010080017A CN111291675B CN 111291675 B CN111291675 B CN 111291675B CN 202010080017 A CN202010080017 A CN 202010080017A CN 111291675 B CN111291675 B CN 111291675B
- Authority
- CN
- China
- Prior art keywords
- hyperspectral
- painting
- data
- spectrum
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000010422 painting Methods 0.000 title claims abstract description 110
- 238000001514 detection method Methods 0.000 title claims abstract description 50
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000013135 deep learning Methods 0.000 title claims abstract description 17
- 230000004927 fusion Effects 0.000 claims abstract description 30
- 230000003595 spectral effect Effects 0.000 claims abstract description 28
- 238000000605 extraction Methods 0.000 claims abstract description 22
- 238000012545 processing Methods 0.000 claims abstract description 10
- 238000012360 testing method Methods 0.000 claims abstract description 8
- 238000000354 decomposition reaction Methods 0.000 claims abstract description 4
- 238000001228 spectrum Methods 0.000 claims description 64
- 230000008569 process Effects 0.000 claims description 22
- 239000013598 vector Substances 0.000 claims description 17
- 238000010586 diagram Methods 0.000 claims description 14
- 238000001914 filtration Methods 0.000 claims description 10
- 239000011159 matrix material Substances 0.000 claims description 8
- 238000000926 separation method Methods 0.000 claims description 8
- 238000012549 training Methods 0.000 claims description 8
- 238000000513 principal component analysis Methods 0.000 claims description 7
- 241001465754 Metazoa Species 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 230000001629 suppression Effects 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 4
- 238000000701 chemical imaging Methods 0.000 claims description 3
- 238000002372 labelling Methods 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 2
- 238000007500 overflow downdraw method Methods 0.000 claims 1
- 230000007547 defect Effects 0.000 abstract description 5
- 238000013528 artificial neural network Methods 0.000 abstract description 3
- 239000000049 pigment Substances 0.000 description 10
- 238000011160 research Methods 0.000 description 8
- 241000282414 Homo sapiens Species 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 241000651994 Curio Species 0.000 description 1
- 241000196324 Embryophyta Species 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000012899 de-mixing Methods 0.000 description 1
- 238000009792 diffusion process Methods 0.000 description 1
- 230000008451 emotion Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 229910052500 inorganic mineral Inorganic materials 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000003340 mental effect Effects 0.000 description 1
- 239000011707 mineral Substances 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Astronomy & Astrophysics (AREA)
- Remote Sensing (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a hyperspectral ancient painting detection and identification method based on deep learning, which comprises the following steps: collecting hyperspectral data of the ancient painting, and constructing a hyperspectral ancient painting data set; performing data expansion on the hyperspectral ancient painting dataset; performing mixed pixel decomposition by using a pseudo-removing projection matching unmixing algorithm; constructing a multi-element feature extraction model based on deep learning, and extracting spectral information and spatial information of the hyperspectral of the ancient painting; constructing a multi-scale characteristic fusion detection and identification model of the multi-element information; and randomly selecting a test sample from the hyperspectral ancient painting data set to form a new data set, and verifying the detection and identification model. The method utilizes the technical advantages of rich hyperspectral image information and the advantages of rapidness, accuracy, high efficiency and the like of the neural network target detection based on deep learning to detect and identify the ancient painting, has the characteristics of rapidness and high efficiency, and overcomes the defect of insufficient spectral information in common painting image processing.
Description
Technical Field
The invention relates to the technical field of hyperspectral image processing, in particular to a hyperspectral antique painting detection and identification method based on deep learning.
Background
Because of the particularity of ancient painting on cultural diffusion, the painting art appreciation and research is necessary, the Chinese cultural background is profound, the painting has the huge characteristics of wide content, large information quantity and large quantity, the contemporaneous painting contains various painting types, the contemporaneous painting types are reflected in the painting of different dynasties, the current painting image appreciation detection work mainly depends on a large number of manual labels for analysis and processing, and the accurate identification of the painting ages is challenging. In recent years, the application of hyperspectral technology in antique and curio is gradually rising, hyperspectral images have rich spectral characteristic information and spatial characteristic information, and hyperspectral data are used as a data cube with mass information, so that the hyperspectral image has great mining value for detection and research of antique painting. The painting of different ages, because the creator belongs to different ages, the pigment and painting style used are different, through detecting and identifying hyperspectral ancient painting, the characteristic information is extracted from the artistic image and detected and identified, not only can the demands of people on artistic and cultural researches be met, but also help can be provided for the repair work of ancient painting cultural relics.
The painting figure detection needs the common support of the fields of hyperspectral image recognition, artistry, computer vision, feature extraction, pattern recognition, artificial intelligence and the like, and the cross disciplines bring technical innovation and make the ancient painting years detection research very challenging. The painting art often causes the difference of pigment use due to the different ages, and in addition, the creator is influenced by the different abstract processes of the culture of the creator at the time and has the specificity of the art expression of lines, so that the content, style and the expression emotion of the painting in different ages also have different characteristics. The cultural relics of each dynode, and in particular the painting cultural relics, most carry political symbolism of each dynode, but the variations between the painting styles and the contents of the similar dynodes are subtle, for example: the wall paintings of the middle and late tangs are similar in both style and content, so that the two dynasty painting creation times are difficult to distinguish by only observing through eyes. At present, research on hyperspectral ancient painting detection and identification is lacking, and painting art research is necessary as pursuit of human beings on the mental world for deep and multi-angle research.
Disclosure of Invention
The invention aims to provide a hyperspectral ancient painting detection and identification method based on deep learning, which aims to solve the technical problems of detection and identification of ancient painting ages, true and false, content expression and the like.
In order to solve the technical problems, the embodiment of the invention provides the following scheme:
a hyperspectral ancient painting detection and identification method based on deep learning comprises the following steps:
s1, collecting hyperspectral data of an ancient painting, and constructing a hyperspectral ancient painting data set;
s2, carrying out data expansion on the hyperspectral ancient painting dataset;
s3, performing mixed pixel decomposition by using a pseudo-removing projection matching unmixing algorithm;
s4, constructing a multi-element feature extraction model based on deep learning, and extracting spectral information and spatial information of the hyperspectral of the ancient painting;
s5, constructing a multi-information multi-scale feature fusion detection recognition model;
s6, randomly selecting a test sample from the hyperspectral antique painting data set to form a new data set, and verifying the detection and identification model.
Preferably, the step S1 includes:
the hyperspectral palace painting data set is constructed by collecting the palace painting hyperspectral data through the existing hyperspectral palace painting public data and using hyperspectral imaging equipment, and the hyperspectral palace painting data set contains hyperspectral palace painting data of character painting images, landscape painting images and animal flower images of different ages;
labeling the sample data in the hyperspectral ancient painting data set, and dividing the sample data into a training sample and a test sample;
and simultaneously establishing a target end member spectrum library.
Preferably, the step S2 includes:
the acquired hyperspectral drawing data are expanded and augmented by adopting a sampling mode respectively, and the data are expanded by randomly cutting and reserving 70% -85% of the area of the original hyperspectral drawing data.
Preferably, the step S3 includes:
respectively extracting hyperspectral data and a spectrum library target end member;
performing minimum noise separation transformation on the hyperspectral data and the target end member;
carrying out matched filtering on the hyperspectral data and the target end member to obtain an abundance image of a possible target end member;
and establishing a hyperspectral high-dimensional convex geometric model, eliminating false positive results, and finally obtaining a target distribution diagram.
Preferably, in the step S4, the step of extracting spectral information of the archaic hyperspectral includes:
the spectral information is converted into the space dimension of the image through the spectral angle conversion, the spectral information is converted into a two-dimensional gray image from a one-dimensional vector, the gray value is high at the place with larger spectral difference, and the gray value is low at the place with smaller spectral difference, so that the characteristic extraction of the spectral information is realized.
Preferably, in the step S4, the step of extracting spatial information of the archaic hyperspectral includes:
and carrying out principal component analysis processing on the hyperspectral image, and extracting the spatial information of the hyperspectral data.
Preferably, the step S5 includes:
the method is characterized in that the spectrum information and the space information of the hyperspectral of the ancient painting are taken as input and are imported into the multi-dimensional characteristic fusion detection and identification model of the multi-element information, and the realization process of the model is as follows:
a depth residual error connection network is used as a main feature extraction network, a plurality of convolution layers are added at the back, the convolution layers gradually reduce the size of the feature map, and a plurality of scale feature maps are fused together;
and after feature fusion, inputting the data into a full-connection layer, outputting a vector probability matrix, determining whether a real data mark is matched with a prediction mark in a data training process, filtering out an optimal prediction result by a non-maximum suppression method, and finally realizing multi-scale detection and identification.
Preferably, in the step S3, the matched filtering implementation process is divided into a process of performing contrast balance between the image variance and the target background, and projecting the average corrected target spectrum onto a generalized inverse matrix of covariance data:
wherein, PV is a matched projection vector, tsmnf is a target spectrum converted into MNF space; dsmnf is the pixel average spectrum of the hyperspectral data converted into MNF space; the score range from 0 to 1 then gives a score PVI value of zero spectrum to the target spectrum:
PVI＝PV*Dmnf
dmnf is an MNF data set, and is realized by searching a part of contrast vectors which are perpendicular to a limited space by utilizing covariance data, a projection vector PV is obtained according to matched projection, image variances of background target separation and output are balanced, and known information under the condition that the background is unknown and mixed pixels exist is positioned.
Preferably, in the step S3, the process of excluding the false positive result is as follows:
and directly identifying and rejecting common false positive results in projection by using a high-dimensional convex geometric model of the mixed spectrum, and eliminating partial false detection results by establishing the high-dimensional convex geometric model of the hyperspectrum to finally obtain a target distribution diagram.
Preferably, the conversion into a gray image using the spectral angle is implemented as follows:
the spectrum angular distance between the sample point pixel and 8 pixels adjacent to the sample point pixel is calculated, the spectrum angular distance value is used as a coordinate value, the spectrum dimension is mapped to a new space dimension, the Euclidean distance from the sample point to the coordinate origin of the 8-dimensional space is calculated, the obtained value is converted into a gray value to be given to the current pixel, the same operation is carried out on each pixel in the hyperspectral image, and finally a gray image is obtained.
The scheme of the invention at least comprises the following beneficial effects:
in the scheme, the advantages of high-spectrum image information quantity and the advantages of rapidness, accuracy, high efficiency and the like of neural network target detection based on deep learning are utilized to detect and identify ancient painting, so that the defects of large workload of traditional painting information feature extraction and detection and identification, large errors caused by human factors and the like are overcome, the method has the characteristics of rapidness and high efficiency, and meanwhile, the defect of insufficient spectrum information in ordinary painting image processing is overcome.
Drawings
FIG. 1 is a flow chart of a hyperspectral antique painting detection and identification method based on deep learning provided by an embodiment of the invention;
FIG. 2 is a flowchart of a hyperspectral archaizing painting detection and identification process in an embodiment of the present invention;
FIG. 3 is an overall schematic diagram of a hyperspectral archaic painting detection and identification process in an embodiment of the invention;
FIG. 4 is a flow chart of a de-pseudo projection matching unmixing algorithm in an embodiment of the invention;
FIG. 5 is a schematic diagram of spectral information feature extraction of hyperspectral data in an embodiment of the present invention;
FIG. 6 is a schematic diagram of hyperspectral data spatial information feature extraction in an embodiment of the present invention;
FIG. 7 is a block diagram of a multi-information multi-scale feature fusion detection recognition model in an embodiment of the invention.
Detailed Description
In order to make the technical problems, technical solutions and advantages to be solved more apparent, the following detailed description will be given with reference to the accompanying drawings and specific embodiments.
The embodiment of the invention provides a hyperspectral ancient painting detection and identification method based on deep learning, which comprises the following steps as shown in fig. 1:
s1, collecting hyperspectral data of an ancient painting, and constructing a hyperspectral ancient painting data set;
s2, carrying out data expansion on the hyperspectral ancient painting dataset;
s3, performing mixed pixel decomposition by using a pseudo-removing projection matching unmixing algorithm;
s4, constructing a multi-element feature extraction model based on deep learning, and extracting spectral information and spatial information of the hyperspectral of the ancient painting;
s5, constructing a multi-information multi-scale feature fusion detection recognition model;
s6, randomly selecting a test sample from the hyperspectral antique painting data set to form a new data set, and verifying the detection and identification model.
The method of the invention utilizes the technical advantages of rich hyperspectral image information quantity and the advantages of rapidness, accuracy, high efficiency and the like of neural network target detection based on deep learning to detect and identify ancient painting, thereby overcoming the defects of large workload of traditional painting information feature extraction and detection identification, large error caused by human factors and the like, having the characteristics of rapidness and high efficiency, and simultaneously making up the defect of insufficient spectral information in common painting image processing.
As a specific embodiment of the method of the present invention, as shown in fig. 2 and 3, the detection and identification process of the hyperspectral antique painting is as follows: acquiring hyperspectral data, constructing a hyperspectral antique painting data set and expanding the data; the mixed pixel is decomposed through a pseudo-projection matching unmixing algorithm, so that the mixed pixel problem caused by pigment mixing is solved, the data volume is reduced, and redundant information is eliminated; constructing a multi-element feature extraction module, and extracting spectrum information and space information of the ancient painting pigment; the multi-scale characteristic fusion detection and identification model of the multi-scale information is constructed, the extracted spectrum information and the spatial information are used as input to be imported into the multi-scale characteristic fusion detection and identification model of the multi-scale information, the model can realize characteristic fusion of various hyperspectral information, and the model can realize characteristic extraction of the ancient painting and effectively identify the age, true and false, content and the like of the ancient painting.
Further, step S1 includes:
the hyperspectral palace painting data set is constructed by collecting the prior hyperspectral palace painting public data and the hyperspectral painting hyperspectral data with the wave band of 400-2500 nm by using hyperspectral imaging equipment, wherein the hyperspectral palace painting data set comprises hyperspectral palace painting data of character painting images, landscape painting images and animal and flower images in different ages;
labeling the sample data in the hyperspectral ancient painting data set, and dividing the sample data into a training sample and a test sample;
and simultaneously establishing a target end member spectrum library.
Further, if the obtained hyperspectral archaizing data volume is smaller, the self-built data set can be expanded and constructed, and step S2 includes:
the acquired hyperspectral drawing data are expanded and augmented by adopting a sampling mode respectively, and the data are expanded by randomly cutting and reserving 70% -85% of the area of the original hyperspectral drawing data.
It is known from a large number of documents that ancient pigment types are not very large, and are basically classified into three categories of mineral pigments, animal pigments and plant pigments, so that the international standard spectrum libraries such as United States Geological Survey (USGS) spectrum library, jet Propulsion Laboratory (JPL) spectrum library, italian national art committee (IFCA) spectrum library, etc., which are most commonly accepted internationally at present, are selected, different target end member spectra are obtained therefrom, and a target end member spectrum library is established.
Further, step S3 includes:
respectively extracting hyperspectral data and a spectrum library target end member;
performing minimum noise separation transformation on the hyperspectral data and the target end member;
carrying out matched filtering on the hyperspectral data and the target end member to obtain an abundance image of a possible target end member;
and establishing a hyperspectral high-dimensional convex geometric model, eliminating false positive results, and finally obtaining a target distribution diagram.
The implementation process of the matched filtering is divided into the steps of carrying out contrast balance between image variance and target background, and projecting the average corrected target spectrum onto a generalized inverse matrix of covariance data:
wherein, PV is a matched projection vector, tsmnf is a target spectrum converted into MNF space; dsmnf is the pixel average spectrum of the hyperspectral data converted into MNF space; the score range from 0 to 1 then gives a score PVI value of zero spectrum to the target spectrum:
PVI＝PV*Dmnf
dmnf is an MNF data set, and is realized by searching a part of contrast vectors which are perpendicular to a limited space by utilizing covariance data, a projection vector PV is obtained according to matched projection, image variances of background target separation and output are balanced, and known information under the condition that the background is unknown and mixed pixels exist is positioned.
According to the pigment used by the object in the general painting content and the environmental background in the ancient painting, the painting object content and the background can be separated, the obtained gray image object is brighter in color, the background is darker in color, and the problem of misclassification caused by high pigment similarity can be solved.
The implementation process for eliminating false positive results comprises the following steps: and directly identifying and rejecting common false positive results in projection by using a high-dimensional convex geometric model of the mixed spectrum, and eliminating partial false detection results by establishing the high-dimensional convex geometric model of the hyperspectrum to finally obtain a target distribution diagram.
Fig. 4 is a flow chart of a de-pseudo projection matching and de-mixing algorithm in an embodiment of the invention. The hyperspectral data and the target end member are subjected to minimum noise separation (MNF) transformation, then matched filtering is carried out on the hyperspectral data and the target end member, and contrast balance is carried out between the image variance and the target background. The projection of the average corrected target spectrum onto the generalized inverse matrix of covariance data is achieved by finding a portion of the contrast vector perpendicular to the finite space using the covariance data, giving a score PVI value of zero spectrum to the target spectrum in terms of a projection score range from 0 to 1. The projection optimally balances the image variance of the separation and output of the background target, locates the known information of the condition that the background is unknown and the mixed pixels exist, and obtains the abundance image of the possible target end member.
In order to eliminate a large number of false positive values possibly existing in a projection result, a high-dimensional convex geometric model of a mixed spectrum is utilized to directly identify and reject a common false positive result in projection, a target distribution diagram is finally obtained by establishing the high-dimensional convex geometric model of the hyperspectrum and eliminating partial false detection results, n classes of target end members are set by the algorithm, and a series of n-dimensional gray images are finally obtained by decomposing the mixed pixels.
Further, in step S4, the step of extracting the spectrum information of the archaic hyperspectral includes:
the spectral information is converted into the space dimension of the image through the spectral angle conversion, the spectral information is converted into a two-dimensional gray image from a one-dimensional vector, the gray value is high at the place with larger spectral difference, and the gray value is low at the place with smaller spectral difference, so that the characteristic extraction of the spectral information is realized.
The realization process of converting the spectrum angle into the gray image comprises the following steps:
the spectrum angular distance between the sample point pixel and 8 pixels adjacent to the sample point pixel is calculated, the spectrum angular distance value is used as a coordinate value, the spectrum dimension is mapped to a new space dimension, the Euclidean distance from the sample point to the coordinate origin of the 8-dimensional space is calculated, the obtained value is converted into a gray value to be given to the current pixel, the same operation is carried out on each pixel in the hyperspectral image, and finally a gray image is obtained.
Fig. 5 is a schematic diagram of spectral information feature extraction of hyperspectral data in an embodiment of the present invention. Let the current pixel be x i,j Its 8 adjacent pixels are x i-1,j-1 、x i-1,j 、x i-1,j+1 、x i,j-1 、x i,j+1 、x i+1,j-1 、x i+1,j 、x i+1,j+1 Calculating the spectrum angle between the pixel and 8 adjacent pixels, and mapping the values of the 8 spectrum angles as coordinate values onto an 8-dimensional space coordinate axis, wherein the 8 coordinate values of the sample point are the spectrum angles of the current pixel and 8 adjacent pixels around, and the Euclidean distance from the sample point to the origin of the 8-dimensional space coordinate represents the pixel x i,j The magnitude of the integrated similarity to its neighboring pixels. If the spectrum difference between a certain pixel and an adjacent pixel is larger, the spectrum angle between the pixel and the adjacent pixel is larger, the 8 coordinate values of the pixel are larger, and the distance from the sample point to the origin of coordinates is far; conversely, if the spectral difference between the current pixel and the neighboring pixel is smaller, its 8 coordinate values are smaller and the sample point is closer to the origin of coordinates. And converting the distance value into a gray value, giving the gray value to the current pixel, and executing the same operation on each pixel in the hyperspectral image to finally obtain a gray image.
Further, in step S4, the step of extracting spatial information of the archaic hyperspectral includes:
and carrying out principal component analysis processing on the hyperspectral image, and extracting the spatial information of the hyperspectral data.
Fig. 6 is a schematic diagram of hyperspectral data spatial information feature extraction in an embodiment of the present invention.
Firstly, carrying out Principal Component Analysis (PCA) processing on hyperspectral painting images, extracting spatial information of targets, such as characteristics of forms, colors, textures and the like of contents in painting, and carrying out characteristic extraction through spatial differences among the content information in different painting.
The method comprises the steps of obtaining the space information of pixels in hyperspectral image data, firstly adopting PCA to reduce the dimension, greatly reducing the problem of low detection efficiency caused by high dimension of the hyperspectral image data, extracting useful space information, removing useless information, extracting target pixels from each wave band of an original image, and then taking pixel blocks of M multiplied by N (for example, 12 multiplied by 12) around the target pixels as training samples and test samples to be sent into a target detection model.
Further, step S5 includes:
the method is characterized in that the spectrum information and the space information of the hyperspectral of the ancient painting are taken as input and are imported into the multi-dimensional characteristic fusion detection and identification model of the multi-element information, and the realization process of the model is as follows:
a depth residual error connection network (ResNet-101) is used as a main feature extraction network, a plurality of convolution layers are added at the back, the convolution layers gradually reduce the size of the feature map, and a plurality of scale feature maps are fused together;
and after feature fusion, inputting the data into a full-connection layer, outputting a vector probability matrix, determining whether a real data mark is matched with a prediction mark in a data training process, filtering out an optimal prediction result by a non-maximum suppression method, and finally realizing multi-scale detection and identification.
FIG. 7 is a block diagram of a multi-information multi-scale feature fusion detection recognition model in an embodiment of the invention. The implementation process of the model is as follows: the method comprises the steps of using ResNet-101 as a main feature extraction network, introducing a 3 rd block into a feature fusion layer to serve as a first part of the feature fusion layer, changing an input image into 19X 512 after passing through the ResNet-101 network, changing the input image into 19X 1024 after passing through a 3X 3 convolution layer to serve as a second part of the feature fusion layer, changing the input image into 10X 512 after passing through a pooling layer to serve as a third part of the feature fusion layer, changing the input image into 5X 256 after passing through a 3X 3 convolution layer to serve as a fourth part of the feature fusion layer, changing the input image into 1X 256 after passing through the 3X 3 convolution layer and the 1X 1 convolution layer to serve as a fifth part of the feature fusion layer, gradually reducing the sizes of the feature images, and fusing a plurality of scale feature images together, wherein the method comprises the steps of:
A i ＝[αX i ,(1-α)Y i ]
wherein A is i Represents the ith feature value, X of the new fusion feature i Representing the ith spectral feature, Y i Representing the ith spatial feature. Alpha is a scaling factor and belongs to 0-1. The characteristic splicing scheme with weight is carried out when the characteristic fusion is carried out, and the scaling factor is an empirical value and needs to be adjusted in the experimental process. Fused feature transfusionAnd (3) inputting the data into a full-connection layer, outputting a vector probability matrix, determining whether a real data mark is matched with a predicted mark in the data training process, filtering out an optimal predicted result by a non-maximum suppression method, and finally achieving the purpose of multi-scale detection and identification.
In conclusion, the method provided by the invention applies the deep learning technology based on the convolutional neural network to the research and analysis of the hyperspectral image of the ancient painting, can effectively identify and detect the age, the true and false of the ancient painting and the painted content, and further solves the problem that the painting identification is time-consuming and labor-consuming by manpower.
While the foregoing is directed to the preferred embodiments of the present invention, it will be appreciated by those skilled in the art that various modifications and adaptations can be made without departing from the principles of the present invention, and such modifications and adaptations are intended to be comprehended within the scope of the present invention.
Claims (5)
1. The hyperspectral ancient painting detection and identification method based on deep learning is characterized by comprising the following steps of:
s1, collecting hyperspectral data of an ancient painting, and constructing a hyperspectral ancient painting data set; labeling the sample data in the hyperspectral ancient painting data set, and dividing the sample data into a training sample and a test sample; simultaneously establishing a target end member spectrum library;
s2, carrying out data expansion on the hyperspectral ancient painting dataset; expanding the original hyperspectral drawing data by randomly cutting and reserving 70% -85% of the area of the hyperspectral drawing data;
s3, performing mixed pixel decomposition by using a pseudo-removing projection matching unmixing algorithm;
the step S3 includes:
respectively extracting hyperspectral data and a spectrum library target end member;
performing minimum noise separation transformation on the hyperspectral data and the target end member;
carrying out matched filtering on the hyperspectral data and the target end member to obtain an abundance image of a possible target end member;
establishing a hyperspectral high-dimensional convex geometric model, eliminating false positive results, and finally obtaining a target distribution diagram;
the implementation process of the matched filtering is divided into the steps of carrying out contrast balance between image variance and target background, and projecting the average corrected target spectrum onto a generalized inverse matrix of covariance data:
wherein, PV is a matched projection vector, tsmnf is a target spectrum converted into MNF space; dsmnf is the pixel average spectrum of the hyperspectral data converted into MNF space; the score range from 0 to 1 then gives a score PVI value of zero spectrum to the target spectrum:
PVI＝PV*Dmnf
dmnf is MNF data set, which is realized by searching a part of contrast vectors which are perpendicular to the limited space by utilizing covariance data, obtaining a projection vector PV according to matched projection, balancing background target separation and output image variance, and positioning known information under the condition that the background is unknown and mixed pixels exist;
s4, constructing a multi-element feature extraction model based on deep learning, and extracting spectral information and spatial information of the hyperspectral of the ancient painting;
the step of extracting the spectrum information of the ancient painting hyperspectrum comprises the following steps:
converting the spectrum information into the space dimension of the image through spectrum angle conversion, converting the spectrum information into a two-dimensional gray image from a one-dimensional vector, and obtaining a gray value at a place with a large spectrum difference and a gray value at a place with a small spectrum difference, so as to realize the feature extraction of the spectrum information;
the step of extracting the spatial information of the hyperspectral of the ancient painting comprises the following steps:
carrying out principal component analysis processing on the hyperspectral image, and extracting space information of hyperspectral data; firstly, carrying out principal component analysis processing on a hyperspectral image, extracting spatial information of a target, including morphology, color and texture characteristics of contents in a drawing, and carrying out characteristic extraction through spatial differences among the content information in different drawings;
s5, constructing a multi-information multi-scale feature fusion detection recognition model;
the depth residual connection network ResNet-101 is used as a main feature extraction network, the 3 rd block is introduced into a feature fusion layer to serve as a first part of the feature fusion layer, an input image is changed into 19 multiplied by 512 after passing through the ResNet-101 network, then is changed into 19 multiplied by 1024 after passing through a 3 multiplied by 3 convolution layer to serve as a second part of the feature fusion layer, then is changed into 10 multiplied by 512 after passing through a pooling layer to serve as a third part of the feature fusion layer, then is changed into 5 multiplied by 256 after passing through the 3 multiplied by 3 convolution layer to serve as a fourth part of the feature fusion layer, finally, is changed into 1 multiplied by 256 after passing through the 3 multiplied by 3 convolution layer and the 1 multiplied by 1 convolution layer to serve as a fifth part of the feature fusion layer, and the convolution layers gradually reduce the sizes of the feature images, so that a plurality of scale feature images are fused together, and the fusion method is as follows:
A i ＝[αX i ,(1-α)Y i ]
wherein A is i Represents the ith feature value, X of the new fusion feature i Representing the ith spectral feature, Y i Representing an ith spatial feature; alpha is a scaling factor, belonging to 0-1; the characteristic splicing scheme with weight is carried out when the characteristic fusion is carried out, and the scaling factor is an empirical value and is adjusted in the experimental process; the fused features are input into a full-connection layer, a vector probability matrix is output, whether a real data mark is matched with a prediction mark or not is determined in the data training process, an optimal prediction result is filtered out through a non-maximum suppression method, and finally the purpose of multi-scale detection and identification is achieved;
s6, randomly selecting a test sample from the hyperspectral antique painting data set to form a new data set, and verifying the detection and identification model.
2. The method for detecting and identifying hyperspectral archaizing painters according to claim 1, wherein the step S1 comprises:
the hyperspectral paleo-painting data set is constructed by collecting paleo-painting hyperspectral data through the existing hyperspectral paleo-painting public data and using hyperspectral imaging equipment, and the hyperspectral paleo-painting data set contains hyperspectral paleo-painting data of character painting images, landscape painting images and animal flower images of different ages.
3. The method for detecting and identifying hyperspectral archaic painting according to claim 1, wherein the step S5 comprises:
and taking spectral information and spatial information of the hyperspectral of the ancient painting as input, and importing the spectral information and the spatial information into the multi-dimensional characteristic fusion detection recognition model of the multi-element information.
4. The hyperspectral archaic painting detection and identification method according to claim 1, wherein in the step S3, the implementation process of eliminating false positive results is as follows:
and directly identifying and rejecting common false positive results in projection by using a high-dimensional convex geometric model of the mixed spectrum, and eliminating partial false detection results by establishing the high-dimensional convex geometric model of the hyperspectrum to finally obtain a target distribution diagram.
5. The hyperspectral archaic painting detection and identification method according to claim 1, wherein the implementation process of converting the spectrum angle into the gray level image is as follows:
the spectrum angular distance between the sample point pixel and 8 pixels adjacent to the sample point pixel is calculated, the spectrum angular distance value is used as a coordinate value, the spectrum dimension is mapped to a new space dimension, the Euclidean distance from the sample point to the coordinate origin of the 8-dimensional space is calculated, the obtained value is converted into a gray value to be given to the current pixel, the same operation is carried out on each pixel in the hyperspectral image, and finally a gray image is obtained.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010080017.8A CN111291675B (en) | 2020-02-04 | 2020-02-04 | Deep learning-based hyperspectral ancient painting detection and identification method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010080017.8A CN111291675B (en) | 2020-02-04 | 2020-02-04 | Deep learning-based hyperspectral ancient painting detection and identification method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111291675A CN111291675A (en) | 2020-06-16 |
| CN111291675B true CN111291675B (en) | 2024-01-26 |
Family
ID=71024408
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010080017.8A Active CN111291675B (en) | 2020-02-04 | 2020-02-04 | Deep learning-based hyperspectral ancient painting detection and identification method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111291675B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112329792B (en) * | 2020-10-30 | 2022-12-09 | 中国电子科技集团公司第五十四研究所 | Hyperspectral image target feature extraction method based on spectrum angle |
| CN112396066B (en) * | 2020-11-27 | 2024-04-30 | 广东电网有限责任公司肇庆供电局 | Feature extraction method suitable for hyperspectral image |
| CN113327218B (en) * | 2021-06-10 | 2023-08-25 | 东华大学 | Hyperspectral and full-color image fusion method based on cascade network |
| CN115587298A (en) * | 2021-07-05 | 2023-01-10 | 中国矿业大学(北京) | Historical Jingdezhen blue and white porcelain age discrimination method based on deep learning |
| CN114445720B (en) * | 2021-12-06 | 2023-06-20 | 西安电子科技大学 | Hyperspectral anomaly detection method based on spatial spectrum depth synergy |
| CN116612337A (en) * | 2023-07-19 | 2023-08-18 | 中国地质大学（武汉） | Object detection method, device and system based on hyperspectral image and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102879099A (en) * | 2012-08-08 | 2013-01-16 | 北京建筑工程学院 | Wall painting information extraction method based on hyperspectral imaging |
| CN107843593A (en) * | 2017-10-13 | 2018-03-27 | 上海工程技术大学 | A kind of textile material recognition methods and system based on high light spectrum image-forming technology |
| CN108416357A (en) * | 2018-03-01 | 2018-08-17 | 北京建筑大学 | A kind of extracting method of colored drawing class historical relic implicit information |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8411162B2 (en) * | 2004-12-23 | 2013-04-02 | Hewlett-Packard Development Company, L.P. | Image processing methods and systems for fine art reproduction |
-
2020
- 2020-02-04 CN CN202010080017.8A patent/CN111291675B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102879099A (en) * | 2012-08-08 | 2013-01-16 | 北京建筑工程学院 | Wall painting information extraction method based on hyperspectral imaging |
| CN107843593A (en) * | 2017-10-13 | 2018-03-27 | 上海工程技术大学 | A kind of textile material recognition methods and system based on high light spectrum image-forming technology |
| CN108416357A (en) * | 2018-03-01 | 2018-08-17 | 北京建筑大学 | A kind of extracting method of colored drawing class historical relic implicit information |
Non-Patent Citations (5)
| Title |
|---|
| Analysis of Imaging Spectrometer Data Using N-Dimensional Geometry and a Mixture-Tuned Matching Filtering Approach;Boardman, Joseph W.;IEEE;第4138-4152页 * |
| 一种基于光谱角空间变换的高光谱图像分割方法;徐君 等;红外技术;第1013-1018页 * |
| 侯妙乐 等.高光谱成像技术在彩绘文物分析中的研究综述.光谱学与光谱分析.2017,第１ ８ ５ ２ －１ ８ ６ ０页. * |
| 成像光谱技术的古画隐藏信息提取;郭新蕾;张立福;吴太夏;张红明;罗旭东;;中国图象图形学报(第10期);全文 * |
| 高光谱成像技术在彩绘文物分析中的研究综述;侯妙乐 等;光谱学与光谱分析;第１ ８ ５ ２ －１ ８ ６ ０页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111291675A (en) | 2020-06-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111291675B (en) | Deep learning-based hyperspectral ancient painting detection and identification method | |
| CN110929607B (en) | Remote sensing identification method and system for urban building construction progress | |
| Ding et al. | Subkilometer crater discovery with boosting and transfer learning | |
| Feng et al. | Robust and efficient algorithms for separating latent overlapped fingerprints | |
| CN109871823B (en) | Satellite image ship detection method combining rotating frame and context information | |
| CN111539409B (en) | Ancient tomb question and character recognition method based on hyperspectral remote sensing technology | |
| CN111652273B (en) | Deep learning-based RGB-D image classification method | |
| CN110909643B (en) | Remote sensing ship image small sample classification method based on nearest neighbor prototype representation | |
| CN112906550B (en) | Static gesture recognition method based on watershed transformation | |
| CN112257741B (en) | Method for detecting generative anti-false picture based on complex neural network | |
| CN115205521B (en) | Kitchen waste detection method based on neural network | |
| CN109766752B (en) | Target matching and positioning method and system based on deep learning and computer | |
| CN111008664A (en) | Hyperspectral sea ice detection method based on space-spectrum combined characteristics | |
| CN111709443B (en) | Calligraphy character style classification method based on rotation invariant convolution neural network | |
| CN112241730A (en) | Form extraction method and system based on machine learning | |
| Musicco et al. | Automatic point cloud segmentation for the detection of alterations on historical buildings through an unsupervised and clustering-based Machine Learning approach | |
| CN111126494A (en) | Image classification method and system based on anisotropic convolution | |
| Shu et al. | Detecting 3D points of interest using projective neural networks | |
| CN111222576B (en) | High-resolution remote sensing image classification method | |
| Raj et al. | Grantha script recognition from ancient palm leaves using histogram of orientation shape context | |
| CN115359304B (en) | Single image feature grouping-oriented causal invariance learning method and system | |
| CN111046883B (en) | Intelligent assessment method and system based on ancient coin image | |
| CN112257544A (en) | High-resolution remote sensing image post-earthquake building detection method based on sparse dictionary | |
| CN109724921B (en) | Painting and calligraphy identification map feature representation method based on spectral imaging technology | |
| Al Mubarok et al. | Handwritten character recognition using hierarchical graph matching |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |