CN114219724A - Multispectral image generation method, terminal device and computer-readable storage medium - Google Patents
Multispectral image generation method, terminal device and computer-readable storage medium Download PDFInfo
- Publication number
- CN114219724A CN114219724A CN202111412206.1A CN202111412206A CN114219724A CN 114219724 A CN114219724 A CN 114219724A CN 202111412206 A CN202111412206 A CN 202111412206A CN 114219724 A CN114219724 A CN 114219724A
- Authority
- CN
- China
- Prior art keywords
- detection
- reconstructed
- image
- spectral
- similar
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 72
- 238000001514 detection method Methods 0.000 claims abstract description 147
- 230000003595 spectral effect Effects 0.000 claims abstract description 42
- 239000013598 vector Substances 0.000 claims description 56
- 238000001228 spectrum Methods 0.000 claims description 47
- 238000004590 computer program Methods 0.000 claims description 21
- 238000004364 calculation method Methods 0.000 claims description 14
- 238000012937 correction Methods 0.000 claims description 10
- 238000000638 solvent extraction Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 abstract description 5
- 238000005286 illumination Methods 0.000 description 14
- 230000008569 process Effects 0.000 description 13
- 238000010586 diagram Methods 0.000 description 9
- 238000012847 principal component analysis method Methods 0.000 description 9
- 238000000701 chemical imaging Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 8
- 238000003384 imaging method Methods 0.000 description 6
- 230000004044 response Effects 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 5
- 238000005192 partition Methods 0.000 description 5
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000009499 grossing Methods 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 238000004088 simulation Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000001678 irradiating effect Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 238000000985 reflectance spectrum Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F18/2135—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10052—Images from lightfield camera
Abstract
The application is applicable to the technical field of image processing, and provides a multispectral image generation method, a terminal device and a computer-readable storage medium, wherein the multispectral image generation method comprises the following steps: acquiring a detection image; carrying out spatial classification according to the spectral similarity of the detected images to determine a similar area; reconstructing a reconstructed multi-spectral curve of the similar region according to the detection average value of the similar region; and correcting the reconstructed multi-spectral curve according to the reconstructed multi-spectral curve and the detection image of the similar region to obtain a target multi-spectral image of the similar region. The similar region in the detected image is determined, the similar region is reconstructed for one time to obtain a reconstructed multi-spectral curve corresponding to the similar region, the calculated amount can be effectively reduced, the reconstructed multi-spectral curve is corrected based on the detected image, and the accuracy of the generated multi-spectral image is effectively improved.
Description
Technical Field
The present application belongs to the field of image processing technologies, and in particular, to a multispectral image generation method, a terminal device, and a computer-readable storage medium.
Background
The multispectral image is an image of an object reflecting and transmitting light in multiple wavelength bands. The traditional multispectral imaging system can perform spectral scanning by utilizing devices such as a grating and the like, so as to obtain a multispectral image. The traditional multispectral imaging system has the defects of complex system structure and high price. In order to save cost and simplify the system, a multispectral imaging system based on LED illumination is proposed at present, which does not need to be provided with a scanning moving part, is convenient for switching wave bands and can avoid the phenomenon of metamerism.
The existing multispectral imaging system based on LED illumination usually uses LED illumination of a few wave bands, sequentially collects images under different LED illumination, obtains reflection spectra of more wave bands by combining a spectrum reconstruction algorithm, and is simple in structure and convenient to implement. However, due to the complex algorithm and the large calculation amount in the process of spectrum reconstruction, a large amount of calculation resources and time need to be consumed, and the problems of low imaging efficiency, poor real-time performance and the like are caused.
Disclosure of Invention
In view of this, embodiments of the present application provide a multispectral image generation method, a terminal device, and a computer-readable storage medium, so as to solve the problems of low imaging efficiency and poor real-time performance caused by a large amount of computing resources and time consumed in the current spectral reconstruction process.
In a first aspect, an embodiment of the present application provides a method for generating a multispectral image, including:
acquiring a detection image;
carrying out spatial classification according to the spectral similarity of the detected images to determine a similar area;
reconstructing a reconstructed multi-spectral curve of the similar region according to the detection average value of the similar region;
and correcting the reconstructed multi-spectral curve according to the reconstructed multi-spectral curve and the detection image of the similar region to obtain a target multi-spectral image of the similar region.
Optionally, the performing spatial classification according to the spectral similarity of the detected image to determine a similar region includes:
setting a reference point and acquiring a reference point detection vector of the reference point;
acquiring detection vectors of all pixel points in the detection image except the reference point;
calculating cosine similarity between the detection vector of each pixel point and the reference point detection vector;
and traversing all pixel points of the detection image, and determining the pixel points as the pixel points in the similar area of the reference point when the cosine similarity between the detection vectors of the pixel points and the reference point detection vectors is smaller than a preset similar threshold.
Optionally, the performing spatial classification according to the spectral similarity of the detected image to determine a similar region further includes:
if the similar region has residual pixel points, resetting the reference points from the residual pixel points;
acquiring a reference point detection vector of the reset reference point;
calculating cosine similarity between the detection vectors of the residual pixel points and the reference point detection vectors of the reset reference point;
and traversing the residual pixel points, and determining the pixel points as the pixel points in the similar area of the reset reference point when the cosine similarity between the detection vectors of the residual pixel points and the reference point detection vectors of the reset reference point is less than a preset similar threshold.
Optionally, the reconstructing a reconstructed multi-spectral curve of the similar region according to the detected average value of the similar region includes:
calculating the detection average value of the similar area;
and reconstructing the similar region according to the detection average value to obtain a reconstructed multi-spectral curve of the similar region.
Optionally, the modifying the reconstructed multi-spectral curve according to the reconstructed multi-spectral curve and the detected image of the similar region to obtain the target multi-spectral image of the similar region includes:
determining a reconstructed spectrum value of each point in the similar region according to the reconstructed multi-spectrum curve of the similar region;
determining a corresponding actual detection value according to the detection image;
and correcting the reconstructed spectrum curve according to the reconstructed spectrum value of each point and the corresponding actual detection value.
Optionally, the modifying the reconstructed spectrum curve according to the reconstructed spectrum value of each point and the corresponding actual detection value includes:
calculating the calculation detection value of each point according to the reconstruction spectrum value of each point;
and performing linear segmented correction on the reconstructed spectral curve according to the ratio of the calculated detection value to the actual detection value of each point.
Optionally, if the detection image has a plurality of similar regions, the modifying the reconstructed multi-spectral curve according to the reconstructed multi-spectral curve of the similar regions and the detection image to obtain the target multi-spectral image of the similar regions includes:
and correcting the reconstructed multi-spectral curve of each similar region to obtain a target multi-spectral curve corresponding to the detected image.
In a second aspect, an embodiment of the present application provides a terminal device, including:
the acquisition module is used for acquiring a detection image;
the partitioning module is used for carrying out spatial classification according to the spectral similarity of the detected images and determining a similar area;
the reconstruction module is used for reconstructing a reconstructed multi-spectral curve of the similar region according to the detection average value of the similar region;
and the correction module is used for correcting the reconstructed multi-spectral curve according to the reconstructed multi-spectral curve and the detection image of the similar region to obtain a target multi-spectral image of the similar region.
In a third aspect, an embodiment of the present application provides a terminal device, where the terminal device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor, when executing the computer program, implements the method according to the first aspect or any optional manner of the first aspect.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium storing a computer program, which when executed by a processor implements the method according to the first aspect or any alternative manner of the first aspect.
In a fifth aspect, embodiments of the present application provide a computer program product, which, when run on a terminal device, causes the terminal device to perform the method of the first aspect or any alternative manner of the first aspect.
The implementation of the multispectral image generation method, the terminal device, the computer readable storage medium and the computer program product provided by the embodiment of the application has the following beneficial effects:
the similar region in the detected image is determined, the similar region is reconstructed for one time to obtain a reconstructed multi-spectral curve corresponding to the similar region, the calculated amount can be effectively reduced, the reconstructed multi-spectral curve is corrected based on the detected image, and the accuracy of the generated multi-spectral image is effectively improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
Fig. 1 is a block diagram of a multispectral imaging system based on LED illumination according to an embodiment of the present disclosure;
FIG. 2 is a schematic flow chart diagram of a multispectral image generation method provided by an embodiment of the present application;
fig. 3 is a schematic structural diagram of a terminal device according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a terminal device according to another embodiment of the present application;
fig. 5 is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It should be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items. Furthermore, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing or implying relative importance.
It should also be appreciated that reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless expressly specified otherwise.
Referring to fig. 1, fig. 1 is a schematic diagram illustrating a multispectral imaging system based on LED illumination. As shown in FIG. 1, the LED illumination-based multispectral imaging system can include a multiband LED illumination source and detector.
The LED illumination light source generally employs several LED bands to illuminate the detected object, the detector collects reflected/transmitted detection images illuminated on the detected object by different LED bands, and spectral reconstruction is performed on a plurality of detection images collected by the detector (after each LED band illuminates on the detected object, the detector will collect one image), so as to obtain a multispectral image (for example, the number of LED bands of the LED illumination light source is N, the number of bands of the obtained multispectral image is L, where N, L is a positive number, and N < L).
In a specific application, the detector may be a video camera, a still camera, or the like capable of capturing a spectral image.
The existing spectrum reconstruction methods include pseudo-inverse method, wiener estimation method, principal component analysis, polynomial regression, machine learning and the like. These methods can improve the accuracy of the reconstructed multi-spectral image, however, can complicate the overall algorithm and can be computationally more complex.
For example, after the detection light of the LED illumination light source is reflected/transmitted by the detected object, the detection image collected by the detector may be represented as:
wherein, I is the detected value (i.e. the value on the detected image) acquired by the detector, SλIs the reflection spectrum of the detected object (i.e. the multispectral image to be reconstructed), CλIs the spectral response of the detected object (measurable by standard test instruments), PλIs the spectral response of the LED light source (as measured by standard test instruments).
Since the spectral reflectance is usually relatively smooth, the spectral reflectance can be reconstructed by using a principal component analysis method, and therefore, the spectrum to be measured (i.e. the reflectance spectrum of the detected object) can be expressed as:
wherein σkIs a characteristic coefficient, bk(λ) is the trained feature vector.
Combining equation (1) and equation (2) yields:
since the spectral values are all positive values, the constraint term is added:
written in matrix form, i.e. Aσ>0. Increasing the second derivative of the spectrum while taking into account the smoothness of the spectrum
The optimal characteristic coefficient sigma can be calculatedkThe calculation formula is as follows:
argminσ[‖Fσ-I‖2 2+α‖Pσ‖2]，Aσ>0； (4)
the reconstructed multispectral image can be obtained by the formula (2).
It should be noted that, in the above-mentioned reconstruction process, the spectrum of one pixel point on the image is reconstructed, the multispectral curve of the pixel point can be obtained, and to obtain the whole multispectral image, each pixel point needs to be processed once, and if the number of the pixel points is large, the overall calculation amount is very large, a large amount of calculation resources and time need to be consumed, and the problems of low imaging efficiency and poor real-time performance are caused.
In order to solve the above problem, an embodiment of the present application provides a multispectral image generation method, where a similar region in a detected image is determined, and the similar region is reconstructed once to obtain a reconstructed multispectral curve corresponding to the similar region, so that the amount of computation can be effectively reduced, and the reconstructed multispectral curve is corrected based on the detected image, thereby effectively improving the accuracy of the generated multispectral image.
The multispectral image generation method, the terminal device and the computer-readable storage medium provided by the embodiments of the present application will be described in detail below:
referring to fig. 2, fig. 2 is a schematic flowchart of a multispectral image generation method according to an embodiment of the present disclosure. In an embodiment of the present application, an execution subject of the multispectral image generation method may be a terminal device. The terminal device includes, but is not limited to, a mobile phone, a tablet computer, a desktop computer, a server, and other devices with computing capabilities.
It should be noted that the multispectral image generation method provided in the embodiment of the present application refers to a process of irradiating a detected object with N bands of LED light by using an LED illumination light source of the multispectral imaging system based on LED illumination as shown in fig. 1, collecting a detection image formed by reflecting the detected object irradiated with each band of LED light by a detector, and reconstructing multispectral images of L bands based on the detection image. The method is different from the existing multispectral image generation method, and the embodiment of the application improves the process of reconstructing multispectral images of L wave bands based on the detection images, effectively improves the calculation speed of the reconstruction process and improves the generation efficiency of the multispectral images.
Specifically, as shown in fig. 2, the multispectral image generation method may include steps S11 to S14, which are detailed as follows:
s11: and acquiring a detection image.
In this embodiment of the application, the detection image refers to an image acquired by an LED illumination light source that irradiates a detected object with N bands of LED light and then a detector, where the number of the detection images is N, and N is a positive integer. Specifically, N is a positive integer greater than 1.
In the embodiment of the application, the acquired detection image can be sent to the terminal equipment in real time through the detector, so that the terminal equipment can acquire the detection image in real time. Of course, after the detector acquires the detection image, the detector may store the acquired detection image in a memory of the detector, and when an image upload instruction issued by the terminal device is detected, the detection image stored in the memory of the detector is sent to the terminal device, so that the terminal device acquires the detection image.
In each detected image, each pixel has its corresponding detected value (herein, referred to as measured value).
S12: and carrying out spatial classification according to the spectral similarity of the detected images to determine a similar area.
In the embodiment of the application, a reference point detection vector I is formed by selecting a reference point and N detection values corresponding to the reference point in N detection imagesrefWhen detecting a detection vector I at any point in an imagep(i.e. the vector formed by the measured values of the pixels in each detected image) and a reference point detection vector IrefWhen the cosine similarity is smaller than a preset similarity threshold, determining the point as a similar point of the reference point, and classifying the point into a similar area of the reference point.
In the embodiment of the application, a detection vector I is calculatedpDetecting vector I with reference pointrefThe formula for calculating the cosine similarity is as follows:
where sim refers to the detection vector IpDetecting vector I with reference pointrefCosine similarity of (1)refIs a reference point detection vector, IpIs a detection vector for detecting an arbitrary point in the image.
It should be noted that the preset similarity threshold may be set based on the accuracy and computational efficiency of reconstruction.
In an embodiment of the present application, the step S12 may include the following steps:
setting a reference point and acquiring a reference point detection vector of the reference point;
acquiring detection vectors of all pixel points in the detection image except the reference point;
calculating cosine similarity between the detection vector of each pixel point and the reference point detection vector;
and traversing all pixel points of the detection image, and determining the pixel points as the pixel points in the similar area of the reference point when the cosine similarity between the detection vectors of the pixel points and the reference point detection vectors is smaller than a preset similar threshold.
In another embodiment of the present application, the step S12 may further include the following steps:
if the similar region has residual pixel points, resetting the reference points from the residual pixel points;
acquiring a reference point detection vector of the reset reference point;
calculating cosine similarity between the detection vectors of the residual pixel points and the reference point detection vectors of the reset reference point;
and traversing the residual pixel points, and determining the pixel points as the pixel points in the similar area of the reset reference point when the cosine similarity between the detection vectors of the residual pixel points and the reference point detection vectors of the reset reference point is less than a preset similar threshold.
In this embodiment of the present application, a plurality of similar regions may exist in the detection image, after a similar region is determined, another reference point may be set in the remaining region, and then the similar region of this reference point is determined by traversing points except the point determined as the previous similar region until each pixel point in the detection image completes the partition of the similar region.
It should be noted that, after determining a similar region, the terminal device may perform the subsequent steps S13 to S14 on the similar region, and then determine whether other similar regions exist; the steps S13 to S14 may be performed on each similar region after all similar regions in the detected image are determined, which is not limited in this application.
S13: and reconstructing a reconstructed multi-spectral curve of the similar region according to the detection average value of the similar region.
In the embodiment of the application, for each similar region, a detection average value needs to be determined, the detection average value is an average value of detection values of each point in the similar region, and then the similar region is reconstructed according to the detection average value, that is, each point in the similar region uses a corresponding multi-spectral curve after reconstruction, that is, one similar region only needs to be reconstructed once, so that the calculation amount is effectively reduced.
In an embodiment of the present application, the step S13 may include the following steps:
calculating the detection average value of the similar area;
and reconstructing the similar region according to the detection average value to obtain a reconstructed multi-spectral curve of the similar region.
After the similar region is determined, a detection average value of the similar region can be calculated.
Specifically, assuming that the number of points included in the similar region determined in S12 is num, the detection average value of the similar region is calculated as: i ismean＝(Iref+...+Inum)/num。
Thus, the characteristic coefficient sigma of the similar region can be determined by using the following optimization functionopt：
argminσ[‖Fσ-Imean‖2 2+α‖Pσ‖2]。
And obtaining a reconstructed multispectral curve of a similar region:
where S' is the reconstructed spectral value of a point in a similar region, bk(λ) is the trained feature vector.
In the embodiment of the present application, if a plurality of similar regions exist in the detected image, the same operation needs to be performed on each similar region to obtain the reconstructed multi-spectral curve of each similar region.
S14: and correcting the reconstructed multi-spectral curve according to the reconstructed multi-spectral curve and the detection image of the similar region to obtain a target multi-spectral image of the similar region.
In the embodiment of the application, the reconstructed spectrum value of each point in the similar region can be obtained according to the reconstructed multi-spectrum curve of the similar region, the actual detection value of the point can be known according to the detection image, the point is corrected according to the reconstructed spectrum value and the actual detection value of the point, and the process is repeated until each point in the similar region is corrected, so that the final target multi-spectrum image can be obtained.
In specific application, the calculated detection value of each point in the similar region can be inversely calculated through the reconstructed multispectral curve of the similar region, then the ratio of the calculated detection value to the actual detection value of the point is calculated, and linear segmented correction is carried out on the reconstructed spectral value of the point based on the ratio.
Specifically, the modified target multispectral curve may be represented as:
scale(t)＝Ip/Ip‘ t＝1,2,3...,N
wherein, Ip' is a calculated detection value, IpIs the actual detection value, S' (λ) is the reconstructed spectral curve of the point in the similar region, and S (λ) is the corrected target multispectral curve.
Note that L is linearly segmented1,L2,…,LNRegarding the spectral response of the N LED light sources, the spectral response of the LEDs is usually distributed in different bands, and the corresponding scale values are used for correction near the response range, for example, if the spectral response range of the first LED is 400-450nm, L1Corresponds to 450nm, on the basis of which the corresponding scale value can be determined.
It should be noted that, if a plurality of similar regions exist in the detected image, after the reconstructed multispectral curve of each similar region is obtained, the reconstructed multispectral curve needs to be corrected to obtain the target multispectral image of the similar region until the reconstructed multispectral curves of all the similar regions are corrected.
In the embodiment of the application, after the reconstructed spectral curves in the similar regions are subjected to linear piecewise correction, smoothing processing can be performed on the reconstructed spectral curves so as to avoid the situation of sudden change.
It should be noted that, smoothing the reconstructed spectral curve after linear piecewise correction may be implemented by using an existing smoothing algorithm, which is not described herein again.
In an embodiment of the present application, the step S14 may include the following steps:
determining a reconstructed spectrum value of each point in the similar region according to the reconstructed multi-spectrum curve of the similar region;
determining a corresponding actual detection value according to the detection image;
and correcting the reconstructed spectrum curve according to the reconstructed spectrum value of each point and the corresponding actual detection value.
In the embodiment of the present application, after the reconstructed multi-spectral curves in each similar region are corrected, a reconstructed multi-spectral image corresponding to the entire detected image can be obtained, and a multi-spectral image corresponding to the detected image is generated.
In this embodiment of the present application, the modifying the reconstructed spectrum curve according to the reconstructed spectrum value of each point and the corresponding actual detection value includes:
calculating the calculation detection value of each point according to the reconstruction spectrum value of each point;
and performing linear segmented correction on the reconstructed spectral curve according to the ratio of the calculated detection value to the actual detection value of each point.
In order to further explain that the multispectral image generation method provided by the embodiment of the application can ensure the spectral accuracy after imaging on the basis of improving the imaging efficiency, the embodiment of the application also verifies the effect of the multispectral image generation method provided by the embodiment of the application through a simulation experiment. Table 1 shows correlation coefficients reconstructed by the existing principal component analysis method, correlation coefficients of a reconstructed multi-spectral curve obtained by partition optimization in the method provided in the embodiment of the present application, and correlation coefficients of a target spectral image obtained by modification of the reconstructed multi-spectral curve in the method provided in the embodiment of the present application. Table 2 shows the cosine similarity of the image reconstructed by the existing principal component analysis method, the cosine similarity of the reconstructed multispectral curve obtained by partition optimization in the method provided in the embodiment of the present application, and the cosine similarity of the target spectral image obtained by correcting the reconstructed multispectral curve in the method provided in the embodiment of the present application. The difference between the reconstructed spectrum curve and the original spectrum is represented by using a correlation coefficient and cosine similarity, the larger the correlation coefficient is, the smaller the difference between the reconstructed spectrum curve and the original spectrum is, and the smaller the cosine similarity is, the smaller the difference between the reconstructed spectrum curve and the original spectrum is.
Table 1:
table 2:
as can be seen from tables 1 and 2, as the preset similarity threshold increases, the number of the similarity points increases, the accuracy of the optimized reconstructed spectral curve after partitioning decreases, and the accuracy of the corrected target multispectral image increases. In addition, when the preset similarity threshold is set to be 0.8, the spectral difference obtained through optimized calculation after partitioning is large, but the corrected spectral curve is obviously improved, although the preset similarity threshold is large, the accuracy of the reconstructed multispectral curve of the method is also good, but as the preset similarity threshold is increased, the correlation coefficient is gradually reduced, the cosine similarity is gradually increased, and the spectral accuracy tends to be reduced, so that the preset similarity threshold is not recommended to be set to be too large.
To further explain that the multispectral image generation method provided by the embodiment of the present application can effectively improve the calculation efficiency and improve the generation efficiency of the multispectral image, table 3 shows the time consumption and the image quality for reconstructing each pixel point based on the existing principal component analysis method obtained through a simulation experiment, and the comparison data of the time consumption and the image quality when reconstructing by the multispectral image generation method provided by the embodiment of the present application.
In a specific application, the simulation experiment is programmed by Matlab R2014a, and an Intel Core i5-6200U CPU @2.3GHz computer is used for reconstructing 512 x 512 images. In order to measure the image quality, the peak Signal-to-Noise ratio PSNR (Peak Signal Noise ratio) and the cosine similarity of the spectrum of the reconstructed image and the original image are compared. The peak signal-to-noise ratio is obtained by respectively comparing the difference of the images of each wave band and then taking the average value of all the wave bands, wherein the higher the value of the peak signal-to-noise ratio is, the higher the quality of the reconstructed image is; the cosine similarity of the spectrum curve is to calculate the cosine angle of the spectrum curve at the same space point, and then calculate the average value of different points, wherein the smaller the numerical value, the closer the reconstructed spectrum curve is to the original spectrum (i.e. the higher the quality of the reconstructed image).
Table 3:
as shown in table 3, it takes 1162.2s to reconstruct each pixel point based on the existing principal component analysis method, but when the multispectral image generation method provided in the embodiment of the present application is used for reconstruction, when the preset similarity threshold is set to 0.1, the time consumption is only 342s, which is 29.4% of the time consumption of the method for reconstructing each pixel point based on the existing principal component analysis method; when the preset similarity threshold is 0.3, the consumed time is only 74.5s, which is 6.4% of the consumed time of the method for reconstructing each pixel point based on the existing principal component analysis method, and when the preset similarity threshold is 0.5, the consumed time is 64s, which is 5.5% of the consumed time of the method for reconstructing each pixel point based on the existing principal component analysis method. As can also be seen from the data in table 3, the image quality of the multispectral image obtained by the multispectral image generation method provided in the embodiment of the present application is superior to the image quality of the multispectral image obtained by reconstructing each pixel point based on the existing principal component analysis method. That is to say, the multispectral image generation method provided by the embodiment of the application can not only improve the calculation speed of image reconstruction and improve the imaging efficiency and real-time performance, but also improve the image quality of the generated multispectral image and improve the image precision.
As can be seen from the above, in the multispectral image generation method provided in the embodiment of the present application, the similar region in the detected image is determined, and the reconstructed multispectral curve corresponding to the similar region is obtained by reconstructing the similar region once, so that the amount of calculation can be effectively reduced, and the reconstructed multispectral curve is corrected based on the detected image, so that the accuracy of the generated multispectral image is effectively improved.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.
Based on the multispectral image generation method provided by the embodiment, the embodiment of the invention further provides an embodiment of the terminal device for realizing the embodiment of the method.
Referring to fig. 3, fig. 3 is a schematic structural diagram of a terminal device according to an embodiment of the present application. In the embodiment of the present application, each unit included in the terminal device is configured to execute each step in the embodiment corresponding to fig. 2. Please refer to fig. 2 and the related description of the embodiment corresponding to fig. 2. For convenience of explanation, only the portions related to the present embodiment are shown. As shown in fig. 3, the terminal device 30 includes: an acquisition module 31, a partitioning module 32, a reconstruction module 33, and a modification module 34.
The acquisition module 31 is used for acquiring a detection image.
The partitioning module 32 is configured to perform spatial classification according to the spectral similarity of the detected images, and determine a similar region.
The reconstruction module 33 is configured to reconstruct a reconstructed multi-spectral curve of the similar region according to the detected average value of the similar region.
The correction module 34 is configured to correct the reconstructed multi-spectral curve according to the reconstructed multi-spectral curve and the detected image of the similar region, so as to obtain a target multi-spectral image of the similar region.
Optionally, the partition unit 32 is specifically configured to:
setting a reference point and acquiring a reference point detection vector of the reference point;
acquiring detection vectors of all pixel points in the detection image except the reference point;
calculating cosine similarity between the detection vector of each pixel point and the reference point detection vector;
and traversing all pixel points of the detection image, and determining the pixel points as the pixel points in the similar area of the reference point when the cosine similarity between the detection vectors of the pixel points and the reference point detection vectors is smaller than a preset similar threshold.
Optionally, the partition unit 32 is further specifically configured to:
the spatial classification is performed according to the spectral similarity of the detected image, and the similar region is determined, further comprising:
if the similar region has residual pixel points, resetting the reference points from the residual pixel points;
acquiring a reference point detection vector of the reset reference point;
calculating cosine similarity between the detection vectors of the residual pixel points and the reference point detection vectors of the reset reference point;
and traversing the residual pixel points, and determining the pixel points as the pixel points in the similar area of the reset reference point when the cosine similarity between the detection vectors of the residual pixel points and the reference point detection vectors of the reset reference point is less than a preset similar threshold.
Optionally, the reconstruction module 33 is specifically configured to:
calculating the detection average value of the similar area;
and reconstructing the similar region according to the detection average value to obtain a reconstructed multi-spectral curve of the similar region.
Optionally, the modification module 34 is specifically configured to:
determining a reconstructed spectrum value of each point in the similar region according to the reconstructed multi-spectrum curve of the similar region;
determining a corresponding actual detection value according to the detection image;
and correcting the reconstructed spectrum curve according to the reconstructed spectrum value of each point and the corresponding actual detection value.
It should be noted that, for the above contents of information interaction, execution process, and the like between the modules/units, the specific functions and technical effects thereof are based on the same concept as those of the embodiment of the method of the present application, and specific reference may be made to the part of the embodiment of the method, which is not described herein again.
Fig. 4 is a schematic structural diagram of a terminal device according to another embodiment of the present application. As shown in fig. 4, the terminal device 4 provided in this embodiment includes: a processor 40, a memory 41 and a computer program 42 stored in said memory 41 and executable on said processor 40, such as a program for cooperative control of a multi-agent system. The processor 40, when executing the computer program 42, implements the steps in the various multispectral image generation method embodiments described above, such as S11-S14 shown in fig. 2. Alternatively, the processor 40, when executing the computer program 42, implements the functions of the modules/units in the terminal device embodiments, such as the functions of the units 31 to 34 shown in fig. 3.
Illustratively, the computer program 42 may be partitioned into one or more modules/units that are stored in the memory 41 and executed by the processor 40 to accomplish the present application. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program 42 in the terminal device 4. For example, the computer program 42 may be divided into units/modules, and the specific functions of each unit/module are described with reference to the corresponding embodiment in fig. 3, which is not described herein again.
The terminal device may include, but is not limited to, a processor 40, a memory 41. Those skilled in the art will appreciate that fig. 4 is merely an example of a terminal device 4 and does not constitute a limitation of terminal device 4 and may include more or fewer components than shown, or some components may be combined, or different components, e.g., the terminal device may also include input-output devices, network access devices, buses, etc.
The Processor 40 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 41 may be an internal storage unit of the terminal device 4, such as a hard disk or a memory of the terminal device 4. The memory 41 may also be an external storage device of the terminal device 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the terminal device 4. Further, the memory 41 may also include both an internal storage unit and an external storage device of the terminal device 4. The memory 41 is used for storing the computer program and other programs and data required by the terminal device. The memory 41 may also be used to temporarily store data that has been output or is to be output.
The embodiment of the application also provides a computer readable storage medium. Referring to fig. 5, fig. 5 is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present disclosure, as shown in fig. 5, a computer program 51 is stored in the computer-readable storage medium 5, and when the computer program 51 is executed by a processor, the multispectral image generation method can be implemented.
The embodiment of the application provides a computer program product, and when the computer program product runs on a terminal device, the multispectral image generation method can be realized when the terminal device executes the computer program product.
It is obvious to those skilled in the art that, for convenience and simplicity of description, the foregoing division of the functional units and modules is merely used as an example, and in practical applications, the foregoing function distribution may be performed by different functional units and modules as needed, that is, the internal structure of the terminal device is divided into different functional units or modules to perform all or part of the above-described functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the description of each embodiment has its own emphasis, and parts that are not described or illustrated in a certain embodiment may refer to the description of other embodiments.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.
Claims (10)
1. A method of generating a multi-spectral image, comprising:
acquiring a detection image;
carrying out spatial classification according to the spectral similarity of the detected images to determine a similar area;
reconstructing a reconstructed multi-spectral curve of the similar region according to the detection average value of the similar region;
and correcting the reconstructed multi-spectral curve according to the reconstructed multi-spectral curve and the detection image of the similar region to obtain a target multi-spectral image of the similar region.
2. The method for generating multispectral images according to claim 1, wherein the spatially classifying according to the spectral similarity of the detected images to determine the similar regions comprises:
setting a reference point and acquiring a reference point detection vector of the reference point;
acquiring detection vectors of all pixel points in the detection image except the reference point;
calculating cosine similarity between the detection vector of each pixel point and the reference point detection vector;
and traversing all pixel points of the detection image, and determining the pixel points as the pixel points in the similar area of the reference point when the cosine similarity between the detection vectors of the pixel points and the reference point detection vectors is smaller than a preset similar threshold.
3. The method for generating multispectral images according to claim 2, wherein the step of spatially classifying the images according to their spectral similarity to determine similar regions further comprises:
if the similar region has residual pixel points, resetting the reference points from the residual pixel points;
acquiring a reference point detection vector of the reset reference point;
calculating cosine similarity between the detection vectors of the residual pixel points and the reference point detection vectors of the reset reference point;
and traversing the residual pixel points, and determining the pixel points as the pixel points in the similar area of the reset reference point when the cosine similarity between the detection vectors of the residual pixel points and the reference point detection vectors of the reset reference point is less than a preset similar threshold.
4. The method for multispectral image generation as recited in claim 1, wherein said reconstructing a reconstructed multispectral curve of the similar region from the detected mean of the similar region comprises:
calculating the detection average value of the similar area;
and reconstructing the similar region according to the detection average value to obtain a reconstructed multi-spectral curve of the similar region.
5. The method for generating multispectral images according to claim 1, wherein the modifying the reconstructed multispectral curve according to the reconstructed multispectral curve and the detected image of the similar region to obtain the target multispectral image of the similar region comprises:
determining a reconstructed spectrum value of each point in the similar region according to the reconstructed multi-spectrum curve of the similar region;
determining a corresponding actual detection value according to the detection image;
and correcting the reconstructed spectrum curve according to the reconstructed spectrum value of each point and the corresponding actual detection value.
6. The method for generating multispectral images as recited in claim 1, wherein the modifying the reconstructed spectral curve according to the reconstructed spectral values and the corresponding actual detection values at the respective points comprises:
calculating the calculation detection value of each point according to the reconstruction spectrum value of each point;
and performing linear segmented correction on the reconstructed spectral curve according to the ratio of the calculated detection value to the actual detection value of each point.
7. The method for generating the multispectral image according to any one of claims 1 to 6, wherein if there are a plurality of similar regions in the detection image, the modifying the reconstructed multispectral curve according to the reconstructed multispectral curve and the detection image of the similar regions to obtain the target multispectral image of the similar regions comprises:
and correcting the reconstructed multi-spectral curve of each similar region to obtain a target multi-spectral curve corresponding to the detected image.
8. A terminal device, comprising:
the acquisition module is used for acquiring a detection image;
the partitioning module is used for carrying out spatial classification according to the spectral similarity of the detected images and determining a similar area;
the reconstruction module is used for reconstructing a reconstructed multi-spectral curve of the similar region according to the detection average value of the similar region;
and the correction module is used for correcting the reconstructed multi-spectral curve according to the reconstructed multi-spectral curve and the detection image of the similar region to obtain a target multi-spectral image of the similar region.
9. A terminal device, characterized in that the terminal device comprises a processor, a memory and a computer program stored in the memory and executable on the processor, the processor implementing the method according to any of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111412206.1A CN114219724A (en) | 2021-11-25 | 2021-11-25 | Multispectral image generation method, terminal device and computer-readable storage medium |
| PCT/CN2021/137612 WO2023092707A1 (en) | 2021-11-25 | 2021-12-13 | Multispectral image generation method, terminal device and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111412206.1A CN114219724A (en) | 2021-11-25 | 2021-11-25 | Multispectral image generation method, terminal device and computer-readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114219724A true CN114219724A (en) | 2022-03-22 |
Family
ID=80698233
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111412206.1A Pending CN114219724A (en) | 2021-11-25 | 2021-11-25 | Multispectral image generation method, terminal device and computer-readable storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN114219724A (en) |
| WO (1) | WO2023092707A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116713892A (en) * | 2023-08-10 | 2023-09-08 | 北京特思迪半导体设备有限公司 | Endpoint detection method and apparatus for wafer film grinding |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8805083B1 (en) * | 2010-03-21 | 2014-08-12 | Jeffrey M. Sieracki | System and method for discriminating constituents of image by complex spectral signature extraction |
| US8811754B2 (en) * | 2011-08-29 | 2014-08-19 | Lawrence Livermore National Security, Llc | Spatial clustering of pixels of a multispectral image |
| CN112418314A (en) * | 2020-11-23 | 2021-02-26 | 广东弓叶科技有限公司 | Threshold setting method and device in spectrum similarity matching classification |
| CN112818794B (en) * | 2021-01-25 | 2022-03-04 | 哈尔滨工业大学 | Hyperspectral remote sensing image generation method based on progressive space-spectrum combined depth network |
| CN113670440B (en) * | 2021-07-21 | 2023-11-10 | 杭州电子科技大学 | Compression spectrum imaging method based on self-adaptive dictionary |
-
2021
- 2021-11-25 CN CN202111412206.1A patent/CN114219724A/en active Pending
- 2021-12-13 WO PCT/CN2021/137612 patent/WO2023092707A1/en unknown
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116713892A (en) * | 2023-08-10 | 2023-09-08 | 北京特思迪半导体设备有限公司 | Endpoint detection method and apparatus for wafer film grinding |
| CN116713892B (en) * | 2023-08-10 | 2023-11-10 | 北京特思迪半导体设备有限公司 | Endpoint detection method and apparatus for wafer film grinding |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2023092707A1 (en) | 2023-06-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhang et al. | SR-SIM: A fast and high performance IQA index based on spectral residual | |
| US9123103B2 (en) | Method and apparatus for image denoising with three-dimensional block-matching | |
| CN104414646B (en) | Imaging device and its imaging method, image processing apparatus and its image processing method | |
| CN106603941B (en) | HDR image conversion method and system with self-adaptive computation complexity | |
| Kaya et al. | Towards spectral estimation from a single RGB image in the wild | |
| US20240021021A1 (en) | Light source spectrum and multispectral reflectivity image acquisition methods and apparatuses, and electronic device | |
| CN111079764A (en) | Low-illumination license plate image recognition method and device based on deep learning | |
| KR102231496B1 (en) | Methods and apparatus for improved 3-d data reconstruction from stereo-temporal image sequences | |
| JP2014027597A (en) | Image processor, object identification device, and program | |
| KR20160132778A (en) | Measuring apparatus, measuring method, and storage medium | |
| CN114219724A (en) | Multispectral image generation method, terminal device and computer-readable storage medium | |
| CN117496560B (en) | Fingerprint line identification method and device based on multidimensional vector | |
| CN113609907B (en) | Multispectral data acquisition method, device and equipment | |
| Shyam et al. | Lightweight hdr camera isp for robust perception in dynamic illumination conditions via fourier adversarial networks | |
| Yang et al. | No-reference image quality assessment based on sparse representation | |
| CN111540042A (en) | Method, device and related equipment for three-dimensional reconstruction | |
| Xiao et al. | Fast ℓ 1-minimization algorithm for robust background subtraction | |
| CN111179245A (en) | Image quality detection method, device, electronic equipment and storage medium | |
| KR101720881B1 (en) | Image denoising method based on non-local means algorithm using principal components analysis and image processing apparatus using the same | |
| CN113812137B (en) | Image processing apparatus, image reading apparatus, image processing method, and recording medium | |
| CN113297977A (en) | Living body detection method and device and electronic equipment | |
| CN113340816A (en) | Light source spectrum and multispectral reflectivity image acquisition method and device and electronic equipment | |
| JPWO2019012647A1 (en) | Image processing apparatus, image processing method, and computer program | |
| Ramesh et al. | Real-time, low-latency image processing with high throughput on a multi-core SoC | |
| KR20130138435A (en) | Image enhancement method and device by brightness-contrast improvement |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |