CN116168226A - Intraoperative tumor rapid classification system based on hyperspectral technology - Google Patents

Abstract

The invention discloses an intraoperative tumor rapid classification system based on hyperspectral technology, which comprises: the image preprocessing module is used for preprocessing the acquired hyperspectral images of a plurality of frequency bands of the tumor lesion tissues; the tumor lesion area identification module is used for selecting a plurality of hyperspectral images with optimal frequencies corresponding to the hyperspectral images respectively, fusing the hyperspectral images to obtain fused spectrum images, detecting tumor targets based on the images, and identifying tumor lesion areas; the tumor lesion type recognition module is used for sampling the recognized tumor lesion area by multiple pixel points, acquiring the frequency characteristic of each preferred frequency of each sampling point, respectively inputting the frequency characteristic into the benign and malignant tumor recognition models, respectively outputting benign and malignant likelihood matrixes, and recognizing the tumor lesion type. The invention judges benign and malignant tumors based on hyperspectral images of tumor lesion tissues, realizes rapid classification of tumor lesion types in operation, shortens rapid recognition time, improves recognition efficiency and improves accuracy of recognition results.

Description

Intraoperative tumor rapid classification system based on hyperspectral technology

Technical Field

The invention relates to the application field of hyperspectral technology, in particular to a hyperspectral technology-based intraoperative tumor rapid classification system.

Background

It is known that tumor is one of diseases with extremely high incidence, and the application range of operation treatment is wide, and the mode is mature, so that the tumor is one of the mainstream treatment modes. The preferential surgery requires defining the nature of tumor lesions in the surgery to determine whether to perform resection of the patient's organ or to determine the scope of the surgery and to determine the next surgical procedure in which the existing primary means of tumor detection include rapid intraoperative frozen pathological examinations, which are generally only used for the identification of benign and malignant tumors.

In the prior art, the main steps of rapid freezing pathological examination in operation comprise tissue freezing, slicing, fixing, staining, sealing and microscopic diagnosis, and the diagnosis is obtained by combining visual observation of a pathologist with empirical knowledge. Under the general condition, a frozen section diagnosis report is generally obtained within 30 minutes after a surgical specimen is sent to a pathology department, the process takes a long time, the waiting time can be prolonged, the burden of a patient is further increased, medical resources are occupied, a pathologist manually judges the nature of tumor lesions according to self experience knowledge, the diagnosis result is greatly influenced by personal factors of the doctor, and the accuracy of the diagnosis result is poor.

Disclosure of Invention

Aiming at the defects of the prior art, the invention provides a hyperspectral technology-based intraoperative tumor rapid classification system which is used for judging benign and malignant tumors based on hyperspectral images of tumor lesion tissues, realizing rapid classification of tumor lesion types in an intraoperative process, reducing rapid recognition time, improving recognition efficiency and improving accuracy of recognition results.

In order to achieve the above purpose, the present invention provides the following technical solutions:

the invention provides an intraoperative tumor rapid classification system based on hyperspectral technology.

An intraoperative tumor rapid classification system based on hyperspectral technology, comprising:

the image preprocessing module is used for preprocessing the acquired hyperspectral images of a plurality of frequency bands of the tumor lesion tissues;

the tumor lesion area identification module is used for selecting a plurality of hyperspectral images with optimal frequencies corresponding to the hyperspectral images respectively, fusing the hyperspectral images to obtain fused spectrum images, detecting tumor targets based on the fused spectrum images, and identifying tumor lesion areas;

the tumor lesion type recognition module is used for sampling the recognized tumor lesion area by multiple pixel points, acquiring the frequency characteristic of each preferred frequency of each sampling point, respectively inputting the frequency characteristic of each sampling point into the benign tumor recognition model and the malignant tumor recognition model, and respectively outputting a benign probability matrix and a malignant probability matrix; based on the benign probability matrix and the malignant probability matrix, the type of neoplastic lesion is identified and determined.

According to a further technical scheme, preprocessing is carried out on the obtained hyperspectral image, and the method comprises the following steps:

removing the images of the invalid frequency bands in the hyperspectral images, and halving the gray values of the pixels in the hyperspectral images of each frequency band, wherein the gray values of the pixels are larger than the gray values of the pixels of the preset values, so as to obtain an initial preprocessing hyperspectral image;

removing the images of the invalid frequency bands in the hyperspectral reflection images, and performing difference on the initial preprocessed hyperspectral images and the pixel points corresponding to the hyperspectral reflection images one by one to obtain the hyperspectral images after preprocessing.

Further, the determining of the preferred frequency includes:

acquiring hyperspectral images of a plurality of frequency bands of tumor lesion tissues, and cutting the hyperspectral images to obtain hyperspectral images of a lesion region and a surrounding normal region;

sampling hyperspectral images of a lesion area by multiple pixel points, and extracting one-dimensional spectrum characteristic data of each sampling point;

carrying out multi-pixel sampling on the hyperspectral image in the normal region, and extracting one-dimensional spectrum characteristic data of each sampling point;

and carrying out random pairwise combination on the one-dimensional spectrum characteristic data of the plurality of sampling points in the lesion area and the one-dimensional spectrum characteristic data of the plurality of sampling points in the normal area to obtain differences, carrying out absolute value summation on a plurality of groups of differences under each frequency to obtain the sum of absolute values of a plurality of groups of differences under each frequency, and selecting the frequency corresponding to the sum of the maximum absolute values as the preferred frequency.

Further technical proposal, fusion to obtain a fusion spectrum image, specifically comprises: and respectively fusing gray images corresponding to the selected hyperspectral images with the preferred frequencies as R, G, B channels of the fused image, so as to realize the fusion of the images and obtain the fused spectral image.

According to a further technical scheme, the fusion spectral image-based tumor target detection and identification of a tumor lesion area is specifically implemented by the following steps:

based on the fusion spectrum image, performing tumor target detection by using a target detection model which is completed through training, identifying a tumor lesion area, and marking the tumor lesion area through a target detection anchor frame;

the target detection model adopts a yolov3 target detection algorithm, and a plurality of fusion spectral images of the artificially marked tumor lesion areas are used as a training sample set for training, so that a trained target detection model is obtained.

According to a further technical scheme, the identified tumor lesion area is sampled by multiple pixels, and the frequency characteristics of each preferable frequency of each sampling point are obtained, specifically:

and determining a tumor lesion area marked by the target detection anchor frame, taking the anchor frame center as a datum point, taking the basic point as a 0 # point, taking the points from the 0 # point to the midpoint of the anchor frame boundary as 1-4 # points, taking the points from the 1-4 # points to the midpoint of the 0 # points as 5-8 # points, and taking the 8 # points as 1-8 # points for multi-pixel point sampling to obtain the frequency characteristic of each preferred frequency of each sampling point.

According to a further technical scheme, the benign tumor identification model is connected with a full connection layer and a classifier by adopting a convolutional neural network and is used for identifying benign tumors and other benign tumors;

the training process of the benign tumor recognition model comprises the following steps: and training by taking the one-dimensional frequency characteristic data of a plurality of sampling points of the hyperspectral image of the benign tumor lesion area as a training sample data set to obtain a benign tumor or other benign tumor identification model.

According to a further technical scheme, the malignant tumor recognition model adopts a convolutional neural network and a classifier and is used for identifying malignant tumors and other malignant tumors;

the training process of the malignant tumor recognition model comprises the following steps: and training by taking the one-dimensional frequency characteristic data of a plurality of sampling points of the hyperspectral image of the malignant tumor lesion area as a training sample data set to obtain a malignant tumor or other malignant tumor recognition model.

According to a further technical scheme, the type of the tumor lesion is identified and determined based on the benign probability matrix and the malignant probability matrix, and the method specifically comprises the following steps:

based on the benign probability matrix and the malignant probability matrix, a probability matrix analysis is performed:

if the benign probability matrix is benign and the malignant probability matrix is other, determining that the type of the final tumor lesion is a benign tumor;

if the benign probability matrix is other and the malignant probability matrix is malignant, determining the type of the final tumor lesion as malignant;

if the benign probability matrix is other and the malignant probability matrix is also other, determining that the identified tumor lesion area is normal tissue or other non-tumor lesions;

if the benign probability matrix is benign and the malignant probability matrix is malignant, then subsequent processing is still required.

According to a further technical scheme, the subsequent treatment is as follows: and inputting the one-dimensional frequency characteristic data of the identified tumor lesion area into the trained classification model, and determining the type of the tumor lesion.

The technical scheme has the following beneficial effects:

1. the intraoperative tumor rapid classification system based on the hyperspectral technology provided by the invention is used for judging benign and malignant tumors based on hyperspectral images of tumor lesion tissues, realizing rapid classification of tumor lesion types in an operation, reducing rapid recognition time, improving recognition efficiency and improving accuracy of recognition results.

2. According to the scheme, a complicated slice manufacturing process is not needed, hyperspectral images of the cut edges or the surfaces of the resected tissues are directly obtained, and all operations can be completed in an operating room, so that the time for rapid diagnosis in a tumor operation is reduced from 30 minutes to 10 minutes, and the time cost is greatly saved.

3. The system does not need artificial judgment and recognition, the final recognition result is not influenced by personal factors of doctors, the accuracy and objectivity of the recognition result are improved, and the labor cost and medical resources are saved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention.

FIG. 1 is a schematic diagram of a system for rapid classification of intraoperative tumors according to an embodiment of the present invention;

FIG. 2 is a flow chart showing an overall procedure of a rapid intraoperative tumor classification system according to the first embodiment of the invention;

FIG. 3 is a flowchart of preprocessing a hyperspectral image according to an embodiment of the present invention;

fig. 4 is a flowchart of determining a preferred frequency band according to a first embodiment of the present invention.

Detailed Description

It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the invention. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the present invention. As used herein, the singular is also intended to include the plural unless the context clearly indicates otherwise, and furthermore, it is to be understood that the terms "comprises" and/or "comprising" when used in this specification are taken to specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof.

The embodiment provides a hyperspectral technology-based intraoperative tumor rapid classification system, which is shown in fig. 1 and comprises an image preprocessing module, a tumor lesion area identification module and a tumor lesion type identification module:

the image preprocessing module is used for preprocessing the acquired hyperspectral images of a plurality of frequency bands of the tumor lesion tissues;

the tumor lesion area identification module is used for selecting a plurality of hyperspectral images with optimal frequencies corresponding to the hyperspectral images respectively, fusing the hyperspectral images to obtain fused spectrum images, detecting tumor targets based on the fused spectrum images, and identifying tumor lesion areas;

the tumor lesion type recognition module is used for sampling the recognized tumor lesion area by multiple pixel points, acquiring the frequency characteristic of each preferred frequency of each sampling point, respectively inputting the frequency characteristic of each sampling point into the benign tumor recognition model and the malignant tumor recognition model, and respectively outputting a benign probability matrix and a malignant probability matrix; based on the benign probability matrix and the malignant probability matrix, the type of neoplastic lesion is identified and determined.

In this embodiment, as shown in fig. 2, first, a hyperspectral image is preprocessed by an image preprocessing module. Before pretreatment, a plurality of hyperspectral images of tumor lesion tissues in operation are acquired, specifically, hyperspectral images of a plurality of frequency bands are acquired by simultaneously imaging the slice in a plurality of continuous and subdivided spectral bands in the ultraviolet, visible light, near infrared and mid-infrared regions of the electromagnetic spectrum through an imaging spectrometer by utilizing a remote sensing technology. The hyperspectral image is obtained while the hyperspectral image of tumor lesion tissue in operation is obtained, the hyperspectral image is directly taken by a hyperspectral camera, and the hyperspectral reflected image is obtained by shooting a beam of light emitted by the hyperspectral camera after the reflected light is reflected by the tissue.

Thereafter, the acquired hyperspectral image is preprocessed, as shown in fig. 3, including:

removing an image with an invalid frequency band before 410nm in the hyperspectral image, and halving the gray value of a pixel point in the hyperspectral image with the gray value of the pixel point larger than the gray value set value of 10 in each frequency band to obtain an initial preprocessing hyperspectral image;

removing an image with the frequency of 410nm in the pre-invalid frequency band in the hyperspectral reflection image, and performing difference on the initial preprocessing hyperspectral image and the pixel points corresponding to each other in the hyperspectral reflection image to obtain the hyperspectral image after preprocessing.

According to the preprocessing scheme for correcting the ambient light, the spectrum information of the ambient light is weakened, the spectrum characteristic of the reflected light is enhanced, the interference of the ambient light on the spectrum characteristic of the hyperspectral image is eliminated, and the accuracy of the subsequent image recognition is ensured by using a physical isolation and image processing method.

After the hyperspectral image after pretreatment is obtained, the hyperspectral image is input into a tumor lesion area identification module for identifying the tumor lesion area. Specifically, a plurality of hyperspectral images with optimal frequencies corresponding to the optimal frequencies are selected, fused to obtain a fused spectrum image, tumor target detection is carried out based on the fused spectrum image, and a tumor lesion area is identified.

Before the above steps are performed for the first time, first, a preferred frequency needs to be determined, in this embodiment, 3 frequencies of the hyperspectral image spectral characteristics of the lesion tissue are determined to be significantly different from those of the hyperspectral image of the normal tissue, and these 3 frequencies are used as the preferred frequencies, specifically as shown in fig. 4, and the method includes the following steps:

acquiring hyperspectral images of a plurality of frequency bands of tumor pathological tissue slices, and cutting the hyperspectral images to obtain hyperspectral images of pathological areas and surrounding normal areas;

sampling a hyperspectral image of a lesion area by a plurality of pixel points, in the embodiment, sampling 24 points, if the size of the image does not meet the 24-point sampling requirement, the sampling points can be reduced, and one-dimensional spectrum characteristic data of each sampling point are extracted;

likewise, multi-pixel sampling is performed on the hyperspectral image in the normal region, in this embodiment, 24-point sampling is performed, if the image size does not meet the 24-point sampling requirement, the sampling points can be reduced, and one-dimensional spectrum characteristic data of each sampling point can be extracted;

and carrying out difference making on the one-dimensional spectrum characteristic data of the plurality of sampling points of the lesion area and the one-dimensional spectrum characteristic data of the plurality of sampling points of the normal area in a random pairwise manner, carrying out absolute value summation on the 24 groups of difference values under each frequency, and obtaining the sum of the absolute values of the 24 groups of difference values under each frequency, wherein the difference between the lesion area and the normal area of the image of the frequency corresponding to the sum of the absolute values of 3 before sequencing is obvious, and selecting the 3 frequencies as preferred frequencies. In this example, the preferred frequencies were found to be 640nm, 550nm, 470nm, respectively.

After the preferred frequencies are determined, selecting hyperspectral images corresponding to the 3 preferred frequencies respectively, and fusing the 3 hyperspectral images to obtain a fused spectrum image. The fusion is to use hyperspectral software or MATLAB programming to fuse the gray images corresponding to the hyperspectral images with the three selected frequencies as R, G, B channels of the fusion image respectively, so as to realize the fusion of the images and obtain the fusion spectral image.

Based on the fusion spectrum image, a trained target detection model is utilized to detect a tumor target, a tumor lesion area is identified, meanwhile, aiming at the identified tumor lesion area, a plurality of anchor frames are marked when the target detection is considered, the possibility that an object in a marked area is a detected object is given out when the detection area is marked by the anchor frames, the possibility is obtained by calculation of a target detection algorithm, the tumor lesion area with the possibility of marking the target detection anchor frames being larger than 0.85 is screened, and the anchor frames are marked one by one. The target detection model adopts a yolov3 target detection algorithm, and takes a fusion spectrum image of a manually marked tumor lesion area as a training sample for training, so that the trained target detection model is obtained.

In practice, there may be an error in detecting a tumor target by the target detection model, that is, the target detection model cannot identify a tumor lesion area, and at this time, the method further includes: judging whether the target detection model detects a tumor lesion area, and if not, outputting the tumor lesion area marked by the anchor frame by the target detection model; if no tumor lesion area is detected, the target detection model outputs an original fusion spectrum image, the tumor lesion area in the image is manually selected, the most suitable anchor frames are manually selected and numbered one by one, and the situation that the target detection model recognizes errors is avoided through manual selection.

The tumor lesion area marked by the anchor frame is input into a tumor lesion type recognition module, the tumor lesion type recognition module carries out multi-pixel point sampling on the recognized tumor lesion area, the frequency characteristic of each preferable frequency of each sampling point is obtained, the frequency characteristic of each sampling point is respectively input into a benign tumor recognition model and a malignant tumor recognition model, a benign probability matrix and a malignant probability matrix are respectively output, the type of tumor lesion is recognized and determined based on the benign probability matrix and the malignant probability matrix, and finally the type of tumor lesion is output.

Specifically, a tumor lesion area marked by a target detection anchor frame is determined, the center of the anchor frame is taken as a reference point, the basic point is a number 0 point, the midpoint from the number 0 point to the boundary of the anchor frame is a number 1-4 point, the midpoint from the number 1-4 point to the number 0 point is a number 5-8 point, 8 points are taken as the total, multi-pixel point sampling is carried out, the frequency characteristics of each preferable frequency of each sampling point are obtained, namely, the one-dimensional frequency characteristic data of 8 sampling points are obtained, and the one-dimensional frequency characteristic data comprises 3 frequency characteristic data of the sampling points.

The one-dimensional frequency characteristic data of each sampling point is respectively input into a benign tumor recognition model and a malignant tumor recognition model, and a benign probability matrix and a malignant probability matrix are respectively output.

The benign tumor recognition model adopts a convolutional neural network Densenet201, and a full-connection layer and a Softmax two-classifier are sequentially connected after the Densenet201 and are used for distinguishing benign tumors and other benign tumors, and the training process of the benign tumor recognition model is as follows: and training by taking the one-dimensional frequency characteristic data of a plurality of sampling points of the hyperspectral image of the benign tumor lesion area as a training sample data set to obtain a benign tumor or other benign tumor identification model.

The malignant tumor recognition model adopts a convolutional neural network Densenet101, and a Softmax two-classifier is connected after the Densenet101 and is used for distinguishing malignant tumors and other malignant tumors, and the training process of the malignant tumor recognition model is as follows: and training by taking the one-dimensional frequency characteristic data of a plurality of sampling points of the hyperspectral image of the malignant tumor lesion area as a training sample data set to obtain a malignant tumor or other malignant tumor recognition model.

The likelihood matrix is an output of the Softmax two-classifier, for example, when the Softmax two-classifier is used in a benign tumor recognition model, the likelihood matrix of the output contains two elements: the probability of benign tumor and other probability are taken as the classification output result with the maximum probability; likewise, when Softmax is classified into two categories in the malignancy recognition model, the probability matrix of the output contains two elements: [ malignancy probability, other probability ], taking the highest probability as the classification output result.

In step 4, the type of final tumor lesion is determined based on the benign probability matrix and the malignant probability matrix.

In this embodiment, based on the benign probability matrix and the malignant probability matrix, a probability matrix analysis is performed, that is, the benign probability matrix is c1, the malignant probability matrix is c2, and 0.85 is used as a standard, if the benign probability matrix c1 is benign and the malignant probability matrix c2 is other, the type of the final tumor lesion is determined to be benign tumor; if the benign probability matrix c1 is other and the malignant probability matrix c2 is malignant, determining the type of the final tumor lesion as malignant; if the benign probability matrix c1 is other and the malignant probability matrix c2 is also other, determining the identified tumor lesion area as normal tissue or other non-tumor lesions; if the benign probability matrix c1 is benign and the malignant probability matrix c2 is malignant, the subsequent processing is still needed, that is, the one-dimensional frequency characteristic data of the identified tumor lesion area is input into a classification model which is completed by training, and the type of the tumor lesion is determined.

The above criterion of 0.85 means that when the maximum element in the likelihood matrix is equal to or greater than 0.85, the matrix is considered to be valid, and further the next determination is kept. In this example, the 0.85 standard referenced the AUC area standard, i.e., results were generally considered more reliable than 0.85. The probability matrix output by a good model is usually about 0.99, and in this embodiment, a mechanism of judging by using two detection channels is used, so that the probability matrix analysis is ensured to be completed smoothly by taking 0.85 as a standard.

The classification model adopts a residual neural network Resnet152, and is connected with a Softmax three-classifier after the Resnet152 for identifying benign tumors, malignant tumors and other tumors, and the training process of the classification model is as follows: taking the one-dimensional frequency characteristic data of a plurality of sampling points of hyperspectral images of benign tumor and malignant tumor lesion areas as a training sample data set, and training to obtain a classification model for identifying benign tumor, malignant tumor or other malignant tumor.

Through the intraoperative tumor rapid classification system based on the hyperspectral technology, benign and malignant tumors are judged based on hyperspectral images of tumor lesion tissues, so that the rapid classification of the tumor lesion types in the operation is realized, the negative influence of artificial identification of the tumor lesion types is avoided, the system is applied to intraoperative diagnosis, the rapid diagnosis time is shortened, the diagnosis efficiency is improved, and the accuracy of the diagnosis result is improved.

The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

While the foregoing description of the embodiments of the present invention has been presented in conjunction with the drawings, it should be understood that it is not intended to limit the scope of the invention, but rather, it is intended to cover all modifications or variations within the scope of the invention as defined by the claims of the present invention.

While the foregoing description of the embodiments of the present invention has been presented in conjunction with the drawings, it should be understood that it is not intended to limit the scope of the invention, but rather, it is intended to cover all modifications or variations within the scope of the invention as defined by the claims of the present invention.

Claims (10)

1. An intraoperative tumor rapid classification system based on hyperspectral technology is characterized by comprising:

the image preprocessing module is used for preprocessing the acquired hyperspectral images of a plurality of frequency bands of the tumor lesion tissues;

the tumor lesion area identification module is used for selecting a plurality of hyperspectral images with optimal frequencies corresponding to the hyperspectral images respectively, fusing the hyperspectral images to obtain fused spectrum images, detecting tumor targets based on the fused spectrum images, and identifying tumor lesion areas;

the tumor lesion type recognition module is used for sampling the recognized tumor lesion area by multiple pixel points, acquiring the frequency characteristic of each preferred frequency of each sampling point, respectively inputting the frequency characteristic of each sampling point into the benign tumor recognition model and the malignant tumor recognition model, and respectively outputting a benign probability matrix and a malignant probability matrix; based on the benign probability matrix and the malignant probability matrix, the type of neoplastic lesion is identified and determined.

2. The hyperspectral technology based intraoperative tumor rapid classification system of claim 1, wherein preprocessing the acquired hyperspectral image comprises:

removing the images of the invalid frequency bands in the hyperspectral images, and halving the gray values of the pixels in the hyperspectral images of each frequency band, wherein the gray values of the pixels are larger than the gray values of the pixels of the preset values, so as to obtain an initial preprocessing hyperspectral image;

removing the images of the invalid frequency bands in the hyperspectral reflection images, and performing difference on the initial preprocessed hyperspectral images and the pixel points corresponding to the hyperspectral reflection images one by one to obtain the hyperspectral images after preprocessing.

3. The hyperspectral technology based intraoperative tumor rapid classification system of claim 1, wherein the determination of the preferred frequency comprises:

acquiring hyperspectral images of a plurality of frequency bands of tumor lesion tissues, and cutting the hyperspectral images to obtain hyperspectral images of a lesion region and a surrounding normal region;

sampling hyperspectral images of a lesion area by multiple pixel points, and extracting one-dimensional spectrum characteristic data of each sampling point;

carrying out multi-pixel sampling on the hyperspectral image in the normal region, and extracting one-dimensional spectrum characteristic data of each sampling point;

and carrying out random pairwise combination on the one-dimensional spectrum characteristic data of the plurality of sampling points in the lesion area and the one-dimensional spectrum characteristic data of the plurality of sampling points in the normal area to obtain differences, carrying out absolute value summation on a plurality of groups of differences under each frequency to obtain the sum of absolute values of a plurality of groups of differences under each frequency, and selecting the frequency corresponding to the sum of the maximum absolute values as the preferred frequency.

4. The rapid intraoperative tumor classification system based on hyperspectral technology as claimed in claim 1, wherein the fusion is carried out to obtain a fused spectrum image, specifically: and respectively fusing gray images corresponding to the selected hyperspectral images with the preferred frequencies as R, G, B channels of the fused image, so as to realize the fusion of the images and obtain the fused spectral image.

5. The hyperspectral technology-based intraoperative tumor rapid classification system as claimed in claim 1, wherein the fusion spectral image-based tumor target detection is used for identifying tumor lesion areas, specifically:

based on the fusion spectrum image, performing tumor target detection by using a target detection model which is completed through training, identifying a tumor lesion area, and marking the tumor lesion area through a target detection anchor frame;

the target detection model adopts a yolov3 target detection algorithm, and a plurality of fusion spectral images of the artificially marked tumor lesion areas are used as a training sample set for training, so that a trained target detection model is obtained.

6. The rapid intra-operative tumor classification system based on hyperspectral technology as claimed in claim 1, wherein the multi-pixel sampling is performed on the identified tumor lesion area to obtain the frequency characteristic of each preferred frequency of each sampling point, specifically:

and determining a tumor lesion area marked by the target detection anchor frame, taking the anchor frame center as a datum point, taking the basic point as a 0 # point, taking the points from the 0 # point to the midpoint of the anchor frame boundary as 1-4 # points, taking the points from the 1-4 # points to the midpoint of the 0 # points as 5-8 # points, and taking the 8 # points as 1-8 # points for multi-pixel point sampling to obtain the frequency characteristic of each preferred frequency of each sampling point.

7. The hyperspectral technology-based intraoperative tumor rapid classification system of claim 1, wherein the benign tumor recognition model adopts a convolutional neural network to connect a fully connected layer and a classifier for distinguishing benign tumors and others;

the training process of the benign tumor recognition model comprises the following steps: and training by taking the one-dimensional frequency characteristic data of a plurality of sampling points of the hyperspectral image of the benign tumor lesion area as a training sample data set to obtain a benign tumor or other benign tumor identification model.

8. The hyperspectral technology-based intraoperative tumor rapid classification system as claimed in claim 1, wherein the malignant tumor recognition model adopts a convolutional neural network and a classifier for identifying malignant tumors and others;

the training process of the malignant tumor recognition model comprises the following steps: and training by taking the one-dimensional frequency characteristic data of a plurality of sampling points of the hyperspectral image of the malignant tumor lesion area as a training sample data set to obtain a malignant tumor or other malignant tumor recognition model.

9. The hyperspectral technology based intraoperative tumor rapid classification system as claimed in claim 1, wherein the type of tumor lesion is identified and determined based on benign probability matrix and malignant probability matrix, in particular:

based on the benign probability matrix and the malignant probability matrix, a probability matrix analysis is performed:

if the benign probability matrix is benign and the malignant probability matrix is other, determining that the type of the final tumor lesion is a benign tumor;

if the benign probability matrix is other and the malignant probability matrix is malignant, determining the type of the final tumor lesion as malignant;

if the benign probability matrix is other and the malignant probability matrix is also other, determining that the identified tumor lesion area is normal tissue or other non-tumor lesions;

if the benign probability matrix is benign and the malignant probability matrix is malignant, then subsequent processing is still required.

10. The hyperspectral technology based intraoperative tumor rapid classification system of claim 9, wherein the subsequent processing is: and inputting the one-dimensional frequency characteristic data of the identified tumor lesion area into the trained classification model, and determining the type of the tumor lesion.

Images