CN115177242A - Tissue blood oxygen evaluation method based on RGB image spectrum reconstruction - Google Patents
Tissue blood oxygen evaluation method based on RGB image spectrum reconstruction Download PDFInfo
- Publication number
- CN115177242A CN115177242A CN202210786161.2A CN202210786161A CN115177242A CN 115177242 A CN115177242 A CN 115177242A CN 202210786161 A CN202210786161 A CN 202210786161A CN 115177242 A CN115177242 A CN 115177242A
- Authority
- CN
- China
- Prior art keywords
- blood oxygen
- image
- hyperspectral
- rgb
- tissue blood
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 239000008280 blood Substances 0.000 title claims abstract description 54
- 210000004369 blood Anatomy 0.000 title claims abstract description 54
- QVGXLLKOCUKJST-UHFFFAOYSA-N atomic oxygen Chemical compound [O] QVGXLLKOCUKJST-UHFFFAOYSA-N 0.000 title claims abstract description 49
- 229910052760 oxygen Inorganic materials 0.000 title claims abstract description 49
- 239000001301 oxygen Substances 0.000 title claims abstract description 49
- 238000001228 spectrum Methods 0.000 title claims abstract description 27
- 238000011156 evaluation Methods 0.000 title abstract description 8
- 238000000034 method Methods 0.000 claims abstract description 25
- 230000003595 spectral effect Effects 0.000 claims abstract description 16
- 235000013405 beer Nutrition 0.000 claims abstract description 5
- 238000012549 training Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 7
- 238000003384 imaging method Methods 0.000 claims description 6
- 238000003062 neural network model Methods 0.000 claims description 6
- 238000005286 illumination Methods 0.000 claims description 4
- 238000010521 absorption reaction Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 230000009471 action Effects 0.000 claims description 2
- 230000017531 blood circulation Effects 0.000 claims description 2
- 230000009467 reduction Effects 0.000 claims description 2
- 238000013528 artificial neural network Methods 0.000 claims 1
- 238000012544 monitoring process Methods 0.000 abstract description 10
- 238000013135 deep learning Methods 0.000 abstract description 7
- 201000010099 disease Diseases 0.000 abstract description 4
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 abstract description 4
- 229910052736 halogen Inorganic materials 0.000 abstract description 2
- 150000002367 halogens Chemical class 0.000 abstract description 2
- XUMBMVFBXHLACL-UHFFFAOYSA-N Melanin Chemical compound O=C1C(=O)C(C2=CNC3=C(C(C(=O)C4=C32)=O)C)=C2C4=CNC2=C1C XUMBMVFBXHLACL-UHFFFAOYSA-N 0.000 description 16
- 108010054147 Hemoglobins Proteins 0.000 description 10
- 102000001554 Hemoglobins Human genes 0.000 description 10
- 238000000701 chemical imaging Methods 0.000 description 8
- 238000002496 oximetry Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000005259 measurement Methods 0.000 description 5
- 238000002835 absorbance Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000002310 reflectometry Methods 0.000 description 4
- 238000001514 detection method Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 208000008960 Diabetic foot Diseases 0.000 description 2
- 206010021137 Hypovolaemia Diseases 0.000 description 2
- 206010021138 Hypovolaemic shock Diseases 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000008033 biological extinction Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000010410 layer Substances 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000006213 oxygenation reaction Methods 0.000 description 2
- 206010040560 shock Diseases 0.000 description 2
- 208000024891 symptom Diseases 0.000 description 2
- INGWEZCOABYORO-UHFFFAOYSA-N 2-(furan-2-yl)-7-methyl-1h-1,8-naphthyridin-4-one Chemical compound N=1C2=NC(C)=CC=C2C(O)=CC=1C1=CC=CO1 INGWEZCOABYORO-UHFFFAOYSA-N 0.000 description 1
- 235000002566 Capsicum Nutrition 0.000 description 1
- 108010064719 Oxyhemoglobins Proteins 0.000 description 1
- 239000006002 Pepper Substances 0.000 description 1
- 235000016761 Piper aduncum Nutrition 0.000 description 1
- 235000017804 Piper guineense Nutrition 0.000 description 1
- 244000203593 Piper nigrum Species 0.000 description 1
- 235000008184 Piper nigrum Nutrition 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000008081 blood perfusion Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 108010002255 deoxyhemoglobin Proteins 0.000 description 1
- 238000012631 diagnostic technique Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000013210 evaluation model Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000011229 interlayer Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000004089 microcirculation Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000001575 pathological effect Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 150000003839 salts Chemical class 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000036559 skin health Effects 0.000 description 1
- 238000004611 spectroscopical analysis Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/145—Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue
- A61B5/14542—Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue for measuring blood gases
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0077—Devices for viewing the surface of the body, e.g. camera, magnifying lens
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/58—Extraction of image or video features relating to hyperspectral data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20032—Median filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Physics & Mathematics (AREA)
- Medical Informatics (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Pathology (AREA)
- Heart & Thoracic Surgery (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Veterinary Medicine (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biophysics (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Databases & Information Systems (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Public Health (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Optics & Photonics (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The invention relates to a tissue blood oxygen assessment method based on RGB image spectral reconstruction, and belongs to the technical field of biomedicine. The invention provides a tissue blood oxygen assessment method based on RGB image spectrum reconstruction. The invention uses halogen lamp light source to irradiate skin tissues of different parts of human body, and simultaneously uses common industrial camera and hyperspectral camera to collect RGB and hyperspectral images of the same skin area as data set. A spectrum reconstruction model from an RGB image to a tissue blood oxygen characteristic hyperspectral image is constructed based on a deep learning method, further, the newly shot skin tissue RGB image is reconstructed into a high-physical-reliability skin tissue hyperspectral image, and a tissue blood oxygen content two-dimensional space distribution map is finally obtained through an improved Lambert beer model. The invention provides a simple and low-cost tissue blood oxygen non-contact monitoring method, and the result can be used as the monitoring and evaluation basis of various diseases.
Description
Technical Field
The invention relates to a tissue blood oxygen assessment method based on RGB image spectral reconstruction, in particular to a method for realizing two-dimensional spatial distribution of tissue blood oxygen content obtained from skin RGB images by using deep learning for spectral reconstruction, and belongs to the technical field of biomedicine.
Background
Tissue oximetry refers to the oxygen saturation of blood in the tissue microcirculation, reflecting the oxygen transport and consumption in the body, which ranges from 60% of venous oximetry to 98% of arterial oximetry. In clinical application, tissue blood oxygen is an important index of blood perfusion and blood oxygenation change, is used for monitoring the functional state of a human tissue region, and is a monitoring and evaluating index of central hypovolemic and hypovolemic shock symptoms, diabetic foot ulcer and other diseases. Conventional methods for measuring tissue oximetry require direct contact with the tissue being measured, which is impractical for most clinically wounded skin and does not allow a complete description of regional tissue oximetry information spatially. Hyper-spectral imaging (HSI), a new, non-contact optical diagnostic technique, may present the results of tissue oximetry measurements in the form of a spatial two-dimensional tissue oximetry map. It combines a hybrid mode of imaging and spectroscopy to produce a three-dimensional data set of spatial and spectral information by collecting spectral information at each pixel of a two-dimensional detector array. The abundant image information and spectral information provide an effective auxiliary diagnosis means for clinical medicine, and have great development potential.
The absorbance of chromophores such as oxygenated hemoglobin, deoxygenated hemoglobin, melanin and the like in biological tissues to different wavelengths of light is different, so that the diffuse reflection spectrum of the chromophores has obvious optical characteristic difference. The hyperspectral imaging technology acquires the hyperspectral image thereof, and establishes a proper hyperspectral imaging human skin tissue evaluation model by adopting a specific evaluation method, so that the content of chromophores in human tissues such as oxygenated hemoglobin and deoxygenated hemoglobin in a tissue area can be obtained, and a two-dimensional distribution map of the tissue blood oxygen content in the specific area is obtained.
However, the hyperspectral equipment is generally complex in structure, expensive in price and complex in operation, is not beneficial to clinical popularization of regional tissue imaging, and is also not beneficial to popularization of daily monitoring of physiological parameters of future family scenes. Compared with the RGB image obtained by the common visible light imaging equipment, the RGB image contains a small amount of information and is easier to obtain. With the development of the deep learning algorithm, the method has the advantages of strong computing power, model transportability, reliability improvement of the model driven by data and the like, and is widely applied to multi-scene hyperspectral image reconstruction. The deep learning technology is applied to the reconstruction of the skin tissue RGB image and the hyperspectral image and is used for evaluating the tissue blood oxygen content, so that the individual daily monitoring of the tissue blood oxygen can be greatly facilitated, and the predictive diagnosis of some diseases can be realized.
Disclosure of Invention
The invention discloses a tissue blood oxygen evaluation method based on RGB image spectrum reconstruction, aiming at the problems that hyperspectral equipment used in the imaging type non-contact monitoring of tissue blood oxygen concentration is expensive, complex in structure, complex to operate and the like. The method is based on a deep learning method to train a spectrum reconstruction model from an RGB image to a hyperspectral image, and a two-dimensional space distribution image of tissue blood oxygen content is obtained by using least square regression for the reconstructed hyperspectral image. The invention provides a simple and low-cost tissue blood oxygen monitoring method. Meanwhile, the method is expected to be combined with a smart phone to realize affordable skin health detection.
The purpose of the invention is realized by the following technical scheme.
A tissue blood oxygen assessment method based on RGB image spectrum reconstruction comprises the following steps:
step 1, respectively acquiring a hyperspectral image and an RGB image containing skin tissues and a calibration plate image in the same illumination area by using a hyperspectral camera and a common industrial camera;
step 2, preprocessing the image acquired in the step 1 to obtain a corresponding HSI-RGB data pair;
1) At the level of the spectral channels: intercepting a characteristic wave band (450 nm-600 nm) aiming at the collected hyperspectral data and carrying out spectrum dimensionality reduction on the wave band data to 31 spectrum channels, wherein the wave band step length is 5nm. (ii) a
2) At the spatial image level: removing background noise by using threshold value and carrying out median filtering processing
Step 3, repeating the steps 1 and 2, collecting and processing HSI and RGB images of hands of different people in different blood oxygen content states;
step 4, pairing the large amount of HSI and RGB data obtained in the step 3 and inputting the paired data serving as a data set into a neural network model for training to obtain a spectrum reconstruction model;
and 5, under the experimental condition of the step 1, acquiring RGB images of a new subject by using an industrial camera, inputting a trained reconstruction model, outputting a hyperspectral image, and then substituting the trained reconstruction model into an improved Lambert beer model to obtain a final tissue blood oxygen content spatial distribution map by using least square regression.
Advantageous effects
1. The invention relates to a tissue blood oxygen evaluation method based on RGB image spectrum reconstruction, which is used for carrying out spectrum reconstruction based on a deep learning technology and extracting physiological parameter information from hyperspectral data by combining least square regression so as to realize non-contact imaging on tissue blood oxygen concentration.
2. The invention is suitable for obtaining the tissue blood oxygen concentration of different parts of a human body, and the result can be used as the daily monitoring and evaluation basis of a plurality of diseases such as central hypovolemic and hypovolemic shock symptoms, diabetic foot and the like.
3. The invention realizes the evaluation of tissue blood oxygen concentration in a non-invasive, simple and convenient way by utilizing a non-contact optical detection mode, and realizes the affordable daily monitoring of the tissue blood oxygen concentration. The hyperspectral equipment with complex structure, high price and complex operation is avoided.
Drawings
Fig. 1 is a schematic diagram of an HSI-RGB data set acquisition of a tissue blood oxygen assessment method based on RGB image spectrum reconstruction according to an embodiment;
fig. 2 is a schematic diagram of RGB image acquisition during actual measurement of the tissue blood oxygen assessment method based on RGB image spectrum reconstruction provided by the embodiment;
FIG. 3 is a schematic diagram of an algorithm flow of a tissue blood oxygen assessment method based on RGB image spectral reconstruction according to an embodiment
Detailed Description
To make the objects, advantages and features of the present invention more apparent, a method and an apparatus for tissue blood oxygenation based on RGB image spectral reconstruction according to the present invention are described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that: the accompanying drawings, which are in a very simplified form and are not to scale, are included for purposes of illustrating embodiments of the invention in a clear and concise manner and are incorporated in and constitute a part of the actual drawings; the RGB image acquisition part is not limited to the hand of the human body, and other parts of the human body are also applicable. The deep learning spectral reconstruction model used in the present invention is not limited to a certain hierarchical regression model.
The present example uses Asian yellow as the subject.
A schematic diagram of data set acquisition is shown in FIG. 1, a schematic diagram of RGB image acquisition during actual measurement is shown in FIG. 2, and a schematic diagram of general algorithm flow is shown in FIG. 3.
Step 1, respectively acquiring a hyperspectral image and an RGB image containing skin tissues and a calibration plate image in the same illumination area by using a hyperspectral camera and a common industrial camera;
step 1-1, starting a light source and a camera:
the examinee sits on a chair statically, the skin tissue of the hand of the human body is irradiated by the halogen lamp light source, the hyperspectral camera and the RGB camera are started simultaneously, and the two cameras are focused to the clear image. The embodiment is described by taking a hand image as a site for acquiring tissue blood oxygen information as an example.
Step 1-2, image acquisition:
the size of images acquired by utilizing the hyperspectral camera and the industrial camera to acquire corresponding illumination areas is 512 multiplied by 482, the corresponding detection wave band of the hyperspectral camera is 387nm to 999nm, the step length is 1.7nm, and the hyperspectral camera corresponds to 360 spectrum channels. The subject remains relatively stationary during the photographing. For the hyperspectral camera, a white board with fixed reflectivity is additionally arranged at a hand position to shoot a white board image, and a camera covers the shot white board image to synthesize a skin diffuse reflection spectrum image.
Step 2, processing the image acquired in the step 1 to obtain corresponding HSI-RGB data pairs;
step 2-1, at the level of the spectral channel: : carrying out feature sampling extraction on original hyperspectral data, and reducing dimensions to obtain 123 spectrum channels with 5nm as step length in an original wavelength range;
step 2-2, at the spatial image level: for dark background areas in the shooting process, the reflectivity of the dark background areas in the spectrum is low, a proper reflectivity threshold value is selected and removed through a threshold processing method, and therefore the reconstruction effect of the whole skin area is not affected. And then, aiming at the problem of more salt and pepper noises, filtering the hyperspectral image by adopting a median filter of a 3 x 3 template.
And 3, repeating the steps 1 and 2, collecting and processing hand HSI and RGB images of different people in different blood oxygen content states, wherein the specific operation method aiming at the different blood oxygen content states comprises the following steps: for the index finger, the rubber band is fastened at the third joint of the finger to prevent blood flow to form blood stasis and high oxygen absorption of tissues, and for the middle finger, the rubber band is fastened at the bottom of the middle finger through the action of downwards squeezing blood to empty blood in the middle finger. Acquiring hand skin images under normal state and 30 seconds, 90 seconds and 120 seconds in a fastening state for each subject, and using other fingers as healthy reference fingers without operation; the original data volume of the skin tissues under different blood oxygen content states is more than 100 groups. In order to reduce the overfitting phenomenon of the network and obtain the network with stronger generalization capability, a data enhancement technology (rotation, overturning and the like) is used for expanding a data set, and the data set is expanded to be four times of the original data set.
Step 4, pairing a large amount of HSI and RGB data obtained in the step 3, inputting the paired data serving as a data set into a neural network model for training, and obtaining a spectrum reconstruction model;
step 4-1, taking the HSI-RGB data pairs obtained by sequentially matching the hyperspectral images and the RGB images acquired in the step 3 as data sets, and dividing the data sets into a training set and a test set;
and 4-2, inputting the training set obtained in the step 4-1 into a neural network model for training. And obtaining a spectrum reconstruction model after training. The neural Network model selects a Hierarchical Regression Network model, which is a 4-layer Hierarchical Regression Network (HRNet), wherein a pixel mixed layer is used as interlayer interaction. And removing artifacts of the real-world RGB image by using the residual set blocks, and establishing an attention mechanism by using the residual global blocks to enlarge a perception field. During training, the blocksize is set to be 16, the initial learning rate is 0.0001, the epoch =3000, the learning rate attenuation coefficient is 0.5, and the attenuation is performed once every 1000 rounds. This model is implemented in Pytorh.
And 5, under the experimental condition of the step 1, acquiring RGB images of a new subject by using an industrial camera, inputting a trained reconstruction model, outputting a hyperspectral image, and then substituting the trained reconstruction model into an improved Lambert beer model to obtain a final tissue blood oxygen content spatial distribution map by using least square regression.
And 5-1, under the experimental condition of the step 1, acquiring RGB images of a new subject by using the industrial camera again, inputting the spectrum reconstruction model trained in the step 4-2, and outputting a reconstructed hyperspectral image.
And 5-1, converting the spectral reflectivity of each pixel point in the input hyperspectral image into the absorbance of the point under each wave band.
Wherein R is (x，y，，λ) Is the reflection spectrum of the measured tissue pixel point under the experimental lighting system, and A (x, y, lambda) isAnd the absorbance of each wave band in the pixel point.
And 5-2, substituting the absorbance of the skin pixel point under each wave band into the improved Lambert beer model.
A＝ε Hb C Hb +ε HbO2 C HbO2 +ε melanin C melanin +G (2)
Wherein epsilon Hb ，ε HbO2 ，ε melanin The molar absorption rate or extinction coefficient (cm) of pure oxyhemoglobin, deoxyhemoglobin and melanin corresponding to a certain wave band -1 mol -1 L -1 ) The partial values being obtained from the standard values previously determined by the investigator, C Hb 、C HbO2 、C melanin Effective concentration of three chromophores (10) -3 mol cm -2 ) G is the fraction of photons lost due to scattering, and is a constant, independent of wavelength.
Then, equation (2) is solved by using the least square method, so that the concentration of each chromophore (oxygenated hemoglobin, deoxygenated hemoglobin, melanin) under the skin pixel point is obtained.
C mod ＝(E′E) -1 E′A (3)
In the formula, C mod Is a simulated effective concentration matrix comprising an effective concentration of a chromophore (C) Hb 、C HbO2 、C melanin ) And scattering constant (G), E is a molar extinction coefficient matrix.
Step 5-3, substituting the concentration of the oxygenated hemoglobin and the concentration of the deoxygenated hemoglobin obtained in the step 5-2 into a formula to calculate the tissue blood oxygen value of the pixel point
Through the steps, the tissue blood oxygen concentrations corresponding to all the skin tissue pixel points in the space are solved, and the final tissue blood oxygen content space distribution map is obtained.
In a specific embodiment, the optimal measurement area in the tissue blood oxygen assessment method based on RGB image spectral reconstruction is the extremity where the capillary vessels of skin tissue are most abundant, and the change of tissue blood oxygen concentration is most easily perceived from the image. The optimal interesting wave band for the hyperspectral image is 450nm-600nm, the part of spectral information covers the characteristic wave bands of oxygenated hemoglobin and deoxygenated hemoglobin, and more accurate measurement results can be obtained by adopting the wave bands to perform tissue blood oxygen assessment. The finally measured tissue blood oxygen content spatial distribution map has important significance for the analysis of further pathological information of patients.
Although the present invention has been described in detail with reference to the above embodiments, it should be understood by those skilled in the art that: modifications and equivalents may be made thereto without departing from the spirit and scope of the invention and it is intended to cover in the claims the invention as defined in the appended claims.
Claims (3)
1. A tissue blood oxygen assessment method based on RGB image spectrum reconstruction is characterized in that: the method comprises the following steps:
step 1, respectively acquiring a hyperspectral image and an RGB image containing skin tissues and a calibration plate image in the same illumination area by using a hyperspectral camera and a common industrial camera;
step 2, carrying out data processing on the image acquired in the step 1 to obtain a corresponding HSI-RGB data pair;
1) At the level of the spectral channels: intercepting a characteristic wave band (450 nm-600 nm) aiming at the collected hyperspectral data and performing spectrum dimensionality reduction on the wave band data to 31 spectrum channels, wherein the wave band step length is 5nm;
2) At the spatial image level: and removing background noise by using a threshold method and carrying out median filtering processing.
And 3, repeating the steps 1 and 2, collecting and processing hand HSI and RGB images of different people in different blood oxygen content states, wherein the specific operation method aiming at the different blood oxygen content states comprises the following steps: for the index finger, the rubber band is fastened at the third joint of the finger to prevent blood flow to form blood stasis and high oxygen absorption of tissues, for the middle finger, the rubber band is fastened at the bottom of the middle finger through the action of downwards squeezing blood to empty the blood of the middle finger, and other fingers do not perform the operation;
step 4, inputting a large number of HSI and RGB data pairs obtained in the step 3 into a neural network model as a data set for training to obtain a spectrum reconstruction model;
and 5, under the experimental condition of the step 1, acquiring RGB images of a new subject by using an industrial camera, inputting a trained reconstruction model, outputting a hyperspectral image, and then substituting the trained reconstruction model into an improved Lambert beer model to obtain a final tissue blood oxygen content spatial distribution map by using least square regression.
2. The invention of claim 1, wherein the invention is adapted for imaging tissue blood oxygen content at different locations in the human body.
3. The method according to claim 1, wherein the reconstruction of the hyperspectral image is applicable to all spectral reconstruction neural network models, and is not limited to a certain neural network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210786161.2A CN115177242B (en) | 2022-07-04 | 2022-07-04 | Tissue blood oxygen assessment method based on RGB image spectral reconstruction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210786161.2A CN115177242B (en) | 2022-07-04 | 2022-07-04 | Tissue blood oxygen assessment method based on RGB image spectral reconstruction |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115177242A true CN115177242A (en) | 2022-10-14 |
| CN115177242B CN115177242B (en) | 2024-06-18 |
Family
ID=83516797
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210786161.2A Active CN115177242B (en) | 2022-07-04 | 2022-07-04 | Tissue blood oxygen assessment method based on RGB image spectral reconstruction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115177242B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU2005327078A1 (en) * | 2004-12-28 | 2006-08-17 | Hypermed Imaging, Inc. | Hyperspectral/multispectral imaging in determination, assessment and monitoring of systemic physiology and shock |
| US8644911B1 (en) * | 2006-06-30 | 2014-02-04 | Hypermed Imaging, Inc. | OxyVu-1 hyperspectral tissue oxygenation (HTO) measurement system |
| CN107348944A (en) * | 2017-07-07 | 2017-11-17 | 深圳市唯特视科技有限公司 | A kind of imaging method based on ultraspectral resolution ratio |
| CN111528792A (en) * | 2020-05-12 | 2020-08-14 | 宁波蓝明信息科技有限公司 | Rapid hyperspectral fundus imaging method based on spectral information compression and recovery |
-
2022
- 2022-07-04 CN CN202210786161.2A patent/CN115177242B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU2005327078A1 (en) * | 2004-12-28 | 2006-08-17 | Hypermed Imaging, Inc. | Hyperspectral/multispectral imaging in determination, assessment and monitoring of systemic physiology and shock |
| US8644911B1 (en) * | 2006-06-30 | 2014-02-04 | Hypermed Imaging, Inc. | OxyVu-1 hyperspectral tissue oxygenation (HTO) measurement system |
| CN107348944A (en) * | 2017-07-07 | 2017-11-17 | 深圳市唯特视科技有限公司 | A kind of imaging method based on ultraspectral resolution ratio |
| CN111528792A (en) * | 2020-05-12 | 2020-08-14 | 宁波蓝明信息科技有限公司 | Rapid hyperspectral fundus imaging method based on spectral information compression and recovery |
Non-Patent Citations (1)
| Title |
|---|
| 戴丽娟;王惠南;钱志余;于国强;: "血液可见吸收光谱与血氧参数神经网络估算法", 光谱学与光谱分析, no. 07, 15 July 2008 (2008-07-15) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115177242B (en) | 2024-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Nishidate et al. | Noninvasive imaging of human skin hemodynamics using a digital red-green-blue camera | |
| US20100210931A1 (en) | Method for performing qualitative and quantitative analysis of wounds using spatially structured illumination | |
| CN105473060B (en) | System and method for extracting physiologic information from the electromagnetic radiation remotely detected | |
| US10013811B2 (en) | Hyperspectral image visualization in patients with medical conditions | |
| CN108471989A (en) | The equipment, system and method for the photo-plethysmographic image of vital sign information for generating carrying object | |
| KR20180078272A (en) | Reflection mode for tissue classification Multispectral time resolved optical imaging methods and devices | |
| WO2006100685A2 (en) | Optical sensor device and image processing unit for measuring chemical concentrations, chemical saturations and biophysical parameters | |
| KR20140034118A (en) | Deternimation of tissue oxygenation in vivo | |
| CN109497977A (en) | Human heart rate and method for detecting blood oxygen saturation and device | |
| Hu et al. | Opto‐Physiological Modeling Applied to Photoplethysmographic Cardiovascular Assessment | |
| He et al. | Analysis of skin morphological features and real-time monitoring using snapshot hyperspectral imaging | |
| Dremin et al. | Hyperspectral imaging of diabetes mellitus skin complications | |
| US20200294228A1 (en) | Non-Contact Multispectral Imaging for Blood Oxygen Level and Perfusion Measurement and Related Systems and Computer Program Products | |
| Li et al. | Edge detection of heterogeneity in transmission images based on frame accumulation and multiband information fusion | |
| Hasan et al. | HeLP ME: Recommendations for non-invasive hemoglobin level prediction in mobile-phone environment | |
| CN115474930A (en) | Hyperspectral image reconstruction-based noninvasive hemoglobin detection method | |
| CN115177242B (en) | Tissue blood oxygen assessment method based on RGB image spectral reconstruction | |
| Hasan | BEst (Biomarker Estimation): health biomarker estimation non-invasively and ubiquitously | |
| Nishidate et al. | Visualization of peripheral vasodilative indices in human skin by use of red, green, blue images | |
| TWI651537B (en) | Non-invasive method for detecting glycated hemoglobin | |
| RU2496417C1 (en) | Method of differential diagnostics of melanocytic neoplasms of skin | |
| CN111991004A (en) | Blood oxygen saturation measuring device, measuring method and measuring apparatus | |
| Blazek et al. | Selected Clinical Applications of Functional PPGI Perfusion Mapping in Dermatology | |
| Suciu | Non-invasive Detection of Peripheral Arterial Disease in the Lower Limb | |
| Harrison | NOVEL NONINVASIVE OPTICAL DIAGNOSTIC TECHNOLOGIES FOR THE MANAGEMENT OF NEONATAL JAUNDICE |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |