CN112330649B - Physiological information acquisition method and device combining multispectral and visible light image - Google Patents
Physiological information acquisition method and device combining multispectral and visible light image Download PDFInfo
- Publication number
- CN112330649B CN112330649B CN202011263000.2A CN202011263000A CN112330649B CN 112330649 B CN112330649 B CN 112330649B CN 202011263000 A CN202011263000 A CN 202011263000A CN 112330649 B CN112330649 B CN 112330649B
- Authority
- CN
- China
- Prior art keywords
- image
- physiological information
- candidate
- view
- full
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
- G06T7/0014—Biomedical image inspection using an image reference approach
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
- G06T7/337—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10056—Microscopic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Image Processing (AREA)
Abstract
The invention provides a method and a device for acquiring physiological information by combining multispectral images and visible light images, wherein the method comprises the following steps: acquiring a full-field image of physiological content, and extracting a plurality of candidate image areas in the full-field image; acquiring a visible light image and multispectral images of a plurality of fields of view of each candidate image region; registering the multispectral image of each view of each candidate image region with the corresponding visible light image to obtain a reference image of each view of each candidate image region; determining reference physiological information corresponding to the reference image for each field of view; determining target physiological information of each candidate image area according to all reference physiological information corresponding to the reference images of all the visual fields; and determining physiological information corresponding to the full-view image according to the target physiological information of all candidate image areas in the full-view image. Therefore, the acquisition accuracy of the physiological information is improved, and the acquisition mode of the physiological information is expanded.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a method and a device for acquiring physiological information by combining multispectral images and visible light images.
Background
With the development of computer technology, new changes are brought to various fields, for example, the method can be applied to the medical field based on image processing, and convenience is brought to medical application.
However, in the prior art, image processing technologies applied in other fields lack specific application modes.
Disclosure of Invention
The present invention is directed to solving, at least to some extent, one of the technical problems in the related art.
Therefore, a first objective of the present invention is to provide a physiological information acquisition method combining multiple spectra and visible light images, so as to improve the accuracy of acquiring physiological information and expand the acquisition modes of physiological information.
A second object of the present invention is to provide a physiological information acquisition device combining multispectral and visible light images.
A third object of the invention is to propose a computer device.
In order to achieve the above object, a first aspect of the present invention provides a method for acquiring physiological information by combining multispectral and visible light images, including: acquiring a full-field image of physiological content, and extracting a plurality of candidate image regions in the full-field image; acquiring a visible light image and multispectral images of a plurality of fields of view of each candidate image region; registering the multispectral image of each view of each candidate image region with the corresponding visible light image to obtain a reference image of each view of each candidate image region; extracting image features of the reference image of each visual field, and determining reference physiological information corresponding to the reference image of each visual field according to the image features; determining target physiological information of each candidate image region according to all the reference physiological information corresponding to the reference images of all the fields of view of each candidate image region; and determining physiological information corresponding to the full-field image according to the target physiological information of all the candidate image areas in the full-field image.
In order to achieve the above object, a second embodiment of the present invention provides a physiological information acquisition device combining multiple spectra and visible light images, comprising: the extraction module is used for acquiring a full-view image of physiological content and extracting a plurality of candidate image areas in the full-view image; the acquisition module is used for acquiring a visible light image and multispectral images of a plurality of visual fields of each candidate image area; and the registration module is used for registering the multispectral image of each view field of each candidate image region with the corresponding visible light image to obtain a reference image of each view field of each candidate image region. The first determining module is used for extracting the image characteristics of the reference image of each visual field and determining the reference physiological information corresponding to the reference image of each visual field according to the image characteristics; a second determining module, configured to determine target physiological information of each candidate image region according to all the reference physiological information corresponding to the reference images of all the fields of view of each candidate image region; and the third determining module is used for determining the physiological information corresponding to the full-field image according to the target physiological information of all the candidate image areas in the full-field image.
To achieve the above object, a third embodiment of the present invention provides a computer device, including: comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method as described in the above embodiments when executing the computer program.
The embodiment of the invention only has the following beneficial technical effects:
acquiring a full-view image of physiological content, extracting a plurality of candidate image areas in the full-view image, acquiring a visible light image of each candidate image area and a multispectral image of a plurality of views, registering the multispectral image of each view of each candidate image area with the corresponding visible light image to obtain a reference image of each view of each candidate image area, extracting the image characteristics of the reference image of each view, determining reference physiological information corresponding to the reference image of each view according to the image characteristics, determining target physiological information of each candidate image area according to all the reference physiological information corresponding to the reference images of all the views of each candidate image area, and further determining the physiological information corresponding to the full-view image according to the target physiological information of all the candidate image areas in the full-view image. Therefore, the acquisition accuracy of the physiological information is improved, the acquisition mode of the physiological information is expanded, the visual result can be displayed on the full-view image, and the acquisition efficiency of the physiological information of the related scene is improved.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The foregoing and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 is a schematic flowchart of a physiological information acquiring method combining multiple spectra and a visible light image according to an embodiment of the present invention;
FIG. 2 is a schematic view of a scene of a candidate image region according to one embodiment of the invention;
FIG. 3 is a scene schematic of image registration according to one embodiment of the invention;
FIG. 4 is a flow diagram of physiological information determination for candidate image regions according to one embodiment of the present invention;
FIG. 5 is a flow diagram of physiological information determination for a full field of view image according to one embodiment of the present invention;
FIG. 6 is a schematic view of a physiological information display scenario according to one embodiment of the present invention; fig. 7 is a schematic structural diagram of a physiological information acquisition device combining multispectral and visible light images according to an embodiment of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
A physiological information acquisition method and apparatus combining multispectral and visible light images according to an embodiment of the present invention will be described below with reference to the accompanying drawings. For convenience of description, the image processing of the embodiment of the present invention is mainly applied to the acquisition of physiological information, which may be understood as microbial information in body fluid separated from a human body, to assist the application of a corresponding scene, where the body fluid and the like in the embodiment of the present invention are described as physiological content in the following embodiments.
Fig. 1 is a schematic flowchart of a physiological information acquisition method combining multiple spectra and a visible light image according to an embodiment of the present invention. As shown in fig. 1, the method includes:
In one embodiment of the present invention, biological content may be placed on a slide, and a full-field image may be taken of an area of the full-field slide, where the acquisition of the full-field image may be understood as the acquisition of a visible image. Firstly, a biological sample in biological content can be dyed and filmed, then the biological sample is placed under a 10X objective lens of a microscope, visible light images are collected on the whole film, and a large image of panoramic visible light is obtained as a full-view image through image synthesis.
In some possible embodiments, an image object recognition model, such as fast RCNN, is obtained by training a large amount of sample data labeled by doctors, so that the full-field image is input to the image object recognition model, and a plurality of candidate image areas are obtained according to an output result of the image object recognition model, for example, the output result is position information of each of the candidate image areas. In other possible embodiments, the full-view image is switched and divided into a plurality of image regions according to a preset size, image features of the full-view image are identified, whether each image region contains microorganism information and the area of the contained microorganism information are judged according to the image features, and when the area of the image region occupied by the area is larger than a certain value, the corresponding region is determined as a candidate image region.
For example, the multispectral image is obtained by acquiring a spectral image every 20NM on a wave band with a central wavelength of 400NM-800NM, so as to acquire a multispectral image of 24 wave bands.
It should be understood that, since the candidate image region is an image region in the full-view image, the determined candidate image region is a full-view visible light image, and in the present embodiment, after the visible light image of each region is acquired, multispectral images of multiple views of each image region are acquired. For example, data acquisition is performed on several candidate regions of interest under a high power mirror such as an oil lens, and a visible light image and a multispectral image of the candidate regions are obtained.
In this embodiment, the multispectral image of each view of each candidate image region is registered with the corresponding visible light image, and the registered multispectral image is overlapped with the visible light image to obtain a reference image, that is, the reference image is obtained by overlapping the registered multispectral image with the corresponding visible light image. Each multispectral image is a single-view image. As shown in fig. 2, the candidate image region 3 may contain a corresponding visible light image and a multi-view multi-spectral image.
In one embodiment of the present invention, a first position Feature of the multispectral image of each field of view in the candidate image region is obtained, the first position Feature being understood as Scale-Invariant Feature Transform (SIFT), a second position Feature of the visible light image of the candidate image region is obtained, the second Feature being understood as Scale-Invariant Feature Transform (SIFT), and further, as shown in fig. 3, the position of the visible light image is adjusted according to the first position Feature and the second position Feature, for example, by performing a shift and a rotation according to the matching of the position features of the multispectral image of each field of view of each candidate image region and the corresponding visible light image, so that the multispectral image of each field of view of each candidate image region is registered with the corresponding visible light image.
And 104, extracting the image characteristics of the reference image of each visual field, and determining the reference physiological information corresponding to the reference image of each visual field according to the image characteristics.
In an embodiment of the present invention, after a reference image is obtained, the reference image may be normalized, and the normalized reference image is subjected to image feature extraction, where the normalization processing on the reference image includes calculating a mean value and a variance of pixel values of the reference image, calculating a difference between pixel brightness of each pixel point in the reference image and the mean value, taking a ratio of the difference to the variance as a pixel value of each pixel point, processing the pixel value of the reference image to a value of 0-1, removing noise influence of the image brightness, and further improving the identification accuracy of the image.
In the present embodiment, image features of a reference image of each field of view are extracted, and reference physiological information corresponding to the reference image of each field of view is determined according to the image features, wherein a correspondence relationship in which the image features can refer to the physiological information may be constructed in advance, and the reference physiological information may be determined according to the correspondence relationship, wherein the reference physiological information may include one or more of a type of microorganism, a total number of microorganisms, a number of types of each microorganism, a location of the microorganism, an area size of the microorganism, a disease type, a label indicating a preset classification category of physiological information, and the like. The types of the microorganisms are related according to scenes, and when the current scene is a disease type for measuring vaginitis, the types of the microorganisms comprise normal flora, flora corresponding to bacterial vaginitis, flora corresponding to aerobic vaginitis, flora corresponding to candida vaginitis, flora corresponding to trichomonas vaginitis, flora corresponding to bacterial vaginitis intermediate type, flora corresponding to flora inhibition, flora corresponding to mixed infection and the like.
In other possible embodiments, models such as a decision tree and an SVM may be obtained according to sample data training of the image features, and the image features are input to the corresponding models to obtain reference physiological information output by the models.
In an embodiment of the present invention, a multi-example learning method may be utilized, and a simplest multi-example learning method, a statistical method, for example, may be used, which microorganism has the largest number of categories in all the fields of view of the reference image, and this microorganism category or the disease type corresponding to the microorganism category is a result of the reference physiological information of the reference image.
In this example, image features of the reference image of each field of view may be extracted, and reference physiological information corresponding to the reference image of each field of view may be determined from the image features.
For example, image features of the reference image of each field of view may be extracted, at least one microorganism type corresponding to the reference image of each field of view and a number corresponding to each microorganism type may be determined based on the image features, and the microorganism type and the corresponding number may be used as a detection result of the corresponding reference physiological information.
And 105, determining target physiological information of each candidate image area according to all reference physiological information corresponding to the reference images of all the fields of view of each candidate image area.
In the present embodiment, the target physiological information of each candidate image region is determined according to all the reference physiological information corresponding to the reference images of all the fields of view of each candidate image region.
In some possible embodiments, all the reference physiological characteristic information may be input to the pre-trained deep learning model to obtain the corresponding target physiological information.
In other possible embodiments, the reference physiological information corresponding to the reference images of all the fields of view of each candidate image region is determined, and the target physiological information of each candidate image region is determined according to all the reference physiological information, for example, the reference physiological information with the highest repetition rate in all the reference physiological information is used as the target physiological information of the corresponding candidate image region.
For example, all the visual field disease types of each candidate image region are determined, for example, all the microorganism types corresponding to the reference image and the total number corresponding to each microorganism type are obtained, and the disease type corresponding to the target microorganism type is used as the target physiological information of each candidate image region according to the target microorganism type with the highest total number.
For example, as shown in fig. 4, when the candidate image region includes N views, after normalization processing is performed on the reference image of each view, image features of the normalized reference image are extracted, reference physiological information of each reference image is determined according to the image features, for example, a disease type is determined according to a type and a number of microorganisms of each reference image, and further, according to the reference physiological feature information of all the reference images of the N views, multiple examples of learning is performed to obtain target physiological information of the corresponding candidate image region, for example, a disease type of the candidate image region is obtained according to a type and a number of microorganisms, wherein the normalization processing on the reference image includes calculating a mean value and a variance of pixel values of the reference image, calculating a difference value between pixel brightness and the mean value of each pixel point in the reference image, taking a ratio of the difference value and the variance as a pixel value of each pixel point, processing the pixel value of the reference image to a value of 0-1, removing noise influence of the image brightness, further improving the image recognition accuracy, and it should be noted that, each M of the candidate image region 1 in fig. 4 is a single spectral image, and may be a natural spectral image, such as a single segment of M-24.
And step 106, determining physiological information corresponding to the full-field image according to the target physiological information of all candidate image areas in the full-field image.
In the present embodiment, as shown in fig. 5, physiological information corresponding to the full-field image is determined from target physiological information of all candidate image regions in the full-field image. For example, a group of target physiological information with the highest repetition rate in all candidate image regions is used as the physiological information corresponding to the full-field image.
In an embodiment of the present invention, when the physiological information is a microorganism type, all target microorganism types of all candidate image regions and the total number of targets corresponding to each target microorganism type may be obtained, the target microorganism type corresponding to the highest total number of targets may be determined, and a disease type corresponding to the target microorganism type may be used as the physiological information corresponding to the full-field image, or of course, a disease type of the top 3 ranks may be determined according to the number and type of microorganisms of each candidate image region, and further, the first disease type with the highest repetition rate among the disease types in all candidate image regions may be used as the physiological information of the full-field image.
Further, in an embodiment of the present invention, related information may also be visually displayed to meet the needs of the scene, for example, to assist medical treatment and the like.
In an embodiment of the present invention, color information corresponding to each candidate image region and reference physiological information corresponding to a reference image of each field of view of each candidate image region are determined, and display colors of the candidate image regions are adjusted in the full-field image according to the color information corresponding to the candidate image regions, that is, as shown in fig. 6, all candidate image regions are mapped into the full-field visible light image-full-field image obtained in step 101 through coordinates, and the candidate regions are marked with different colors or the same color (different gray values in the figure represent different colors).
In another embodiment of the present invention, the corresponding physiological information may also be visually displayed, all reference physiological information of all reference images corresponding to a target candidate region in the plurality of candidate image regions is obtained in response to a viewing instruction for the target candidate region, all reference images corresponding to the target candidate region are displayed, and the corresponding reference physiological information is displayed in each reference image.
For example, with continued reference to fig. 6, when a viewing instruction for the candidate image region 3 is detected, for example, when it is detected that the microscope is aligned with the candidate image region 3, the reference images of the 9 fields of view corresponding to the candidate image region 3 are displayed, together with the reference physiological information of each image.
To sum up, the physiological information acquiring method combining the multispectral image and the visible light image according to the embodiment of the present invention acquires a full-field image of physiological content, extracts a plurality of candidate image regions in the full-field image, acquires the visible light image of each candidate image region and the multispectral images of a plurality of fields, registers the multispectral image of each field of each candidate image region with the corresponding visible light image to obtain a reference image of each field of each candidate image region, extracts an image feature of the reference image of each field, determines reference physiological information corresponding to the reference image of each field according to the image feature, determines target physiological information of each candidate image region according to all reference physiological information corresponding to the reference images of all fields of each candidate image region, and further determines physiological information corresponding to the full-field image according to the target physiological information of all candidate image regions in the full-field image. Therefore, the acquisition accuracy of the physiological information is improved, the acquisition mode of the physiological information is expanded, the visual result can be displayed on the full-view image, and the acquisition efficiency of the physiological information of the related scene is improved.
In order to achieve the above embodiments, the present invention further provides a physiological information acquisition apparatus combining a multispectral image and a visible light image, fig. 7 is a schematic structural diagram of the physiological information acquisition apparatus combining a multispectral image and a visible light image according to an embodiment of the present invention, and as shown in fig. 7, the physiological information acquisition apparatus combining a multispectral image and a visible light image includes: an extraction module 710, an acquisition module 720, a registration module 730, a first determination module 740, a second determination module 750, and a third determination module 760, wherein,
an extraction module 710, configured to acquire a full-field image of physiological content, and extract a plurality of candidate image regions in the full-field image;
an obtaining module 720, configured to obtain a visible light image and a multispectral image of multiple fields of view of each of the candidate image regions;
a registration module 730, configured to register the multispectral image of each field of view of each candidate image region with the corresponding visible light image, so as to obtain a reference image of each field of view of each candidate image region.
A first determining module 740, configured to extract image features of the reference image of each view, and determine, according to the image features, reference physiological information corresponding to the reference image of each view;
a second determining module 750, configured to determine target physiological information of each candidate image region according to all the reference physiological information corresponding to the reference images of all the fields of view of each candidate image region;
a third determining module 760, configured to determine physiological information corresponding to the full-view image according to target physiological information of all the candidate image regions in the full-view image.
It should be noted that the foregoing explanation of the method embodiments is also applicable to the real-time apparatus of the present invention, and the implementation principle and technical effect are similar, and are not described herein again.
In order to implement the foregoing embodiment, the present invention further provides a computer device, including: a processor, and a memory for storing processor-executable instructions. Wherein the processor is configured to: the method as described in the above embodiment is performed.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or to implicitly indicate the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless explicitly specified otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are well known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried out in the method of implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and the program, when executed, includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (9)
1. A physiological information acquisition method combining multispectral and visible light images is characterized by comprising the following steps:
acquiring a full-field image of physiological content, and extracting a plurality of candidate image areas in the full-field image;
acquiring a visible light image and multispectral images of a plurality of fields of view of each candidate image region;
registering the multispectral image of each view of each candidate image region with the corresponding visible light image to obtain a reference image of each view of each candidate image region; the reference image is obtained by overlapping the registered multispectral image with the corresponding visible light image;
extracting image features of the reference image of each visual field, and determining reference physiological information corresponding to the reference image of each visual field according to the image features;
determining target physiological information of each candidate image region according to all the reference physiological information corresponding to the reference images of all the fields of view of each candidate image region;
determining physiological information corresponding to the full-view image according to target physiological information of all the candidate image regions in the full-view image;
the acquiring a full-field image of physiological content and extracting a plurality of candidate image regions in the full-field image comprises:
and inputting the full-view image into an image target recognition model, and acquiring the candidate image areas according to an output result of the image target recognition model.
2. The method of claim 1, wherein said registering the multispectral image of each field of view of each of said candidate image regions with the corresponding visible light image comprises:
acquiring a first position feature of a multispectral image of each field of view in the candidate image region;
acquiring a second position characteristic of a visible light image of the candidate image area;
adjusting the position of the visible light image based on the first and second location features such that the multispectral image of each field of view of each of the candidate image regions is registered with the corresponding visible light image.
3. The method of claim 1, wherein the extracting image features of the reference image of each field of view and determining reference physiological information corresponding to the reference image of each field of view based on the image features comprises:
extracting image features of the reference image of each view;
and determining reference physiological information corresponding to the reference image of each visual field according to the image characteristics.
4. The method as claimed in claim 3, wherein said determining the target physiological information of each candidate image region according to all the reference physiological information corresponding to the reference images of all the fields of view of each candidate image region comprises:
determining all reference physiological information corresponding to the reference images of all the visual fields of each candidate image area;
and determining target physiological information of each candidate image area according to all the reference physiological information.
5. The method as claimed in claim 4, wherein the determining the physiological information corresponding to the full-field image according to the target physiological information of all the candidate image regions in the full-field image comprises:
acquiring all target physiological information of all candidate image areas;
and determining physiological information corresponding to the full-view image according to all the target physiological information.
6. The method of claim 1, further comprising:
determining color information corresponding to each of the candidate image regions and reference physiological information corresponding to a reference image for each field of view for each of the candidate image regions;
and adjusting the display color of each candidate image area in the full-view image according to the color information corresponding to each candidate image area.
7. The method of claim 1, further comprising:
responding to a viewing instruction of a target candidate region in the candidate image regions, and acquiring all reference physiological information of all reference images corresponding to the target candidate region;
and displaying all reference images corresponding to the target candidate region, and displaying corresponding reference physiological information in each reference image.
8. A physiological information acquisition apparatus combining multispectral and visible-light images, comprising:
the extraction module is used for acquiring a full-view image of physiological content and extracting a plurality of candidate image areas in the full-view image;
the acquisition module is used for acquiring a visible light image and multispectral images of a plurality of fields of view of each candidate image area;
a registration module for registering the multispectral image of each field of view of each of the candidate image regions with the corresponding visible light image to obtain a reference image of each field of view of each of the candidate image regions; the reference image is obtained by overlapping the registered multispectral image with the corresponding visible light image;
the first determining module is used for extracting image characteristics of the reference image of each visual field and determining reference physiological information corresponding to the reference image of each visual field according to the image characteristics;
the second determination module is used for determining target physiological information of each candidate image area according to all the reference physiological information corresponding to the reference images of all the visual fields of each candidate image area;
a third determining module, configured to determine physiological information corresponding to the full-field image according to target physiological information of all the candidate image regions in the full-field image;
the extraction module is further configured to:
and inputting the full-view image into an image target recognition model, and acquiring the candidate image regions according to an output result of the image target recognition model.
9. A computer arrangement comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method according to any of claims 1-7 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011263000.2A CN112330649B (en) | 2020-11-12 | 2020-11-12 | Physiological information acquisition method and device combining multispectral and visible light image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011263000.2A CN112330649B (en) | 2020-11-12 | 2020-11-12 | Physiological information acquisition method and device combining multispectral and visible light image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112330649A CN112330649A (en) | 2021-02-05 |
| CN112330649B true CN112330649B (en) | 2022-11-04 |
Family
ID=74318114
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011263000.2A Active CN112330649B (en) | 2020-11-12 | 2020-11-12 | Physiological information acquisition method and device combining multispectral and visible light image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112330649B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110298273A (en) * | 2019-06-17 | 2019-10-01 | 深圳大学 | A kind of 3D based on multispectral image refers to vein extracting method and system |
| CN110363176A (en) * | 2019-07-23 | 2019-10-22 | 中国科学院寒区旱区环境与工程研究所 | A kind of image analysis method and device |
| CN111726533A (en) * | 2020-06-30 | 2020-09-29 | RealMe重庆移动通信有限公司 | Image processing method, image processing device, mobile terminal and computer readable storage medium |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6448996B2 (en) * | 2014-11-25 | 2019-01-09 | オリンパス株式会社 | Microscope system |
| WO2017087821A2 (en) * | 2015-11-18 | 2017-05-26 | Lightlab Imaging, Inc. | X-ray image feature detection and registration systems and methods |
| US10580137B2 (en) * | 2018-01-30 | 2020-03-03 | International Business Machines Corporation | Systems and methods for detecting an indication of malignancy in a sequence of anatomical images |
| US10896503B2 (en) * | 2018-03-23 | 2021-01-19 | International Business Machines Corporation | Identification of areas of interest in imaging applications |
| US11657497B2 (en) * | 2019-03-26 | 2023-05-23 | The Johns Hopkins University | Method and apparatus for registration of different mammography image views |
-
2020
- 2020-11-12 CN CN202011263000.2A patent/CN112330649B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110298273A (en) * | 2019-06-17 | 2019-10-01 | 深圳大学 | A kind of 3D based on multispectral image refers to vein extracting method and system |
| CN110363176A (en) * | 2019-07-23 | 2019-10-22 | 中国科学院寒区旱区环境与工程研究所 | A kind of image analysis method and device |
| CN111726533A (en) * | 2020-06-30 | 2020-09-29 | RealMe重庆移动通信有限公司 | Image processing method, image processing device, mobile terminal and computer readable storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 基于图像融合的目标探测与定位技术研究;李洪博;《中国优秀硕士学位论文全文数据库（电子期刊）信息科技辑》;20200115;I138-1902 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112330649A (en) | 2021-02-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Xie et al. | Skin lesion segmentation using high-resolution convolutional neural network | |
| CN107492071B (en) | Medical image processing method and equipment | |
| CN111951221B (en) | Glomerular cell image recognition method based on deep neural network | |
| CN108961229A (en) | Cardiovascular OCT image based on deep learning easily loses plaque detection method and system | |
| US10360474B2 (en) | Image processing device, endoscope system, and image processing method | |
| Banerjee et al. | Automated 3D segmentation of brain tumor using visual saliency | |
| CN111105412B (en) | Intelligent auxiliary system for intestinal polyp detection and identification | |
| CN110619318B (en) | Image processing method, microscope, system and medium based on artificial intelligence | |
| CN112991238B (en) | Food image segmentation method, system and medium based on texture and color mixing | |
| CN111931751A (en) | Deep learning training method, target object identification method, system and storage medium | |
| CN112132166A (en) | Intelligent analysis method, system and device for digital cytopathology image | |
| Zhang et al. | Detection of regions of interest in a high-spatial-resolution remote sensing image based on an adaptive spatial subsampling visual attention model | |
| CN111599444A (en) | Intelligent tongue diagnosis detection method and device, intelligent terminal and storage medium | |
| Asmara et al. | Identification of Toga Plants Based on Leaf Image Using the Invariant Moment and Edge Detection Features | |
| CN114549455A (en) | Image processing method and related device, electronic equipment and storage medium | |
| CN118230166A (en) | Corn canopy organ identification method and canopy phenotype detection method based on improved Mask2YOLO network | |
| CN108765399B (en) | Lesion site recognition device, computer device, and readable storage medium | |
| CN112330649B (en) | Physiological information acquisition method and device combining multispectral and visible light image | |
| Chen et al. | Saliency modeling via outlier detection | |
| Minija et al. | Food recognition using neural network classifier and multiple hypotheses image segmentation | |
| Martins et al. | A multi-object statistical atlas adaptive for deformable registration errors in anomalous medical image segmentation | |
| CN114998980B (en) | Iris detection method and device, electronic equipment and storage medium | |
| Riana et al. | Color canals modification with canny edge detection and morphological reconstruction for cell nucleus segmentation and area measurement in normal Pap smear images | |
| JP6702118B2 (en) | Diagnosis support device, image processing method in the diagnosis support device, and program | |
| CN113344911B (en) | Method and device for measuring size of calculus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |