CN110751112B - Computer vision-based mouse brain map drawing auxiliary system and method - Google Patents
Computer vision-based mouse brain map drawing auxiliary system and method Download PDFInfo
- Publication number
- CN110751112B CN110751112B CN201911017554.1A CN201911017554A CN110751112B CN 110751112 B CN110751112 B CN 110751112B CN 201911017554 A CN201911017554 A CN 201911017554A CN 110751112 B CN110751112 B CN 110751112B
- Authority
- CN
- China
- Prior art keywords
- microscopic image
- mouse brain
- brain
- module
- mouse
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/20—Drawing from basic elements, e.g. lines or circles
- G06T11/206—Drawing of charts or graphs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30016—Brain
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Geometry (AREA)
- Computer Graphics (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a computer vision-based mouse brain map drawing auxiliary system and a computer vision-based mouse brain map drawing auxiliary method, wherein the system comprises a preprocessing module; the input end of the detection module is connected with the first output end of the pretreatment module, and the neuron cell body is detected; the input end of the registration module is connected with the second output end of the preprocessing module, and registration comparison is carried out; the input end of the identification partitioning module is connected with the output end of the registration module, and the mouse brain microscopic image is partitioned; and the first input end of the mapping module is connected with the output end of the detection module, the second input end of the mapping module is connected with the output end of the identification partition module, and the neuron cell bodies and the mouse brain microscopic image are mapped one by one to complete auxiliary drawing of the mouse brain map. The invention solves the problems of insufficient accuracy and incomplete application of the existing software and deep learning algorithm, and realizes the automatic and semi-automatic operation of the auxiliary system for drawing the brain map of the mouse by means of image decomposition and image feature extraction and detection by means of a computer vision algorithm.
Description
Technical Field
The invention relates to the field of computer vision software, in particular to a mouse brain map drawing auxiliary system and method based on computer vision.
Background
The drawing of the brain connection map has important significance for understanding brain functions deeply and simulating a brain neural network and developing brain-like artificial intelligence. As a basic unit of the neural network, neurons can be classified into various types according to differences in morphology, development, connection, function, and gene expression. The neural circuits formed by the interconnections between different types of neurons are technical structures that carry various brain functions. Early in the development of neuroscience research, the brain atlas was lacking in neuronal level resolution, neither distinguishing neuronal species nor verifying the actual presence of synaptic connections or distinguishing direct from indirect connections. However, with the development of genetic labeling methods and microscopic imaging techniques, the specificity and resolution of brain connective mapping by optical imaging are rapidly improved. Rapid advances in labeling and imaging techniques and rapid accumulation of data place higher demands on the hardware and software of data acquisition, storage and analysis, particularly on the identification and reconstruction of neuronal morphology.
Some existing commercial or open source software (e.g. NeuroStudio, simple Neurite Tracer, neuTube, virtual Finger, etc.) that assists manual reconstruction can provide some basic algorithms for automatic reconstruction, but the accuracy is insufficient at full brain scale, cell resolution level reconstruction, manual correction of the reconstruction results is required, and large data processing of TB magnitude cannot be supported. The automatic identification of neuron cell bodies does not have better molding software at present, so the automatic identification mainly depends on manual operation assisted by software.
In recent years, deep learning algorithms based on brain-like neural networks have made a significant breakthrough in image legend and have begun to be applied to reconstruction and recognition of neurons. But the application of these algorithms to neurons is currently limited to specific brain regions or specific functional implementations, such as methods specifically directed to deformed brain image and brain atlas registration (patent publication No. CN103268605A, CN 106920228A) or data set calibration of three-dimensional reconstructed brain atlas (patent publication No. CN108564607 a), and does not form a complete set of brain atlas drawing assistance software.
Disclosure of Invention
The invention aims to provide a mouse brain map drawing auxiliary system and method based on computer vision. The system solves the problems of insufficient accuracy of the existing software and incomplete application of the deep learning algorithm, and aims at special requirements of the drawing of the brain map of the mouse, and the functions of visualization, interaction and automatic and semi-automatic operation of the auxiliary system for drawing the brain map of the mouse are realized by decomposing images of large orders of magnitude and extracting and detecting automatic image features by means of a computer vision algorithm.
In order to achieve the above purpose, the invention provides a mouse brain map drawing auxiliary system based on computer vision, which comprises a preprocessing module, a detection module, a registration module, an identification partition module and a mapping module. The preprocessing module is used for preprocessing the rat brain microscopic image; the input end of the detection module is connected with the first output end of the preprocessing module, and the neuron cell bodies in the rat brain microscopic image are detected based on a computer vision algorithm; the input end of the registration module is connected with the second output end of the pretreatment module, and registration comparison is carried out on the pretreated mouse brain microscopic image and the standard brain map to obtain the mouse brain microscopic image characteristics; the input end of the identification partitioning module is connected with the output end of the registration module, the partitioning in the standard brain map is automatically identified, and the rat brain microscopic image is partitioned according to the characteristic of the rat brain microscopic image, so that the rat brain microscopic image partitioning is obtained; and the mapping module is connected with the output end of the detection module, the second input end of the mapping module is connected with the output end of the identification partition module, neuron cell bodies and mouse brain microscopic image partitions are mapped one by one, partition information of the mouse brain microscopic image partitions is obtained, and auxiliary drawing of mouse brain maps is completed.
The invention also provides a computer vision-based auxiliary method for drawing the mouse brain map, which is realized based on a computer vision-based auxiliary system for drawing the mouse brain map, and comprises the following steps:
step 1: transmitting the rat brain microscopic image to a preprocessing module, preprocessing to obtain a preprocessed rat brain microscopic image, and dividing the preprocessed rat brain microscopic image into two paths of transmission;
step 2: transmitting the first path of preprocessed mouse brain microscopic image to a detection module, and detecting neuron cell bodies in the preprocessed mouse brain microscopic image based on a computer vision algorithm;
step 3: transmitting the second-path preprocessed rat brain microscopic image to a registration module, and carrying out registration comparison with a standard brain map to obtain the characteristic of the rat brain microscopic image;
step 4: transmitting the characteristics of the rat brain microscopic image to an identification partition module, automatically identifying partitions in the standard brain map by the identification partition module, and partitioning the rat brain microscopic image according to the characteristics of the rat brain microscopic image and the partitions in the standard brain map to obtain rat brain microscopic image partitions;
step 5: and transmitting the neuron cell bodies and the mouse brain microscopic image partitions to a mapping module to complete mapping, obtaining partition information of the mouse brain microscopic image partitions, and completing auxiliary drawing of a mouse brain map.
Most preferably, the pretreatment further comprises the steps of:
step 1.1: extracting a picture sequence from a mouse brain microscopic image, and calculating outline Hu moment based on the picture sequence to obtain an outline standard;
step 1.2: and carrying out contour matching and adjustment on the rat brain microscopic image according to contour standards to obtain a pretreated rat brain microscopic image.
Most preferably, the detecting further comprises the steps of:
step 2.1: manually labeling the limited neuron cell bodies in the preprocessed mouse brain microscopic image, and cutting the preprocessed mouse brain microscopic image into a plurality of mouse brain slice microscopic images with the same pixels;
step 2.2: pretraining the multi-layer convolutional neural network through the COCO data set, and fine-tuning the pretrained multi-layer convolutional neural network according to the marked neuron cell bodies to generate a detection neural network;
step 2.3: transmitting the microscopic image of the mouse brain slice to a detection neural network, and detecting neuron cell bodies in the microscopic image of the mouse brain slice based on a computer vision algorithm;
step 2.4: and predicting the neuron cell bodies in the pretreated mouse brain microscopic image according to the neuron cell bodies in the mouse brain slice microscopic image.
Most preferably, detecting neuronal cell bodies in microscopic images of mouse brain sections further comprises the steps of:
step 2.3.1: positioning the center point of a neuron cell body in a rat brain slice microscopic image and taking the size of the center point as a size standard;
step 2.3.2: inputting a detection neural network to predict based on a target detection algorithm in computer vision according to a size standard, and obtaining a confidence thermodynamic diagram;
step 2.3.3: and calculating the confidence thermodynamic diagram through two regression algorithm branches, and calculating the neuron cell bodies in the mouse brain slice microscopic image.
Most preferably, the detection is an automatic/semi-automatic adjustment, and the automatic/semi-automatic detection of the neuron cell bodies is respectively completed by a detection module/manually.
Most preferably, the registration comparison is to compare the features in the preprocessed rat brain microscopic image with the features in the standard brain map, so as to obtain the rat brain microscopic image features in the preprocessed rat brain microscopic image, which are in one-to-one correspondence with the features in the standard brain map.
Most preferably, the comparison is that the characteristics in the preprocessed rat brain microscopic image are finely adjusted according to the characteristics in the standard brain map by a grid deformation fine adjustment algorithm, so that the characteristics in the preprocessed rat brain microscopic image correspond to the characteristics in the standard brain map.
Most preferably, the registration alignment is a fundamental interoperation at the level of accuracy of the size of the partitions of the microimages of the mouse brain to the resolution of the standard brain atlas.
Most preferably, the drawn mouse brain map is a three-dimensional mouse brain map model; the auxiliary drawing of the mouse brain map is completed according to the mouse brain microscopic image partition, and the auxiliary drawing is completed at any angle in a three-dimensional model.
The invention solves the problems of insufficient accuracy of the existing software and incomplete application of the deep learning algorithm, and aims at the special requirements of the mouse brain map drawing, and the functions of visualization, interaction and automatic and semi-automatic operation of the auxiliary system for the mouse brain map drawing are realized by decomposing images of large orders of magnitude and extracting and detecting the features of the automatic images by means of the computer vision algorithm.
Compared with the prior art, the invention has the following beneficial effects:
1. the auxiliary system for drawing the mouse brain map solves the problems that the existing software is insufficient in accuracy and the deep learning algorithm is not fully applied.
2. The auxiliary system for drawing the brain pattern of the mouse is based on expert knowledge of brain nerve researchers, and the large-order images are decomposed and the automatic image features are extracted and detected through a computer vision algorithm, so that the auxiliary system for drawing the brain pattern of the mouse is automatically and semi-automatically operated, the redundant and boring manual burden of the researchers on drawing the brain pattern is reduced, and an effective auxiliary function is provided.
3. The auxiliary system for drawing the brain map of the mouse provided by the invention realizes the functions of visualization and interaction of the auxiliary system for drawing the brain map of the mouse.
Drawings
Fig. 1 is a schematic diagram of each module structure in the auxiliary system for drawing the brain map of the mouse;
FIG. 2 is a schematic flow chart of an auxiliary method for drawing a brain map of a mouse;
FIG. 3 is a schematic diagram of a flow chart of preprocessing a mouse brain spectrogram provided by the invention;
FIG. 4 is a schematic flow chart of the microscopic image detection of the mouse brain provided by the invention;
fig. 5 is a schematic flow chart of microscopic image detection of rat brain sections provided by the invention.
Detailed Description
The invention is further described by the following examples, which are given by way of illustration only and are not limiting of the scope of the invention.
The invention relates to a mouse brain map drawing auxiliary system based on computer vision, which is a visual and interactive software system realized by a machine learning method and the assistance of expert knowledge.
As shown in fig. 1, the system includes a preprocessing module, a detection module, a registration module, an identification partition module, and a mapping module. The preprocessing module is used for preprocessing the rat brain microscopic image; the input end of the detection module is connected with the first output end of the preprocessing module, and the neuron cell bodies in the rat brain microscopic image are detected based on a computer vision algorithm; the input end of the registration module is connected with the second output end of the pretreatment module, and registration comparison is carried out on the pretreated mouse brain microscopic image and the standard brain map to obtain the mouse brain microscopic image characteristics; the input end of the recognition partitioning module is connected with the output end of the registration module, the partitioning in the standard brain map is automatically recognized, and the mouse brain microscopic image is partitioned according to the mouse brain microscopic image characteristics, so that the mouse brain microscopic image partitioning is obtained; and the mapping module is connected with the output end of the detection module, the second input end of the mapping module is connected with the output end of the identification partition module, neuron cell bodies and mouse brain microscopic image partitions are mapped one by one, partition information of the mouse brain microscopic image partitions is obtained, and auxiliary drawing of mouse brain maps is completed.
The system has four basic functions; the four basic functions comprise a registration function, an automatic identification partition function, a detection function and a three-dimensional modeling function; the four basic functions can assist researchers in brain map drawing, reduce the manual complexity of brain map drawing of the whole brain, and also can only provide map auxiliary information of a concerned specific brain region according to the needs.
The invention also provides a computer vision-based mouse brain map drawing auxiliary method, which is realized based on a computer vision-based mouse brain map drawing auxiliary system, as shown in fig. 2, and comprises the following steps:
step 1: and transmitting the rat brain microscopic image to a preprocessing module for preprocessing, obtaining a preprocessed rat brain microscopic image, and dividing the preprocessed rat brain microscopic image into two paths of transmission.
The acquired microscopic images of the mouse brain have the distortion and deviation of the spectrogram of the mouse caused by the difference in construction and imaging technology, and pretreatment is needed to be carried out for registration calculation with the standard brain spectrogram in a registration module. As shown in fig. 3, the pretreatment further comprises the steps of:
step 1.1: extracting a picture sequence from a mouse brain microscopic image, and calculating contour image moment (Hu moment) based on the picture sequence in the mouse brain microscopic image to obtain a contour standard;
step 1.2: and performing contour matching and adjustment on the rat brain microscopic image according to contour criteria, and selecting the pretreated rat brain microscopic image.
Step 2: transmitting the first path of preprocessed mouse brain microscopic image to a detection module, and detecting neuron cell bodies in the preprocessed mouse brain microscopic image based on a computer vision algorithm; as shown in fig. 4, the detection further includes the steps of:
step 2.1: marking the neuron cell bodies with limited quantity in the preprocessed mouse brain microscopic image manually, and cutting the preprocessed mouse brain microscopic image into a plurality of mouse brain slice microscopic images with the same pixel;
step 2.2: the multi-layer convolutional neural network layer is pretrained through a Microsoft common target data set (MicroSoft Common Objects inContext, COCO data set), and is subjected to fine adjustment according to the neuron cell bodies in the marked mouse brain microscopic image, so that a detection neural network is generated, and the detection neural network has high recognition accuracy aiming at the neuron cell bodies in the mouse brain microscopic image under the condition of a small number of data sets.
Step 2.3: transmitting the microscopic image of the mouse brain slice to a detection neural network, and detecting neuron cell bodies in the microscopic image of the mouse brain slice based on a computer vision algorithm; the detection of neuronal cell bodies in microscopic images of mouse brain sections further comprises the steps of:
step 2.3.1: the size of the mouse brain microscopic image is large, the number of neuron cells in the mouse brain microscopic image to be identified is large, the area in the mouse brain microscopic image is small, and the mouse brain microscopic image is cut into a plurality of mouse brain slice microscopic images with the same pixels, so that the accuracy of detecting the neuron cells in the mouse brain microscopic image is improved; taking any one of the rat brain section microscopic images, and positioning the center point and the size of a neuron cell body in the rat brain section microscopic image as a size standard;
step 2.3.2: inputting a detection neural network to predict based on a target detection algorithm in computer vision according to a size standard, and obtaining a confidence thermodynamic diagram;
step 2.3.3: and calculating the confidence thermodynamic diagram through two regression algorithm branches, and calculating the neuron cell bodies in the mouse brain slice microscopic image.
Step 2.4: and predicting the neuron cell bodies in the pretreated mouse brain microscopic image according to the neuron cell bodies in the mouse brain slice microscopic image.
The neuronal cell bodies in the pre-treated mouse brain microscopic image were predicted to be calculated from the dimensions of the neuronal cell bodies in the mouse brain slice microscopic image.
The detection is automatic/semiautomatic adjustment, and the automatic/semiautomatic detection of the neuron cell bodies in the pretreated rat brain microscopic image is finished by a detection module/manually.
Step 3: in order to realize the function recognition and tracking functions of the neuron cell bodies in the mouse brain microscopic image, the partition of the mouse brain map of the neuron cell bodies in the mouse brain microscopic image needs to be defined, so that the mouse brain microscopic image after the second path of pretreatment is transmitted to a registration module to be registered and compared with the standard brain map, and the characteristics of the mouse brain microscopic image are obtained.
Step 4: and transmitting the characteristics of the rat brain microscopic image to an identification partition module, wherein the identification partition module automatically identifies partitions in the standard brain map, and partitions the rat brain microscopic image according to the characteristics of the rat brain microscopic image and the partitions in the standard brain map to obtain rat brain microscopic image partitions.
The registration comparison is to compare the features in the preprocessed mouse brain microscopic image with the features in the standard brain map to obtain the features of the mouse brain microscopic image which are in one-to-one correspondence with the features in the standard brain map in the preprocessed mouse brain microscopic image.
The one-to-one comparison is to finely adjust the characteristics in the preprocessed mouse brain microscopic image according to the characteristics in the standard brain map through a grid deformation fine adjustment algorithm, so that the characteristics in the preprocessed mouse brain microscopic image correspond to the characteristics in the standard brain map.
The registration calculation adopts a machine learning algorithm, and different algorithm models are adopted according to different complexity requirements of brain map drawing, when facing a rat brain image with a large data volume level, the accuracy and the operation efficiency of the algorithm are comprehensively considered.
Registration alignment is a fundamental interoperation at the level of accuracy of the size of the partitions of the microscopic image of the mouse brain to the resolution of the standard brain atlas. The operation of the standard brain atlas is carried out under the accuracy of the resolution of the standard brain atlas, the operation of the rat brain microscopic image is carried out under the resolution of the rat brain microscopic image, the standard brain atlas and the rat brain microscopic image are associated through the image registration of the registration module, and the accuracy of the standard brain atlas and the rat brain microscopic image is not lost in the registration process.
Step 5: and transmitting the neuron cell bodies in the mouse brain microscopic image and the mouse brain microscopic image partition to a mapping module to complete mapping, obtaining partition information of the mouse brain microscopic image partition, and completing auxiliary drawing of a mouse brain map.
The drawn mouse brain map is a three-dimensional mouse brain map model; the auxiliary drawing of the mouse brain map is completed according to the mouse brain microscopic image partition by any angle in the three-dimensional model, so that the visual angle range of the standard brain map can be enlarged, and the angle distortion caused by the mouse brain microscopic image in the detection process is simulated.
The working principle of the invention is as follows:
transmitting the rat brain microscopic image to a preprocessing module, preprocessing to obtain a preprocessed rat brain microscopic image, and dividing the preprocessed rat brain microscopic image into two paths of transmission; transmitting the first path of preprocessed mouse brain microscopic image to a detection module, and detecting neuron cell bodies in the preprocessed mouse brain microscopic image based on a computer vision algorithm; transmitting the second-path preprocessed rat brain microscopic image to a registration module, and carrying out registration comparison with a standard brain map to obtain the characteristic of the rat brain microscopic image; transmitting the characteristics of the rat brain microscopic image to an identification partition module, automatically identifying partitions in the standard brain map by the identification partition module, and partitioning the rat brain microscopic image according to the characteristics of the rat brain microscopic image and the partitions in the standard brain map to obtain rat brain microscopic image partitions; and transmitting the neuron cell bodies and the mouse brain microscopic image partitions to a mapping module to complete mapping, obtaining partition information of the mouse brain microscopic image partitions, and completing auxiliary drawing of a mouse brain map.
In summary, the mouse brain map drawing auxiliary system and the method based on computer vision solve the problems of insufficient accuracy and incomplete application of a deep learning algorithm of the existing software, and aiming at special requirements of mouse brain map drawing, the computer vision algorithm is used for decomposing large-order images and extracting and detecting automatic image features, so that the functions of visualization, interaction and automatic and semi-automatic operation of the mouse brain map drawing auxiliary system are realized.
While the present invention has been described in detail through the foregoing description of the preferred embodiment, it should be understood that the foregoing description is not to be considered as limiting the invention. Many modifications and substitutions of the present invention will become apparent to those of ordinary skill in the art upon reading the foregoing. Accordingly, the scope of the invention should be limited only by the attached claims.
Claims (8)
1. The mouse brain map drawing auxiliary method based on computer vision is characterized by being realized based on a mouse brain map drawing auxiliary system based on computer vision, wherein the mouse brain map drawing auxiliary system based on computer vision comprises the following components: the preprocessing module is used for preprocessing the rat brain microscopic image; the input end of the detection module is connected with the first output end of the preprocessing module, and the neuron cell bodies in the rat brain microscopic image are detected based on a computer vision algorithm; the input end of the registration module is connected with the second output end of the pretreatment module, and registration comparison is carried out on the pretreated mouse brain microscopic image and the standard brain map to obtain the mouse brain microscopic image characteristics; the input end of the identification partitioning module is connected with the output end of the registration module, the partitioning in the standard brain atlas is automatically identified, and the mouse brain microscopic image is partitioned according to the mouse brain microscopic image characteristics, so that the mouse brain microscopic image partition is obtained; the first input end of the mapping module is connected with the output end of the detection module, the second input end of the mapping module is connected with the output end of the identification partition module, the neuron cell bodies and the mouse brain microscopic image partitions are mapped one by one, partition information of the mouse brain microscopic image partitions is obtained, and auxiliary drawing of a mouse brain map is completed;
the auxiliary method for drawing the brain map of the mouse based on computer vision comprises the following steps of:
step 1: transmitting the rat brain microscopic image to the preprocessing module, preprocessing to obtain a preprocessed rat brain microscopic image, and dividing the preprocessed rat brain microscopic image into two paths for transmission;
step 2: transmitting the first path of preprocessed mouse brain microscopic image to the detection module, and detecting neuron cell bodies in the preprocessed mouse brain microscopic image based on a computer vision algorithm;
the method comprises the following steps:
step 2.1: manually labeling the limited neuron cell bodies in the preprocessed mouse brain microscopic image, and cutting the preprocessed mouse brain microscopic image into a plurality of mouse brain slice microscopic images with the same pixels;
step 2.2: pretraining the multi-layer convolutional neural network through the COCO data set, and fine-tuning the pretrained multi-layer convolutional neural network according to the marked neuron cell bodies to generate a detection neural network;
step 2.3: transmitting the mouse brain slice microscopic image to the detection neural network, and detecting neuron cell bodies in the mouse brain slice microscopic image based on a computer vision algorithm; the detecting the neuron cell bodies in the rat brain slice microscopic image further comprises the following steps:
step 2.3.1: positioning the center point of a neuron cell body in the rat brain slice microscopic image and taking the size of the center point as a size standard;
step 2.3.2: inputting the detection neural network for prediction based on a target detection algorithm in computer vision according to the size standard, and obtaining a confidence thermodynamic diagram;
step 2.3.3: calculating the confidence thermodynamic diagram through two regression algorithm branches, and calculating neuron cell bodies in the rat brain section microscopic image
Step 2.4: predicting the neuron cell bodies in the pretreated mouse brain microscopic image according to the neuron cell bodies in the mouse brain slice microscopic image;
step 3: transmitting the second path of preprocessed rat brain microscopic image to the registration module, and carrying out registration comparison with a standard brain map to obtain the characteristic of the rat brain microscopic image;
step 4: transmitting the characteristics of the rat brain microscopic image to the identification partition module, wherein the identification partition module automatically identifies partitions in the standard brain map, and partitions the rat brain microscopic image according to the characteristics of the rat brain microscopic image and the partitions in the standard brain map to obtain rat brain microscopic image partitions;
step 5: and transmitting the neuron cell bodies and the mouse brain microscopic image partitions to the mapping module to complete mapping, obtaining partition information of the mouse brain microscopic image partitions, and completing auxiliary drawing of a mouse brain map.
2. The computer vision-based mouse brain mapping assistance method according to claim 1, wherein the preprocessing further comprises the steps of:
step 1.1: extracting a picture sequence from a mouse brain microscopic image, and calculating outline Hu moment based on the picture sequence to obtain an outline standard;
step 1.2: and performing contour matching and adjustment on the rat brain microscopic image according to the contour standard to obtain the pretreated rat brain microscopic image.
3. The computer vision-based mouse brain mapping auxiliary method according to claim 2, wherein the detection is automatic/semi-automatic adjustment, and the automatic/semi-automatic detection of the neuron cell bodies is completed by the detection module/manually respectively.
4. The computer vision-based mouse brain atlas drawing assisting method according to claim 3, wherein the registration comparison is to compare the features in the preprocessed mouse brain microscopic image with the features in the standard brain atlas, and obtain the features of the mouse brain microscopic image, which are in one-to-one correspondence with the features in the standard brain atlas, in the preprocessed mouse brain microscopic image.
5. The computer vision based mouse brain atlas drawing assisting method according to claim 4, wherein the one-to-one comparison is to fine-tune features in the pre-processed mouse brain microscopic image according to features in the standard brain atlas by a mesh deformation fine-tuning algorithm so that the features in the pre-processed mouse brain microscopic image correspond to the features in the standard brain atlas.
6. The computer vision based mouse brain atlas drawing assistance method of claim 5, wherein the registration alignment is by basic interoperation of the size of the mouse brain microimage partition with the accuracy level of the resolution of the standard brain atlas.
7. The computer vision-based mouse brain map drawing assisting method according to claim 6, wherein the drawn mouse brain map is a three-dimensional mouse brain map model; the auxiliary drawing of the mouse brain map is completed according to the mouse brain microscopic image partition, and the auxiliary drawing is completed at any angle in a three-dimensional model.
8. A computer vision-based mouse brain mapping assistance system for implementing the computer vision-based mouse brain mapping assistance method according to any one of claims 1 to 7, comprising:
the preprocessing module is used for preprocessing the rat brain microscopic image;
the input end of the detection module is connected with the first output end of the preprocessing module, and the neuron cell bodies in the rat brain microscopic image are detected based on a computer vision algorithm;
the input end of the registration module is connected with the second output end of the pretreatment module, and registration comparison is carried out on the pretreated mouse brain microscopic image and the standard brain map to obtain the mouse brain microscopic image characteristics;
the input end of the identification partitioning module is connected with the output end of the registration module, the partitioning in the standard brain atlas is automatically identified, and the mouse brain microscopic image is partitioned according to the mouse brain microscopic image characteristics, so that the mouse brain microscopic image partition is obtained;
and the mapping module is connected with the output end of the detection module, the second input end of the mapping module is connected with the output end of the identification partition module, the neuron cell bodies and the mouse brain microscopic image partitions are mapped one by one, partition information of the mouse brain microscopic image partitions is obtained, and auxiliary drawing of the mouse brain map is completed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911017554.1A CN110751112B (en) | 2019-10-24 | 2019-10-24 | Computer vision-based mouse brain map drawing auxiliary system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911017554.1A CN110751112B (en) | 2019-10-24 | 2019-10-24 | Computer vision-based mouse brain map drawing auxiliary system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110751112A CN110751112A (en) | 2020-02-04 |
| CN110751112B true CN110751112B (en) | 2023-10-03 |
Family
ID=69279718
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911017554.1A Active CN110751112B (en) | 2019-10-24 | 2019-10-24 | Computer vision-based mouse brain map drawing auxiliary system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110751112B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111887813A (en) * | 2020-08-11 | 2020-11-06 | 南通大学 | Method and device for recognizing brain region position of fresh in-vitro tissue |
| CN115049626A (en) * | 2022-06-27 | 2022-09-13 | 华中科技大学苏州脑空间信息研究院 | Map-assisted drawing method, computer-readable storage medium and electronic device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2827167A1 (en) * | 2013-07-17 | 2015-01-21 | Samsung Electronics Co., Ltd | Method and apparatus for selecting seed area for tracking nerve fibers in brain |
| CN106920228A (en) * | 2017-01-19 | 2017-07-04 | 北京理工大学 | The method for registering and device of brain map and brain image |
| CN108197564A (en) * | 2017-12-29 | 2018-06-22 | 复旦大学附属中山医院 | A kind of assessment system and method for drawing clock experiment |
| CN110197729A (en) * | 2019-05-20 | 2019-09-03 | 华南理工大学 | Tranquillization state fMRI data classification method and device based on deep learning |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9600778B2 (en) * | 2013-07-02 | 2017-03-21 | Surgical Information Sciences, Inc. | Method for a brain region location and shape prediction |
| US10643396B2 (en) * | 2016-11-11 | 2020-05-05 | Microbrightfield, Inc. | Methods and software for creating a 3D image from images of multiple histological sections and for mapping anatomical information from a reference atlas to a histological image |
| US10984054B2 (en) * | 2017-07-27 | 2021-04-20 | Robert Bosch Gmbh | Visual analytics system for convolutional neural network based classifiers |
| JP6979533B2 (en) * | 2017-10-02 | 2021-12-15 | ブラックソーン セラピューティクス インコーポレイテッド | Methods and tools for detecting, diagnosing, predicting, predicting, or treating neurobehavioral phenotypes in a subject. |
-
2019
- 2019-10-24 CN CN201911017554.1A patent/CN110751112B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2827167A1 (en) * | 2013-07-17 | 2015-01-21 | Samsung Electronics Co., Ltd | Method and apparatus for selecting seed area for tracking nerve fibers in brain |
| CN106920228A (en) * | 2017-01-19 | 2017-07-04 | 北京理工大学 | The method for registering and device of brain map and brain image |
| CN108197564A (en) * | 2017-12-29 | 2018-06-22 | 复旦大学附属中山医院 | A kind of assessment system and method for drawing clock experiment |
| CN110197729A (en) * | 2019-05-20 | 2019-09-03 | 华南理工大学 | Tranquillization state fMRI data classification method and device based on deep learning |
Non-Patent Citations (1)
| Title |
|---|
| 大鼠脑组织切片的显微反射红外光谱;姚杰，李茜，陈维，刘玉芳，王丹;光谱学与光谱分析;第36卷(第S1期);137-138 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110751112A (en) | 2020-02-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110059558B (en) | Orchard obstacle real-time detection method based on improved SSD network | |
| Li et al. | Cross‐scene pavement distress detection by a novel transfer learning framework | |
| US10204299B2 (en) | Unsupervised matching in fine-grained datasets for single-view object reconstruction | |
| CN105069423B (en) | A kind of human body attitude detection method and device | |
| CN104063702B (en) | Three-dimensional gait recognition based on shielding recovery and partial similarity matching | |
| CN110298387A (en) | Incorporate the deep neural network object detection method of Pixel-level attention mechanism | |
| CN113033520B (en) | Tree nematode disease wood identification method and system based on deep learning | |
| CN110163069B (en) | Lane line detection method for driving assistance | |
| CN107862694A (en) | A kind of hand-foot-and-mouth disease detecting system based on deep learning | |
| Batsos et al. | Recresnet: A recurrent residual cnn architecture for disparity map enhancement | |
| CN109029363A (en) | A kind of target ranging method based on deep learning | |
| CN111915584B (en) | Focus follow-up evaluation method and system based on CT image | |
| CN112418330A (en) | Improved SSD (solid State drive) -based high-precision detection method for small target object | |
| CN110751112B (en) | Computer vision-based mouse brain map drawing auxiliary system and method | |
| CN116843650A (en) | SMT welding defect detection method and system integrating AOI detection and deep learning | |
| CN112749675A (en) | Potato disease identification method based on convolutional neural network | |
| CN117437382B (en) | Updating method and system for data center component | |
| CN114359702A (en) | Method and system for identifying building violation of remote sensing image of homestead based on Transformer | |
| CN111461006A (en) | Optical remote sensing image tower position detection method based on deep migration learning | |
| CN114273826A (en) | Automatic identification method for welding position of large-sized workpiece to be welded | |
| CN116883650A (en) | Image-level weak supervision semantic segmentation method based on attention and local stitching | |
| CN108537825A (en) | A kind of method for tracking target based on transfer learning Recurrent networks | |
| CN115410024A (en) | Power image defect detection method based on dynamic activation thermodynamic diagram | |
| CN113298065B (en) | Eye melanin tumor identification method based on self-supervision learning | |
| CN114494773A (en) | Part sorting and identifying system and method based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |