CN112205968A - Method, device and equipment for measuring myocardial cell contractility based on video - Google Patents
Method, device and equipment for measuring myocardial cell contractility based on video Download PDFInfo
- Publication number
- CN112205968A CN112205968A CN202011063412.1A CN202011063412A CN112205968A CN 112205968 A CN112205968 A CN 112205968A CN 202011063412 A CN202011063412 A CN 202011063412A CN 112205968 A CN112205968 A CN 112205968A
- Authority
- CN
- China
- Prior art keywords
- frame
- image
- depth
- video
- map
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000002107 myocardial effect Effects 0.000 title claims abstract description 49
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000012545 processing Methods 0.000 claims abstract description 41
- 210000004027 cell Anatomy 0.000 claims description 55
- 210000004413 cardiac myocyte Anatomy 0.000 claims description 44
- 238000010009 beating Methods 0.000 claims description 37
- 238000004590 computer program Methods 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 230000008859 change Effects 0.000 claims description 5
- 230000008602 contraction Effects 0.000 claims description 5
- 238000005259 measurement Methods 0.000 claims description 4
- 239000000203 mixture Substances 0.000 claims 2
- 239000002994 raw material Substances 0.000 claims 2
- 238000007877 drug screening Methods 0.000 abstract description 6
- 238000011282 treatment Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 6
- 208000024172 Cardiovascular disease Diseases 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000034994 death Effects 0.000 description 3
- 231100000517 death Toxicity 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 238000000338 in vitro Methods 0.000 description 3
- BHPQYMZQTOCNFJ-UHFFFAOYSA-N Calcium cation Chemical compound [Ca+2] BHPQYMZQTOCNFJ-UHFFFAOYSA-N 0.000 description 2
- 241000282414 Homo sapiens Species 0.000 description 2
- 230000036982 action potential Effects 0.000 description 2
- 229910001424 calcium ion Inorganic materials 0.000 description 2
- 238000002659 cell therapy Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 239000007850 fluorescent dye Substances 0.000 description 2
- 238000001215 fluorescent labelling Methods 0.000 description 2
- PGHMRUGBZOYCAA-ADZNBVRBSA-N ionomycin Chemical compound O1[C@H](C[C@H](O)[C@H](C)[C@H](O)[C@H](C)/C=C/C[C@@H](C)C[C@@H](C)C(/O)=C/C(=O)[C@@H](C)C[C@@H](C)C[C@@H](CCC(O)=O)C)CC[C@@]1(C)[C@@H]1O[C@](C)([C@@H](C)O)CC1 PGHMRUGBZOYCAA-ADZNBVRBSA-N 0.000 description 2
- PGHMRUGBZOYCAA-UHFFFAOYSA-N ionomycin Natural products O1C(CC(O)C(C)C(O)C(C)C=CCC(C)CC(C)C(O)=CC(=O)C(C)CC(C)CC(CCC(O)=O)C)CCC1(C)C1OC(C)(C(C)O)CC1 PGHMRUGBZOYCAA-UHFFFAOYSA-N 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- OYPRJOBELJOOCE-UHFFFAOYSA-N Calcium Chemical group [Ca] OYPRJOBELJOOCE-UHFFFAOYSA-N 0.000 description 1
- 206010007558 Cardiac failure chronic Diseases 0.000 description 1
- 102000004190 Enzymes Human genes 0.000 description 1
- 108090000790 Enzymes Proteins 0.000 description 1
- 206010028980 Neoplasm Diseases 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 229910052791 calcium Inorganic materials 0.000 description 1
- 239000011575 calcium Substances 0.000 description 1
- 201000011510 cancer Diseases 0.000 description 1
- 230000007012 clinical effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000008828 contractile function Effects 0.000 description 1
- 230000009089 cytolysis Effects 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000001727 in vivo Methods 0.000 description 1
- 230000002427 irreversible effect Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 208000010125 myocardial infarction Diseases 0.000 description 1
- 210000000107 myocyte Anatomy 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 238000012402 patch clamp technique Methods 0.000 description 1
- 230000008506 pathogenesis Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000010287 polarization Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 108090000623 proteins and genes Proteins 0.000 description 1
- 102000004169 proteins and genes Human genes 0.000 description 1
- 238000005086 pumping Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
- 238000011269 treatment regimen Methods 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Physics & Mathematics (AREA)
- Animal Behavior & Ethology (AREA)
- Pathology (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physiology (AREA)
- Psychiatry (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a method, a device and equipment for measuring myocardial cell contractility based on video, belonging to the field of myocardial cells, wherein the method comprises the steps of processing the video to obtain an inter-frame parallax image and a 3D depth image of each frame of image in the video; selecting at least one peak point from a 3D depth map of any frame of image; determining pixels corresponding to at least one peak point in an inter-frame disparity map of any frame image; searching pixels in the inter-frame parallax images of the adjacent frame images, determining the positions of the pixels in the inter-frame parallax images of the adjacent frame images, and determining the depth of the pixels in the 3D depth images of the adjacent frame images until the depth of the pixels in the 3D depth images of all the frame images is obtained; and obtaining the contractility of the at least one myocardial cell according to the pixel and the depth of the pixel in the 3D depth map of the whole frame image. The method does not need expensive and complex-operated equipment, can simultaneously measure a plurality of myocardial cells, and is suitable for high-throughput drug screening.
Description
Technical Field
The invention relates to the field of myocardial cells, in particular to a method, a device, equipment and a medium for measuring myocardial cell contractility based on video.
Background
The number of deaths of cardiovascular diseases is about 1770 thousands, the death rate of the cardiovascular diseases is the first in the global range, the death rate is far higher than that of other diseases such as cancer, and the health of human beings in the world is seriously threatened. The heart is an important organ of the human body, the normal pumping of the heart is life-sustaining, and the damage of the heart is irreversible because mature myocardial cells can hardly proliferate and regenerate, which severely restricts the intervention and treatment of cardiovascular diseases. The beating of the heart depends on the beating of the cardiomyocytes, and regular contractile movement is the most important function of the cardiomyocytes.
In recent years, researchers are constantly exploring pathogenesis and treatment strategies of cardiovascular diseases, wherein cell therapy has wide prospects as being used for treating myocardial infarction and end-stage heart diseases, and is receiving wide attention as an emerging treatment means. However, the clinical effect is not ideal at present, and one of the main reasons is the beating of the cardiomyocytes cultured in vitro and the heterogeneity of the cardiomyocytes in vivo. Therefore, measuring the beating of the myocardial cells can bring important breakthrough to the treatment and recovery of the injured heart, high-flux drug screening and the like, and has important significance to the treatment, prevention and control of cardiovascular diseases.
In the field of modern biomedical research and applications, researchers have developed different platforms for measuring the beating of cardiomyocytes.
The first is a microscopic living cell calcium ion concentration detection system based on a fluorescence labeling method (EWAN D.Fowler et al.PNAS 2019, Timothy S Luongo et al.Nature 2017.), which laterally evaluates the contractility of cardiac muscle cells through the change of fluorescence labeling calcium ions, but the detection of the change of fluorescence requires the assistance of instruments such as a laser confocal microscope, a living cell workbench and the like, the equipment tools are expensive, and the physiological characteristics of single cardiac muscle cells can only be evaluated by utilizing the change of calcium signals in the cardiac muscle cells, so that the beating condition of the group cardiac muscle cells cannot be detected.
The second is the traditional patch clamp technique (Kohlhaas Michael et al circulation 2010, Rami Shinnawi et al jacc 2019), which uses a micropipette as a microelectrode to record the electrophysiological activity of the cardiomyocytes, and although a good electrical signal result can be obtained, the operation is complicated, the microelectrode has large damage to the cells, and the cells cannot be accurately manipulated. The method is used for laterally evaluating the beating of a single myocardial cell through an electrical signal in the myocardial cell, and also can not be used for evaluating the beating capacity of the same myocardial cell, and can not be used for experiment high-throughput drug screening and the like.
The third is a microelectrode array (Enrique g. navarree et al circulation2013, Xiaojie dual et al nature nanotechnology 2011.) developed based on patch clamp technology, which cultures cardiomyocytes directly on the microelectrode array, enabling an accurate measurement of the action potential of cardiomyocytes for a long period of time without damage. However, in this method, the position of the electrodes in the microelectrode array is fixed, and the cells grow randomly, so that the position of the cells cannot be selected.
The fourth method is to use the Fel ixgX Cell moving edge detection system to measure the contraction function of the myocardial cells (Ermin Li et al. protein & Cell 2020, Yongkun Zhan et al. JMCC 2018.), which is a software for instrument control and accessory control, FelixGX provides a whole set of data acquisition method through a powerful ASOC-10 USB interface, controls an electric polarizer based on time polarization, automatically measures the difference between the G factor in all polarizer directions and the sample background, and can be used for evaluating the contraction function of the myocardial cells. The method also needs the assistance of a plurality of instruments such as a microscope, a living cell workbench and the like, can only detect the contractile function of a single cell, and cannot evaluate the beating of a plurality of myocardial cells.
In addition, there is a chinese patent publication No. CN105527462A, which discloses a method for measuring action potential and contractility of single living myocardial cells by atomic force microscope, which relies on the expensive atomic microscope equipment, thereby limiting its application in ordinary laboratories.
The above documents and patents have advantages, but have general disadvantages: 1. the contractility of a single cell can be evaluated, the contractility of a plurality of myocardial cells cannot be evaluated, and the beating of the myocardial cells in the heart is consistent, so that the in vitro research is urgently needed to be a method for measuring the beating of the whole myocardial cells; 2. the required detection equipment is complicated and expensive, the operation method is complex, and high-throughput drug screening cannot be generally carried out, so that the function evaluation of the myocardial cells and the application of in-vitro treatment are also hindered. Therefore, a more convenient and faster measurement means is developed, and the method has important significance for the application of cell therapy in medical clinic.
Disclosure of Invention
The invention aims to provide a method, a device and equipment for measuring the contractility of myocardial cells based on video, aiming at the problems that the measurement and evaluation method of the contractility of the myocardial cells in the prior art can not evaluate a plurality of myocardial cells and needs expensive and complicated instruments and equipment.
In order to achieve the purpose, the technical scheme of the invention is as follows:
in one aspect, the present invention provides a method for measuring contractility of cardiomyocytes based on video, comprising the steps of,
processing a video to obtain an inter-frame disparity map and a 3D depth map of each frame of image in the video;
selecting at least one peak point from a 3D depth map of any frame of image;
determining pixels corresponding to the at least one peak point in an inter-frame disparity map of any frame image according to the at least one peak point;
searching the pixel in the inter-frame disparity map of the adjacent frame image, and determining the position of the pixel in the inter-frame disparity map of the adjacent frame image;
determining the depth of the pixel in the 3D depth map of the adjacent frame image according to the position of the pixel in the inter-frame disparity map of the adjacent frame image;
repeating the two steps to obtain the depth of the pixel in the 3D depth map of all the frame images;
and obtaining the contractility of at least one myocardial cell according to the pixel and the depth of the pixel in the 3D depth map of the whole frame image.
Preferably, the step of obtaining the contractile ability of the at least one cardiomyocyte according to the pixels and the depths of the pixels in the 3D depth map of the whole frame image comprises,
constructing a beating data map of at least one myocardial cell by taking time, the pixel and the depth of the pixel in the 3D depth maps of all the frame images as coordinate axes;
and obtaining one or more beating frequency and beating intensity change curves of the myocardial cells according to the beating data graph of the at least one myocardial cell.
Preferably, the step of selecting at least one peak point from the 3D depth map of any frame image comprises,
dividing the 3D depth map of any frame of image into a plurality of regions to be selected;
confirming a region high point with the maximum depth in each region to be selected;
determining whether the region high point is a point with the maximum depth within a preset range by taking each region high point as a center, if so, determining the region high point as a peak point to be selected, otherwise, discarding the region high point;
and determining the at least one peak point from the peak points to be selected according to the sequence from large to small.
Preferably, the video is shot by a monocular camera, and the step of processing the video to obtain the inter-frame disparity map and the 3D depth map of each frame image in the video comprises,
splitting a video into RGB pictures according to a frame rate, wherein the RGB pictures comprise an initial frame image and a plurality of subsequent frame images;
converting all the RGB pictures into gray level images;
calculating the inter-frame disparity map of each subsequent frame image on the basis of the gray scale map of the initial frame image, wherein the inter-frame disparity map is a disparity map between the gray scale map of any subsequent frame image and the gray scale map of the previous frame image;
and obtaining a 3D depth map of the subsequent frame image according to the inter-frame disparity map of the subsequent frame image.
In still another aspect, the present invention provides an apparatus for measuring myocardial cell contractility based on video, comprising,
the video processing module is used for processing the video and obtaining an inter-frame parallax image and a 3D depth image of each frame of image in the video;
the device comprises a peak point selection module, a peak point selection module and a depth value selection module, wherein the peak point selection module is used for selecting at least one peak point from a 3D depth map of any frame of image;
a conversion determination module for mutually determining a peak point in the 3D depth map and a pixel in the inter-frame disparity map;
the searching and confirming module is used for searching and confirming the position of the pixel in the inter-frame disparity map of the adjacent frame image according to the pixel in the inter-frame disparity map of any frame image;
and the data processing module is used for processing the pixels and the depths of the pixels in the 3D depth maps of all the frame images to obtain the contractility of the myocardial cells.
Preferably, the video processing module includes,
the video splitting unit is used for splitting a video into a plurality of frames of images according to a frame rate;
the gray-scale image unit is used for processing each frame of image and obtaining a gray-scale image;
the parallax image unit is used for processing the gray-scale image of each frame of image and obtaining an inter-frame parallax image;
and the 3D depth map unit is used for processing the inter-frame disparity map of each frame of image and obtaining a 3D depth map.
Preferably, the data processing module comprises,
a data map generating unit configured to generate, from the pixel and the depth of the pixel in the 3D depth maps of all the frame images, a jitter data map having the pixel as an X-axis, a time as a Y-axis, and the depth of the pixel in the 3D depth maps of all the frame images as a Z-axis, respectively;
and the curve generating unit is used for determining a curve graph which takes time as an abscissa and takes the depth of one or more pixels in the 3D depth maps of all the frame images as an ordinate according to the jitter data map.
In still another aspect, the present invention provides an apparatus for measuring contractility of cardiomyocytes based on video, comprising,
a processor;
a memory storing an executable computer program of the processor, wherein the processor is configured to perform the steps of the above method via execution of the computer program.
In yet another aspect, the present invention also provides a computer-readable storage medium storing a computer program, characterized in that: which when executed performs the steps of the above-described method.
By adopting the technical scheme, the video image is processed to obtain a 3D depth map, one or more peak points are selected from the 3D depth map of any frame of image, the pixel corresponding to the selected peak point is determined corresponding to the inter-frame disparity map of any frame of image, so that the depth data of the pixel in any frame of image can be obtained, the position of the pixel is continuously searched in the adjacent frame of image, the depth data of the pixel in the adjacent frame of image is further determined, the depth data of the pixel in all the frames of image can be obtained by analogy, the data can be sorted to obtain the contractility of the myocardial cells represented by the pixel, such as the beating frequency and the beating intensity, and as the pixel corresponds to the peak point, the contractility of a plurality of myocardial cells can be obtained by selecting a plurality of peak points, so that the high-throughput drug screening can be carried out; therefore, the method only needs to acquire the video when being implemented, and therefore investment of expensive equipment with complex operation can be avoided.
Drawings
FIG. 1 is a flow chart of a method for measuring contractility of cardiomyocytes based on video according to the present invention;
FIG. 2 is a flow chart of obtaining an inter-frame disparity map and a 3D depth map of an image based on video captured by a monocular camera;
fig. 3 is a gray scale image obtained by performing gray scale processing on a part of RGB images split from the cardiomyocyte beating video;
FIG. 4 is an inter-frame disparity map obtained by processing the grayscale map of FIG. 3;
fig. 5 is a 3D depth map obtained by processing the inter-frame disparity map in fig. 4;
FIG. 6 is a flow chart of a method for selecting at least one peak from a 3D depth map of any frame of image;
fig. 7 is a schematic diagram illustrating a position of a pixel corresponding to a peak point of a 3D depth map of any frame of image in an inter-frame disparity map;
FIG. 8 is a flowchart of a method for obtaining contractility of at least one cardiomyocyte according to a pixel corresponding to a peak and a depth of the pixel in a 3D depth map of all frame images;
FIG. 9 is a graph of cardiomyocyte beat data;
FIG. 10 is a graph showing the variation of beating frequency and intensity of myocardial cells;
FIG. 11 is a schematic diagram of an apparatus for measuring the contractile ability of cardiomyocytes according to the present invention;
FIG. 12 is a diagram of a video processing module according to the present invention;
FIG. 13 is a schematic diagram of a data processing module according to the present invention;
fig. 14 is a block diagram of the apparatus for measuring the contractile ability of cardiac muscle cells based on video according to the present invention.
Detailed Description
The following further describes embodiments of the present invention with reference to the drawings. It should be noted that the description of the embodiments is provided to help understanding of the present invention, but the present invention is not limited thereto. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
Examples of the first aspect of the invention
A method for measuring myocardial cell contractility based on video, as shown in fig. 1, includes step S102, step S104, step S106, step S108, step S1010, step S1012, and step S1014.
The execution main body of the method can be terminal equipment or a server, wherein the server can be an independent server, or a server cluster formed by a plurality of servers, and the like, and the server can be a background server of financial service or online shopping service, or a background server of an application program, and the like. The terminal device can be a mobile terminal device such as a mobile phone or a tablet personal computer and the like, and can also be a device such as a personal computer and the like. In the embodiments of the present specification, the execution subject is taken as an example of a terminal device for detailed description, and for the case of the server, reference may be made to the following related contents, which are not described herein again.
Step S102, processing the video to obtain an inter-frame parallax image and a 3D depth image of each frame of image in the video;
wherein the video of a period of time that the video was shot through high-speed camera devices such as camera the video of the cardiomyocyte condition of beating, this embodiment adopts ISISPM12000KPA optical instrument industry camera (be high-speed camera device) and supporting software system to connect the microscope, places the cardiomyocyte on microscope stand thing platform during the shooting, keeps the desktop steady, gathers the video of beating of cardiomyocyte, and wherein the frame rate is: 134.5 frames/sec, time: for 15 seconds.
Further, in order to obtain clearer video data, the cardiomyocytes in this embodiment are isolated from primary rat or suckling mouse cardiomyocytes by an enzyme lysis method, and after 48 hours, the cardiomyocytes are attached to the wall and repeatedly jumped, and then 5 μ M ionomycin treatment is performed for 5min, and a blank control group which is not subjected to ionomycin treatment is set.
Since videos are divided into monocular camera shooting and two-sided camera shooting, a method and steps for processing a video image shot by a monocular camera to obtain an inter-frame disparity map and a 3D depth map of each frame image are slightly different from a method and steps for processing a video image shot by a binocular camera to obtain an inter-frame disparity map and a 3D depth map of each frame image.
In this embodiment, a video shot by a monocular camera is taken as an example for detailed description, and as shown in fig. 2, step S102 specifically includes step S1022, step S1024, step S1026 and step S1028.
Step S1022, the video is split into RGB pictures according to the frame rate, and the RGB pictures include an initial frame image and a plurality of subsequent frame images.
Step S1024, converting all of the RGB images obtained in step S1022 into grayscale images, as shown in fig. 3;
the RGB image can be converted into a gray image after being converted by the encoder.
Step S1026, calculating an inter-frame disparity map of each subsequent frame image based on the gray-scale map of the initial frame image, as shown in fig. 4, where the inter-frame disparity map is a disparity map between the gray-scale map of any subsequent frame image and the gray-scale map of the previous frame image;
the encoder processes the gray scale map of the image to output a position vector map including data in the standard avi format and a position vector map in the direction indicated by the vector arrow X, Y, Z.
Step 1028, obtaining a 3D depth map of the subsequent frame image according to the inter-frame disparity map of the subsequent frame image, as shown in fig. 5;
the obtained inter-frame parallax image of the subsequent frame image can be used for obtaining the corresponding relation between the pixel position of each frame image and the pixel position of the initial frame image, and the mapping from the image to the other image can be obtained by combining the space field around each pixel in one image with the other image, so that the 3D depth image of the corresponding image is obtained.
However, for videos shot by a binocular camera, methods and steps for obtaining a 3D depth map of the images thereof are disclosed in the prior art, and are not described in detail herein.
On the basis of obtaining the inter-frame disparity map and the 3D depth map of each frame image in the video, the following steps are continued,
step S104, selecting at least one peak point from the 3D depth map of any frame of image, wherein the peak point selects a point capable of representing the beating of the central myocyte in the whole visual field;
specifically, in this embodiment, the step of selecting at least one peak point from the 3D depth map of any frame of image is shown in fig. 6, and includes step S1042, step S1044, step S1046, and step S1048.
Step S1042, dividing the 3D depth map of any frame of image into a plurality of regions to be selected;
for example, the 3D depth map of any frame image is divided into meshes, each mesh having the same or equivalent area.
Step S1042, confirming the area high point with the maximum depth in each area to be selected;
in the step, for each region to be selected, the region high point with the maximum depth in each region to be selected can be selected only by simply comparing the sizes.
Step S1046, taking each region high point as a center, determining whether the region high point is the point with the maximum depth in a preset range taking the region high point as the center, if so, determining the region high point as a peak point to be selected, otherwise, abandoning the region high point;
it will be appreciated that the peak point is the apex of a "mountain peak", and when the tip of the "mountain peak" spans two or more candidate regions, it may occur that the regional high points of the multiple candidate regions all originate from the same "mountain peak", which is clearly contrary to the ultimate goal of selecting the peak point, and therefore it is desirable to avoid that the selected regional high points cannot be non-apex locations of the "mountain peak". Therefore, on the basis of step S1044, it is determined whether there is a point with a higher depth within a preset range (for example, within a range of one circle) with the regional high point as the center, that is, a cross-domain search is performed to exclude the above. And when no point with higher depth exists in a certain range near the high point of the region, the high point of the region can be determined as the top point of a certain peak.
Step S1048, determining at least one peak point from the peak points to be selected according to the descending order.
After the step S1046, the obtained points are all the vertexes of the "peak" in the 3D depth map, but there may be a case where the vertexes of the "peak" are too many, and therefore after the sorting, a preset number of "peak" vertexes are selected from the "peak" as the peak points in the embodiment of the present invention. And through the screening of the steps, the finally obtained peak point can represent the beating of the myocardial cells in the visual field.
Step S106, determining pixels corresponding to at least one peak point in an inter-frame disparity map of any frame image according to the at least one peak point;
since the peak of the 3D depth map of a single frame image cannot reflect the contractile ability of the myocardial cells, the expansion is required to extend to other frame images, but the peak cannot extend between adjacent frame images through the 3D depth map. The peak points in the 3D depth map can actually find corresponding pixels in the inter-frame disparity map, and the pixels can extend between adjacent frame images. Therefore, by performing the intermediate conversion in this step, after the peak point in the 3D depth map of any one frame image is determined, the pixel corresponding to the peak point is determined in the inter-frame disparity map of any one frame image, as shown in fig. 7. The number of the peak points and the number of the pixels are the same, and the peak points and the pixels are in one-to-one correspondence.
Step S108, searching the pixels in the inter-frame parallax images of the adjacent frame images, and determining the positions of the pixels in the inter-frame parallax images of the adjacent frame images;
in step S106, by searching for the inter-frame disparity map of the image adjacent to the image in any frame, the corresponding pixels can be extended to the inter-frame disparity map of the adjacent frame image, and bidirectional extension of the pixels on the time axis can be realized.
Step S1010, determining the depth of a pixel (namely a peak point) in a 3D depth map of an adjacent frame image according to the position of the pixel in an inter-frame disparity map of the adjacent frame image;
in contrast to step S106, in this step, on the basis of determining the position of the corresponding pixel in the inter-frame disparity map of the adjacent frame image, the position of the peak point (which may no longer be the peak point at this time) in the 3D depth map of the adjacent frame image is obtained in reverse, and the corresponding depth is obtained.
Step S1012, repeating the above two steps, that is, repeating step S108 and step S1010, and obtaining the depth of the pixel corresponding to the peak point in the 3D depth maps of all the frame images;
it can be understood that, after the above two steps are performed in a loop, the peak point selected in step S106 may be extended in two directions of the time axis, and then the depth data of the peak point (corresponding to the pixel) in all the frame images may be obtained.
Step 1014, obtaining the contractile ability of at least one cardiomyocyte according to the pixel and the depth of the pixel in the 3D depth map of all the frame images.
When the depth data of the myocardial cells corresponding to the pixels, time (all frame images), and pixels (myocardial cells) in the 3D depth map of all frame images are known, the contractile ability of the myocardial cells can be obtained.
Specifically, in the present embodiment, the contractile ability of the cardiomyocytes refers to the beating strength and beating frequency of the cardiomyocytes. As shown in fig. 8, the step of obtaining the contractile ability of the at least one cardiomyocyte according to the pixels and the depths of the pixels in the 3D depth maps of all the frame images in step S1014 specifically includes step S10142 and step S10142.
Step S10142, constructing a beating data map of at least one cardiomyocyte with the time, the pixel corresponding to the peak point and the depth of the pixel in the 3D depth maps of all the frame images as coordinate axes, as shown in fig. 9;
in step S10144, one or more curves of the beating frequency and the beating intensity of the cardiomyocytes are obtained according to the beating data map of the at least one cardiomyocyte in step S10142, as shown in fig. 10.
Because at least one peak point is selected in step S106, the method of the embodiment of the present invention can obtain the variation curve of the beating frequency and the beating intensity of a single cardiomyocyte, can also obtain the variation curves of the beating frequency and the beating intensity of a plurality of cardiomyocytes at the same time, and is suitable for observing the reaction of the whole cardiomyocyte in high-throughput drug screening.
Examples of the second aspect of the invention
An apparatus for measuring myocardial cell contractility based on video, as shown in fig. 11, specifically includes a video processing module 21, a peak point selecting module 22, a conversion determining module 23, a search confirming module 24, and a data processing module 25.
The video processing module 21 is configured to process a video and obtain an inter-frame disparity map and a 3D depth map of each frame of image in the video;
the peak point selecting module 22 is configured to select at least one peak point from the 3D depth map of any frame of image;
the conversion determining module 23 is configured to determine a peak point in the 3D depth map of any frame of image and a pixel in the inter-frame disparity map from each other;
the searching and confirming module 24 is configured to search and confirm a position of a pixel in the inter-frame disparity map of an adjacent frame image according to the pixel in the inter-frame disparity map of any frame image;
the data processing module 25 is configured to process the pixels and the depths of the pixels in the 3D depth maps of all the frame images, so as to obtain the contractility of the cardiomyocytes.
For a video shot by a monocular camera, as shown in fig. 12, the video processing module 21 in the present apparatus specifically includes a video splitting unit 211, a grayscale map unit 212, a disparity map unit 213, and a 3D depth map unit 214.
The video splitting unit 211 is configured to split a video into multiple frames of RGB images according to a frame rate;
the grayscale map unit 212 is used to process each frame of RGB image and obtain a grayscale map;
the disparity map unit 213 is configured to process the grayscale map of each frame of image and obtain an inter-frame disparity map;
the 3D depth map unit 214 is configured to process the inter-frame disparity map of each frame of image and obtain a 3D depth map.
In this embodiment, as shown in fig. 13, the data processing module 25 specifically includes a data map generating unit 251 and a curve generating unit 252.
The data map generating unit 251 is configured to generate a jitter data map with the pixel corresponding to the peak point as an X-axis, the time as a Y-axis, and the depth of the pixel in the 3D depth maps of all the frame images as a Z-axis, respectively, according to the pixel corresponding to the peak point and the depth of the pixel in the 3D depth maps of all the frame images;
the curve generating unit 252 is configured to determine a graph of one or more pixels, which is plotted with time as abscissa and depth of the pixel in the 3D depth maps of all the frame images as ordinate, according to the above-mentioned jitter data map.
Examples of the third aspect of the invention
An apparatus for measuring myocardial cell contraction ability based on video may be the terminal apparatus or the server provided in the above first aspect.
The device may vary significantly depending on configuration or performance, and may include one or more processors 301 and memory 302, where the memory 302 may store one or more stored applications or data. Memory 302 may be, among other things, transient storage or persistent storage. The application program stored in memory 302 may include one or more modules (not shown), each of which may include a series of computer-executable instructions for the device. Still further, the processor 301 may be arranged in communication with the memory 302 to execute a series of computer-executable instructions in the memory 302 on the device. The apparatus may also include one or more power supplies 303, one or more wired or wireless network interfaces 304, one or more input-output interfaces 305, one or more keyboards 306, as shown in fig. 14.
In particular, in this embodiment, the apparatus comprises a memory, and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may comprise one or more modules, and each module may comprise a series of computer-executable instructions for the scrub and scrub management apparatus, and the one or more programs configured to be executed by the one or more processors comprise steps for performing the method of the first aspect embodiment of the present invention.
In an embodiment of the fourth aspect of the invention,
there is provided a computer readable storage medium for storing a computer program which, when executed, performs the steps of the method in the above-described first aspect embodiment.
The embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. It will be apparent to those skilled in the art that various changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, and the scope of protection is still within the scope of the invention.
Claims (9)
1. A method for measuring myocardial cell contractility based on video, comprising: comprises the following steps of (a) carrying out,
processing a video to obtain an inter-frame disparity map and a 3D depth map of each frame of image in the video;
selecting at least one peak point from a 3D depth map of any frame of image;
determining pixels corresponding to the at least one peak point in an inter-frame disparity map of any frame image according to the at least one peak point;
searching the pixel in the inter-frame disparity map of the adjacent frame image, and determining the position of the pixel in the inter-frame disparity map of the adjacent frame image;
determining the depth of the pixel in the 3D depth map of the adjacent frame image according to the position of the pixel in the inter-frame disparity map of the adjacent frame image;
repeating the two steps to obtain the depth of the pixel in the 3D depth map of all the frame images;
and obtaining the contractility of at least one myocardial cell according to the pixel and the depth of the pixel in the 3D depth map of the whole frame image.
2. The video-based method for measuring contractility of cardiomyocytes according to claim 1, wherein: the step of deriving the contractile ability of the at least one cardiomyocyte according to the pixels and the depths of the pixels in the 3D depth map of the full frame image comprises,
constructing a beating data map of at least one myocardial cell by taking time, the pixel and the depth of the pixel in the 3D depth maps of all the frame images as coordinate axes;
and obtaining one or more beating frequency and beating intensity change curves of the myocardial cells according to the beating data graph of the at least one myocardial cell.
3. The video-based method for measuring contractility of cardiomyocytes according to claim 1, wherein: the step of selecting at least one peak point from the 3D depth map of any frame of image comprises,
dividing the 3D depth map of any frame of image into a plurality of regions to be selected;
confirming a region high point with the maximum depth in each region to be selected;
determining whether the region high point is a point with the maximum depth within a preset range by taking each region high point as a center, if so, determining the region high point as a peak point to be selected, otherwise, discarding the region high point;
and determining the at least one peak point from the peak points to be selected according to the sequence from large to small.
4. The video-based method for measuring contractility of cardiomyocytes according to claim 1, wherein: the video is shot by a monocular camera, and the step of processing the video to obtain the inter-frame disparity map and the 3D depth map of each frame image in the video comprises the steps of,
splitting a video into RGB pictures according to a frame rate, wherein the RGB pictures comprise an initial frame image and a plurality of subsequent frame images;
converting all the RGB pictures into gray level images;
calculating the inter-frame disparity map of each subsequent frame image on the basis of the gray scale map of the initial frame image, wherein the inter-frame disparity map is a disparity map between the gray scale map of any subsequent frame image and the gray scale map of the previous frame image;
and obtaining a 3D depth map of the subsequent frame image according to the inter-frame disparity map of the subsequent frame image.
5. An apparatus for measuring myocardial cell contractility based on video, comprising: comprises the steps of (a) preparing a mixture of a plurality of raw materials,
the video processing module is used for processing the video and obtaining an inter-frame parallax image and a 3D depth image of each frame of image in the video;
the device comprises a peak point selection module, a peak point selection module and a depth value selection module, wherein the peak point selection module is used for selecting at least one peak point from a 3D depth map of any frame of image;
a conversion determination module for mutually determining a peak point in the 3D depth map and a pixel in the inter-frame disparity map;
the searching and confirming module is used for searching and confirming the position of the pixel in the inter-frame disparity map of the adjacent frame image according to the pixel in the inter-frame disparity map of any frame image;
and the data processing module is used for processing the pixels and the depths of the pixels in the 3D depth maps of all the frame images to obtain the contractility of the myocardial cells.
6. The apparatus for video-based measurement of cardiomyocyte contraction ability according to claim 5, wherein: the video processing module comprises a video processing module and a video processing module,
the video splitting unit is used for splitting a video into a plurality of frames of images according to a frame rate;
the gray-scale image unit is used for processing each frame of image and obtaining a gray-scale image;
the parallax image unit is used for processing the gray-scale image of each frame of image and obtaining an inter-frame parallax image;
and the 3D depth map unit is used for processing the inter-frame disparity map of each frame of image and obtaining a 3D depth map.
7. The apparatus for video-based measurement of cardiomyocyte contraction ability according to claim 5, wherein: the data processing module comprises a data processing module and a data processing module,
a data map generating unit configured to generate, from the pixel and the depth of the pixel in the 3D depth maps of all the frame images, a jitter data map having the pixel as an X-axis, a time as a Y-axis, and the depth of the pixel in the 3D depth maps of all the frame images as a Z-axis, respectively;
and the curve generating unit is used for determining a curve graph which takes time as an abscissa and takes the depth of one or more pixels in the 3D depth maps of all the frame images as an ordinate according to the jitter data map.
8. An apparatus for measuring myocardial cell contractility based on video, characterized by: comprises the steps of (a) preparing a mixture of a plurality of raw materials,
a processor;
memory storing an executable computer program of the processor, wherein the processor is configured to perform the steps of the method of any one of claims 1-4 via execution of the computer program.
9. A computer-readable storage medium storing a computer program, characterized in that: the computer program when executed implements the steps of the method of any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011063412.1A CN112205968B (en) | 2020-09-30 | 2020-09-30 | Method, device and equipment for measuring myocardial cell contraction capacity based on video |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011063412.1A CN112205968B (en) | 2020-09-30 | 2020-09-30 | Method, device and equipment for measuring myocardial cell contraction capacity based on video |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112205968A true CN112205968A (en) | 2021-01-12 |
| CN112205968B CN112205968B (en) | 2024-03-08 |
Family
ID=74052026
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011063412.1A Active CN112205968B (en) | 2020-09-30 | 2020-09-30 | Method, device and equipment for measuring myocardial cell contraction capacity based on video |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112205968B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030153823A1 (en) * | 1998-08-25 | 2003-08-14 | Geiser Edward A. | Method for automated analysis of apical four-chamber images of the heart |
| JP2006047317A (en) * | 1994-09-19 | 2006-02-16 | Tsukasa Yamamoto | Method for evaluating myocardial viability, and myocardial tomographic image processing apparatus |
| EP1728856A1 (en) * | 2005-06-01 | 2006-12-06 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | A method for optically determining the dynamic behaviour of contracting muscle cells |
| CN101046470A (en) * | 2007-04-20 | 2007-10-03 | 浙江大学 | Myocardial cell pulsation analyzing process and system |
| JP2013097772A (en) * | 2011-11-07 | 2013-05-20 | Toshiba Corp | Medical image diagnostic device and image processing device |
| US20150265187A1 (en) * | 2013-09-11 | 2015-09-24 | Xerox Corporation | Non-contact monitoring of spatio-temporal respiratory mechanics via depth sensing |
| JP2017021849A (en) * | 2011-02-28 | 2017-01-26 | ソニー株式会社 | Image forming device and method, and program |
| CN110866906A (en) * | 2019-11-12 | 2020-03-06 | 安徽师范大学 | Three-dimensional culture human myocardial cell pulsation detection method based on image edge extraction |
| KR20200068238A (en) * | 2018-12-05 | 2020-06-15 | 재단법인대구경북과학기술원 | Methods and apparatus for analyzing single cell motility |
-
2020
- 2020-09-30 CN CN202011063412.1A patent/CN112205968B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006047317A (en) * | 1994-09-19 | 2006-02-16 | Tsukasa Yamamoto | Method for evaluating myocardial viability, and myocardial tomographic image processing apparatus |
| US20030153823A1 (en) * | 1998-08-25 | 2003-08-14 | Geiser Edward A. | Method for automated analysis of apical four-chamber images of the heart |
| EP1728856A1 (en) * | 2005-06-01 | 2006-12-06 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | A method for optically determining the dynamic behaviour of contracting muscle cells |
| CN101046470A (en) * | 2007-04-20 | 2007-10-03 | 浙江大学 | Myocardial cell pulsation analyzing process and system |
| JP2017021849A (en) * | 2011-02-28 | 2017-01-26 | ソニー株式会社 | Image forming device and method, and program |
| JP2013097772A (en) * | 2011-11-07 | 2013-05-20 | Toshiba Corp | Medical image diagnostic device and image processing device |
| US20150265187A1 (en) * | 2013-09-11 | 2015-09-24 | Xerox Corporation | Non-contact monitoring of spatio-temporal respiratory mechanics via depth sensing |
| KR20200068238A (en) * | 2018-12-05 | 2020-06-15 | 재단법인대구경북과학기술원 | Methods and apparatus for analyzing single cell motility |
| CN110866906A (en) * | 2019-11-12 | 2020-03-06 | 安徽师范大学 | Three-dimensional culture human myocardial cell pulsation detection method based on image edge extraction |
Non-Patent Citations (1)
| Title |
|---|
| VENIUS，J，ET AL: "High Resolution Imaging of the Human Cardiac Conduction System Using Reflectance Confocal Microscopy", TOHOKU JOURNAL OF EXPERIMENTAL MEDICINE, vol. 229, no. 1, pages 67 - 73 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112205968B (en) | 2024-03-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Meijering et al. | Tracking in molecular bioimaging | |
| Schaub et al. | Comparative maps of motion and assembly of filamentous actin and myosin II in migrating cells | |
| Zhu et al. | Variability of action potentials within and among cardiac cell clusters derived from human embryonic stem cells | |
| US20230293079A1 (en) | Electrocardiogram image processing method and device, medium, and electrocardiograph | |
| Sabinina et al. | Three-dimensional superresolution fluorescence microscopy maps the variable molecular architecture of the nuclear pore complex | |
| Bodvard et al. | Continuous light exposure causes cumulative stress that affects the localization oscillation dynamics of the transcription factor Msn2p | |
| Mokhtari et al. | Automated characterization and parameter-free classification of cell tracks based on local migration behavior | |
| WO2021095037A3 (en) | Method and apparatus for visualization of bone marrow cell populations | |
| US20130244318A1 (en) | Automated Transient Image Cytometry | |
| Sun et al. | Segmentation of overlapping chromosome images using U-Net with improved dilated convolutions | |
| US11967063B2 (en) | Automatic bio-specimen inspection system and inspection method thereof | |
| CN112205968B (en) | Method, device and equipment for measuring myocardial cell contraction capacity based on video | |
| JP2023015674A (en) | Internal estimation method of cell mass, program, and image processing device | |
| CN108335732A (en) | A kind of the case recommendation method and its system of OCT images | |
| WO2015008459A1 (en) | Image processing device, method, and program | |
| Liu et al. | The analysis of yeast cell morphology features in exponential and stationary Phase | |
| CN110916649B (en) | Processing device, processing method and detection device for long-range electrocardiogram scatter diagram | |
| Ottman et al. | Quantitative analysis with deep learning segmentation and modeling of 3D electron microscopy data | |
| CN112466437A (en) | Apoplexy information processing system | |
| JP2019024495A (en) | Cardiomyocyte type determination system, and cardiomyocyte type determination method | |
| JPWO2018230615A1 (en) | Image processing apparatus, computer program and image complementing method | |
| US10936788B2 (en) | Display control system and display control method | |
| Tonello et al. | Organic substrates for novel printed sensors in neural interfacing: A measurement method for cytocompatibility analysis | |
| US20220155277A1 (en) | Machine-Learning Program, Method, and Apparatus for Measuring, by Pore Electric Resistance Method, Transient Change in Ion Current Associated with Passage of Target Particles through Pores and for Analyzing Pulse Waveform of Said Transient Change | |
| Liu et al. | Integrating Intra-and Intercellular Simulation of a 2D HL-1 Cardiac Model Based on Embedded GPUs |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |