WO2021151272A1

WO2021151272A1 - Method and apparatus for cell image segmentation, and electronic device and readable storage medium

Info

Publication number: WO2021151272A1
Application number: PCT/CN2020/098966
Authority: WO
Inventors: 谢春梅; 侯晓帅; 李风仪; 王佳平; 南洋
Original assignee: 平安科技（深圳）有限公司
Priority date: 2020-05-20
Filing date: 2020-06-29
Publication date: 2021-08-05
Also published as: CN111696084B; CN111696084A

Abstract

A cell image segmentation method, comprising: performing downsampling operation on an original cell image to obtain a downsampled image (S1), inputting the downsampled image into a pre-constructed convolutional segmentation network model to perform segmentation to obtain a first segmented image (S2), on the basis of a pre-constructed pixel coordinate transformation model and a bilinear interpolation algorithm, upsampling the first segmented image to the same resolution size as the original cell image to obtain a second segmented image (S3), by means of a pre-set geometric constraint and an image feature matching method, merging the second segmented image with the original cell image on corresponding color channels to acquire an upsampled image (S4); and performing morphological algorithm pre-processing on the upsampled image and inputting the result into the convolutional segmentation network model to perform segmentation to obtain a segmented image (S5). The method further relates to the field of blockchain technology, and the upsampled image is stored in a blockchain. The invention is able to solve the problems of too-high cell image resolution and low algorithm calculation efficiency leading to computation speed and segmentation precision being impacted.

Description

Cell image segmentation method, device, electronic equipment and readable storage medium

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on May 20, 2020, the application number is 202010435101.7, and the invention title is "Cell Image Segmentation Method, Apparatus, Electronic Equipment, and Readable Storage Medium", and its entire contents Incorporated in the application by reference.

Technical field

This application relates to the field of artificial intelligence technology, and in particular to a method, device, electronic device, and readable storage medium for cell image segmentation.

Background technique

With the penetration and application of deep learning in the field of medical imaging, AI technology can help doctors locate lesion cells to analyze the condition, and assist doctors in making accurate and rapid diagnosis. At present, AI applications in the field of medical imaging mainly focus on lung nodules, fundus, and liver. With the continuous advancement of AI technology and the increasing clinical demand, AI technology has also been applied in digital pathological diagnosis.

The inventor realizes that in clinical tumor cell detection, the patient takes CT first, and the doctor judges the patient by looking at whether there are tumor cells in the CT image based on his own experience. However, because the CT image is a series of frames with a large number, In addition, tumor cells tend to be relatively small in the entire CT image, and the contrast is not high. Therefore, doctors need to spend a lot of time to observe and judge. The reason is that deep learning algorithms need to perform a large number of feature calculations, so the segmentation accuracy is not high under the premise of occupying computer computing resources.

Summary of the invention

The embodiments of the present application provide a cell image segmentation method, device, electronic equipment, and computer-readable storage medium.

A cell image segmentation method provided in this application includes:

Perform a down-sampling operation on the original cell image to obtain a down-sampled image;

Input the down-sampled image into a pre-built convolutional segmentation network model for segmentation to obtain a first segmentation map;

According to a pre-built pixel coordinate conversion model and a bilinear interpolation algorithm, up-sampling the first segmentation image to the same resolution size as the original cell image to obtain a second segmentation image;

Combining the second segmentation map and the original cell image on the corresponding color channel by a preset geometric constraint and image feature matching method to obtain an up-sampled image;

The up-sampled image is preprocessed by a morphological algorithm and input into the convolutional segmentation network model for segmentation to obtain a segmented image.

This application also provides an electronic device, which includes:

At least one processor; and,

A memory communicatively connected with the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the cell image segmentation method as described below:

The up-sampled image is preprocessed by a morphological algorithm, and input to the convolutional segmentation network model for segmentation to obtain a segmented image.

This application also provides a computer-readable storage medium, including a storage data area and a storage program area. The storage data area stores data created according to the use of blockchain nodes, and the storage program area stores a computer program, wherein the computer When the program is executed by the processor, the cell image segmentation method as described below is realized:

The present application also provides a cell image segmentation device, which includes:

The down-sampling module is used to down-sample the original cell image to obtain the down-sampled image;

A low-resolution segmentation module, configured to input the down-sampled image into a pre-built convolutional segmentation network model for segmentation to obtain a first segmentation map;

The up-sampling segmentation module is used to up-sample the first segmentation image to the same resolution size as the original cell image according to a pre-built pixel coordinate conversion model and a bilinear interpolation algorithm to obtain a second segmentation image;

The cell image segmentation module is used to combine the second segmentation map and the original cell image on the corresponding color channel through a preset geometric constraint and image feature matching method to obtain an up-sampled image, and the up-sampling The image is preprocessed by a morphological algorithm, and is input to the convolutional segmentation network model for segmentation to obtain a segmented image.

Description of the drawings

FIG. 1 is a schematic flowchart of a cell image segmentation method provided by an embodiment of the application;

2 is a detailed flowchart of step S2 of the cell image segmentation method provided by an embodiment of the application;

3 is a detailed flowchart of step S4 of the cell image segmentation method provided by an embodiment of the application;

FIG. 4 is a detailed flow diagram of the preprocessing of the morphological algorithm of the cell image segmentation method provided by an embodiment of the application

5 is a schematic diagram of modules of a cell image segmentation device provided by an embodiment of the application;

6 is a schematic diagram of the internal structure of an electronic device provided by an embodiment of the application;

The realization of the objectives, functional characteristics, and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

Detailed ways

It should be understood that the specific embodiments described here are only used to explain the present application, and are not used to limit the present application.

This application provides a method for cell image segmentation. Referring to FIG. 1, it is a schematic flowchart of a cell image segmentation method provided by an embodiment of this application. The method can be executed by a device, and the device can be implemented by software and or hardware.

In this embodiment, the cell image segmentation method includes:

S1. Acquire an original cell image, and perform a down-sampling operation on the original cell image to obtain a down-sampled image.

In the embodiment of the present application, the original cell image may be a CT image of a pathological site obtained through a machine scan of the radiology department of a hospital, such as a CT image of a tumor site. X-rays are emitted by the machine of the radiology department of the hospital. It is captured by the X-ray detector, and the CT image of the tumor site is obtained according to the difference between the X-ray transmittance of the tumor and the X-ray transmittance of other organs.

In detail, the performing down-sampling operation on the original cell image to obtain the down-sampled image includes: performing down-sampling operation on the original cell image with a size of M×N according to a set down-sampling ratio s to obtain a size of

Of the down-sampled image, where s is the common divisor of M and N.

For example, the resolution size of the original cell image is 1000*1000, after the downsampling operation of the downsampling ratio of 10, the resolution size of the down-sampled image obtained becomes 100*100.

S2. Input the down-sampled image into a pre-built convolutional segmentation network model for segmentation to obtain a first segmentation map.

In the embodiment of the present application, the convolutional segmentation network model is a two-cascade network model constructed based on an improved full convolutional neural network (Convolutional Networks for Biomedical Image Segmentation, U-net network for short). The improved U-net network mainly adds a low-resolution fully connected layer to the traditional U-net network to achieve the purpose of roughly segmenting the down-sampled image, and then cascade the standard convolutional neural network model Perform finer segmentation to obtain the first segmentation map.

In detail, the construction process of the pre-built convolutional segmentation network model includes: cascading fully connected layers in the fully convolutional neural network according to the cascading rules set in the preview, and adding the multi-layer convolutional neural network to the The fully convolutional neural network of the fully connected layer is cascaded to obtain the convolutional segmentation network model.

For example, after adding a fully connected layer to the fully connected layer, the standard VGG network (Very deep convolutional networks) is added after the fully connected layer to obtain a convolutional segmentation network model.

The VGG network is a standard convolutional neural network, which is often used in feature extraction and image segmentation. Among them, the most widely used are VGG16 and VGG19, which represent 16 and 19 layers of convolutional network respectively.

Further, please refer to the detailed flowchart of FIG. 2 in the description of the accompanying drawings. The input of the downsampled image into a pre-built convolutional segmentation network model for segmentation to obtain a first segmentation map includes:

S21: Perform a convolution operation on the down-sampled image through the convolutional segmentation network model to generate a down-sampled convolution feature map;

S22: Perform a deconvolution operation on the down-sampled convolution feature map to obtain a deconvolution feature map;

S23. Input the deconvolution feature map into the classification function of the convolution segmentation network model, and calculate the probability value of each segmentation area of the downsampled image;

S24. Segment the down-sampled image according to the probability value of each segmented region to generate the first segmentation map.

In detail, the convolution operation is as follows:

a _l = f(w _l *a _l-1 +b _l )

Where a _l is the output value of the convolution operation, f(·) is the activation function of the convolution operation, w _l is the convolution kernel, * represents the convolution operation, b _l is the bias parameter, and a _{l -1} is the pixel value of the down-sampled image.

The deconvolution is also called transposed, and its calculation process is just the opposite of the convolution operation.

Further, the softmax classification function is:

Where m represents the number of pixels in the deconvolution feature map, ω represents the preset weight value, x represents the deconvolution feature map, K represents the preset number of divided regions, and I{·} is an indicative function , Y ⁽ⁱ⁾ represents the probability value of the i-th segmented region.

Further, after the probability value of each segmentation area is obtained, the embodiment of the present application can segment the down-sampled image according to the probability value of each segmentation area. For example, the down-sampled image is calculated to be divided into 10 segmentation areas, and pass The above operation generates 100 segmented regions and probability values corresponding to the 100 segmented regions, and extracts the 10 segmented regions with the highest probability value to obtain the first segmentation map.

S3. According to the pre-built pixel coordinate conversion model and the bilinear interpolation algorithm, the first segmentation image is up-sampled to the same resolution size as the original cell image to obtain a second segmentation image.

In detail, the pixel coordinate conversion model is f(x,y)=b1+b2x+b3y+b4xy, where (x,y) is the coordinate of the pixel of the original cell image in the coordinate system, b1, b2, b3, and b4 are preset coefficients. In the embodiment of the present application, the coordinate conversion of the original cell image is performed by using the pixel point coordinate conversion model.

After the pixel point coordinate conversion is completed, the embodiment of the present application uses the currently disclosed bilinear interpolation algorithm to insert the pixel point after the pixel point coordinate conversion is completed into the first segmentation image to obtain the second segmentation image.

S4. Combining the second segmentation map and the original cell image on the corresponding color channel through a preset geometric constraint and image feature matching method to obtain an up-sampled image.

In detail, the S4 please refer to the detailed process of Figure 3 in the accompanying drawings, including:

S41. According to a preset matching rule, select SIFT (Scale-invariant feature transform) feature points from the second segmentation map, and sequentially match the SIFT feature points of the original cell image to obtain The original set of matching pairs.

In detail, the matching rule can have many rule settings, such as taking one SIFT key point A1 in the second segmentation image, and finding the first two SIFT key points B1 and B2 that are closest to the Euclidean distance in the original cell image , Get two matched pairs A1-B1 and A1-B2.

S42. Calculate the interior point rate of each group of matching pairs in the original matching pair set, and eliminate matching pairs whose interior point rate is less than the preset value α to obtain a primary matching pair set;

Similarly, as the above two matched pairs A1-B1 and A1-B2, the ratio value obtained by dividing the shortest distance B1 by the second short distance B2 in Euclidean distance is used. If the ratio value is less than the threshold T, then these two groups are accepted For the matching pairs A1-B1 and A1-B2, if the ratio value is greater than the threshold T, then these two sets of matching pairs A1-B1 and A1-B2 will be eliminated.

S43. Calculate the basic matrix of the primary matched pair set according to the primary matched pair set, calculate the corresponding rank according to the basic matrix, and eliminate the matched pairs whose rank is greater than a preset rank threshold γ to obtain a standard matched pair set.

In the field of pictures and computer vision, the fundamental matrix (Fundamental matrix) is generally a 3×3 matrix that represents the correspondence between pixels. The calculation of the fundamental matrix can use the currently published random sampling consistency and minimum Two multiplication. The rank number of the basic matrix is the number of vectors contained in the linearly independent maximal group in the basic matrix, that is, the rank number. Through the comparison between the rank number and the rank threshold γ, the basic matrix is eliminated if the basic matrix is not satisfied. The matching pairs of, so as to obtain the standard matching pair set.

S44. Insert the standard matching pair set into the second segmentation image according to a preset insertion rule to obtain the up-sampled image.

It can be seen from the above that if the above two sets of matching pairs A1-B1 and A1-B2 are in the standard matching pair set, then the two SIFT key points B1 and B2 corresponding to the pixels in the original cell image are extracted , Insert the corresponding pixel into the second segmentation image, and when the insertion operation of all standard matching pairs in the standard matching pair set is completed, the up-sampled image is obtained. It should be emphasized that, in order to further ensure the privacy and security of the above-mentioned up-sampled image, the above-mentioned up-sampled image may also be stored in a node of a blockchain.

S5. Perform preprocessing of the up-sampled image with a morphological algorithm, and input it into the convolutional segmentation network model for segmentation to obtain a segmented image.

In detail, please refer to the detailed process of morphological algorithm preprocessing in FIG. 4. The preprocessing of the up-sampled image on the morphological algorithm includes:

S51: Perform denoising filtering preprocessing on the up-sampled image to obtain a first preprocessed image;

S52: Perform a binarization operation and a reversal operation on the first pre-processed image by using a preset large law threshold to generate a second pre-processed image;

S53. Smooth the boundary of the second pre-processed image through a morphological corrosion operation, and fill in the void formed by the second pre-processed image during the smoothing process through a morphological expansion operation, to obtain a result after the pre-processing operation The upsampled image.

Among them, the big law threshold is also called the maximum between-class variance method (otsu), which is an adaptive threshold segmentation method. It is mainly assumed that the image is divided into two categories, and then an optimal threshold is calculated to divide the image into two The class maximizes the variance between the classes. For example, this application uses the large law threshold to divide the image into black and white, that is, the binarization operation.

Further, the morphological expansion operation is to obtain the maximum value of the local (such as the boundary of the application) of the second preprocessed image, and perform the replacement operation on the boundary of the application according to the maximum value, and so on, the morphology Learning the corrosion operation is to find the local minimum value of the second pre-processed image (such as the boundary of this application), and perform a replacement operation on the image boundary according to the minimum value.

The preprocessed up-sampled image is input into the convolutional segmentation network model for segmentation, and the segmentation method is the same as the above-mentioned S2 operation step, until the segmented image is obtained.

The embodiment of the application first performs a down-sampling operation on the original cell image to obtain a down-sampled image. The down-sampling operation reduces the resolution of the original cell image and reduces the subsequent calculation pressure, and at the same time, in order to prevent the reduction of the resolution of the original cell image from affecting the subsequent Cell image segmentation accuracy, first use the pre-built convolutional segmentation network model for the first segmentation to obtain the first segmentation image, merge the first segmentation image with the original cell image to obtain an up-sampled image, and use the convolutional segmentation network The model performs the second segmentation to obtain the segmented image. Since the second segmentation is based on the original cell image with reduced resolution, relatively speaking, the convolutional segmentation network model does not require a lot of calculations. In addition, the original cell image and the first The segmentation maps are merged to provide a segmentation direction for the second segmentation, so the accuracy of cell image segmentation is improved. Therefore, the vehicle damage assessment method, device, and computer-readable storage medium proposed in this application can solve the problem of high resolution of CT images and low algorithm calculation efficiency, which affects calculation speed and style accuracy.

As shown in FIG. 5, it is a functional block diagram of the cell image segmentation device of the present application.

The cell image segmentation device 100 described in this application can be installed in an electronic device. According to the implemented functions, the cell image segmentation device may include a down-sampling module 101, a low-resolution segmentation module 102, an up-sampling segmentation module 103, and a cell image segmentation module 104. The module in this application can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can complete fixed functions, and are stored in the memory of the electronic device.

In this embodiment, the functions of each module/unit are as follows:

The down-sampling module 101 is used to perform down-sampling operations on the original cell image to obtain a down-sampled image;

The low-resolution segmentation module 102 is configured to input the down-sampled image into a pre-built convolutional segmentation network model for segmentation to obtain a first segmentation map;

The up-sampling segmentation module 103 is used to up-sample the first segmentation image to the same resolution size as the original cell image according to the pre-built pixel coordinate conversion model and the bilinear interpolation algorithm to obtain a second segmentation image ；

The cell image segmentation module 104 is configured to combine the second segmentation map and the original cell image on the corresponding color channel by using a preset geometric constraint and image feature matching method to obtain an up-sampled image, and to upgrade the The sampled image is preprocessed by a morphological algorithm, and is input to the convolutional segmentation network model for segmentation to obtain a segmented image. It should be emphasized that, in order to further ensure the privacy and security of the above-mentioned up-sampled image, the above-mentioned up-sampled image may also be stored in a node of a blockchain.

In detail, the specific implementation steps of each module of the cell image segmentation device can refer to the description of the relevant steps in the embodiment corresponding to FIG. 1, which will not be repeated here.

As shown in FIG. 6, it is a schematic diagram of the structure of the electronic device of the present application.

The electronic device 1 may include a processor 10, a memory 11, and a bus, and may also include a computer program stored in the memory 11 and running on the processor 10, such as a cell image segmentation program 12.

Wherein, the memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), magnetic memory, magnetic disk, CD etc. The memory 11 may be an internal storage unit of the electronic device 1 in some embodiments, for example, a mobile hard disk of the electronic device 1. In other embodiments, the memory 11 may also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart media card (SMC), and a secure digital (Secure Digital) equipped on the electronic device 1. , SD) card, flash card (Flash Card), etc. Further, the memory 11 may also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 can be used not only to store application software and various data installed in the electronic device 1, such as cell image segmentation codes, etc., but also to temporarily store data that has been output or will be output.

The processor 10 may be composed of integrated circuits in some embodiments, for example, may be composed of a single packaged integrated circuit, or may be composed of multiple integrated circuits with the same function or different functions, including one or more Combinations of central processing unit (CPU), microprocessor, digital processing chip, graphics processor, and various control chips, etc. The processor 10 is the control unit of the electronic device, which uses various interfaces and lines to connect the various components of the entire electronic device, and runs or executes programs or modules stored in the memory 11 (such as executing Cell image segmentation, etc.), and call data stored in the memory 11 to execute various functions of the electronic device 1 and process data.

The bus may be a peripheral component interconnect standard (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus and so on. The bus is configured to implement connection and communication between the memory 11 and at least one processor 10 and the like.

FIG. 6 only shows an electronic device with components. Those skilled in the art can understand that the structure shown in FIG. 6 does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown in the figure. Components, or a combination of certain components, or different component arrangements.

For example, although not shown, the electronic device 1 may also include a power source (such as a battery) for supplying power to various components. Preferably, the power source may be logically connected to the at least one processor 10 through a power management device, thereby controlling power The device implements functions such as charge management, discharge management, and power consumption management. The power supply may also include any components such as one or more DC or AC power supplies, recharging devices, power failure detection circuits, power converters or inverters, and power status indicators. The electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

Further, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and/or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is usually used in the electronic device 1 Establish a communication connection with other electronic devices.

Optionally, the electronic device 1 may also include a user interface. The user interface may be a display (Display) and an input unit (such as a keyboard (Keyboard)). Optionally, the user interface may also be a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display can also be appropriately called a display screen or a display unit, which is used to display the information processed in the electronic device 1 and to display a visualized user interface.

It should be understood that the embodiments are only for illustrative purposes, and are not limited by this structure in the scope of the patent application.

The cell image segmentation 12 stored in the memory 11 in the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can realize:

Specifically, the specific method for the processor 10 to implement the foregoing instructions includes:

Step 1: Obtain an original cell image, and perform a down-sampling operation on the original cell image to obtain a down-sampled image.

Of the down-sampled image, where s is the common divisor of M and N.

Step 2: Input the down-sampled image into a pre-built convolutional segmentation network model for segmentation to obtain a first segmentation image.

Further, the inputting the down-sampled image into a pre-built convolutional segmentation network model for segmentation to obtain a first segmentation map includes:

Step A: Perform a convolution operation on the down-sampled image through the convolutional segmentation network model to generate a down-sampled convolution feature map;

Step B: Perform a deconvolution operation on the down-sampled convolution feature map to obtain a deconvolution feature map;

Step C: Input the deconvolution feature map into the softmax classification function in the convolution segmentation network model, and calculate the probability value of each segmentation area of the downsampled image;

Step D: Segment the down-sampled image according to the probability value of each segmented region to generate the first segmentation map.

In detail, the convolution operation is as follows:

a _l = f(w _l *a _l-1 +b _l )

Further, the softmax classification function is:

Step 3: According to the pre-built pixel coordinate conversion model and the bilinear interpolation algorithm, the first segmentation image is up-sampled to the same resolution size as the original cell image to obtain a second segmentation image.

Step 4: Using a preset geometric constraint and image feature matching method, the second segmentation image and the original cell image are combined on the corresponding color channel to obtain an up-sampled image.

In detail, the combination of the second segmentation map and the original cell image on the corresponding color channel through a preset geometric constraint and image feature matching method to obtain an up-sampled image includes:

Step a: According to preset matching rules, select SIFT (Scale-invariant feature transform) feature points from the second segmentation map, and sequentially match them with the SIFT feature points of the original cell image. Get the original matching pair set.

In detail, there are many matching rules, such as taking one SIFT key point A1 in the second segmentation image, and finding the first two SIFT key points B1 and B2 with the closest Euclidean distance in the original cell image to obtain two sets of matches For A1-B1 and A1-B2.

Step b: Calculate the interior point rate of each matching pair in the original matching pair set, and eliminate the matching pairs whose interior point rate is less than the preset value α to obtain the primary matching pair set.

Step c. Calculate the basic matrix of the primary matched pair set according to the primary matched pair set, calculate the corresponding rank according to the basic matrix, and eliminate the matched pairs whose rank is greater than the preset rank threshold γ to obtain a standard match Right set.

Step d: Insert the standard matching pair set into the second segmentation map according to a preset insertion rule to obtain the up-sampled image.

It can be seen from the above that if the above two sets of matching pairs A1-B1 and A1-B2 are in the standard matching pair set, then the two SIFT key points B1 and B2 corresponding to the pixels in the original cell image are extracted , Insert the corresponding pixel into the second segmentation image, and when the insertion operation of all standard matching pairs in the standard matching pair set is completed, the up-sampled image is obtained.

Step 5. The up-sampled image is preprocessed by a morphological algorithm, and input into the convolutional segmentation network model for segmentation to obtain a segmented image.

In detail, the preprocessing of the up-sampled image with a morphological algorithm includes:

Performing denoising filtering preprocessing on the upsampled image to obtain a first preprocessed image;

Performing a binarization operation and a reversal operation on the first preprocessed image by using a preset large law threshold to generate a second preprocessed image;

The boundary of the second pre-processed image is smoothed through a morphological erosion operation, and the hole formed by the second pre-processed image during the smoothing process is filled in through a morphological expansion operation to obtain the pre-processed image Upsample the image.

The pre-processed up-sampled image is input into the convolutional segmentation network model for segmentation, and the segmentation method is the same as the above step 2 until the segmented image is obtained.

Further, if the integrated module/unit of the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) ). The computer-readable storage medium may be non-volatile or volatile.

In the several embodiments provided in this application, it should be understood that the disclosed equipment, device, and method may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional modules in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware, or may be implemented in the form of hardware plus software functional modules.

For those skilled in the art, it is obvious that the present application is not limited to the details of the foregoing exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or basic characteristics of the application.

Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of this application is defined by the appended claims rather than the above description, and therefore it is intended to fall into the claims. All changes in the meaning and scope of the equivalent elements of are included in this application. Any associated diagram marks in the claims should not be regarded as limiting the claims involved.

The blockchain referred to in this application is a new application mode of computer technology such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. Blockchain, essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information for verification. The validity of the information (anti-counterfeiting) and the generation of the next block. The blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.

In addition, it is obvious that the word "including" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claims can also be implemented by one unit or device through software or hardware. The second class words are used to indicate names, and do not indicate any specific order.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the application and not to limit them. Although the application has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the application can be Make modifications or equivalent replacements without departing from the spirit and scope of the technical solution of the present application.

Claims

A cell image segmentation method, wherein the method includes:

Perform a down-sampling operation on the original cell image to obtain a down-sampled image;

Input the down-sampled image into a pre-built convolutional segmentation network model for segmentation to obtain a first segmentation map;

According to a pre-built pixel coordinate conversion model and a bilinear interpolation algorithm, up-sampling the first segmentation image to the same resolution size as the original cell image to obtain a second segmentation image;

Combining the second segmentation map and the original cell image on the corresponding color channel by a preset geometric constraint and image feature matching method to obtain an up-sampled image;

The up-sampled image is preprocessed by a morphological algorithm and input into the convolutional segmentation network model for segmentation to obtain a segmented image.
The cell image segmentation method according to claim 1, wherein the performing down-sampling operation on the original cell image to obtain the down-sampled image comprises:

Perform a down-sampling operation on the original cell image with a size of M×N according to the preset down-sampling ratio s, and the size is
The down-sampled image of;

Among them, s is the common divisor of M and N.
The cell image segmentation method according to claim 1, wherein the construction process of the pre-built convolutional segmentation network model comprises:

According to the cascade rules set in the preview, cascade the fully connected layers in the fully convolutional neural network;

The convolutional segmentation network model is obtained by cascading the multi-layer convolutional neural network and the fully-convolutional neural network to which the fully-connected layer has been added.
The cell image segmentation method according to claim 3, wherein said inputting said down-sampled image into a pre-built convolutional segmentation network model for segmentation to obtain a first segmentation map comprises:

Performing a convolution operation on the down-sampled image by using the convolutional segmentation network model to generate a down-sampled convolution feature map;

Performing a deconvolution operation on the down-sampled convolution feature map to obtain a deconvolution feature map;

Input the deconvolution feature map into the classification function of the convolution segmentation network model, and calculate the probability value of each segmentation area of the downsampled image;

The down-sampled image is segmented according to the probability value of each segmented region to generate the first segmentation map.
The cell image segmentation method of claim 1, wherein the up-sampled image is stored in a blockchain, and the second segmentation image is compared with the second segmentation image through a preset geometric constraint and image feature matching method. The original cell images are merged on the corresponding color channels to obtain an up-sampled image, including:

According to a preset matching rule, select SIFT feature points from the second segmentation map, and sequentially match them with the SIFT feature points of the original cell image to obtain an original matching pair set;

Calculate the interior point rate of each matched pair in the original matching pair set, and remove the matching pairs whose interior point rate is less than the preset value α to obtain the primary matching pair set;

Calculating a basic matrix of the primary matching pair set according to the primary matching pair set, calculating the rank of the basic matrix, and removing the matching pairs whose rank is greater than a preset rank threshold to obtain a standard matching pair set;

According to a preset insertion rule, the standard matching pair set is inserted into the second segmentation image to obtain the up-sampled image.
The cell image segmentation method according to any one of claims 1 to 5, wherein the preprocessing of the up-sampled image on a morphological algorithm comprises:

Performing denoising filtering preprocessing on the upsampled image to obtain a first preprocessed image;

Performing a binarization operation and a reversal operation on the first preprocessed image by using a preset large law threshold to generate a second preprocessed image;

The boundary of the second pre-processed image is smoothed through a morphological erosion operation, and the hole formed by the second pre-processed image during the smoothing process is filled in through a morphological expansion operation to obtain the pre-processing operation. Upsample the image.
An electronic device, wherein the electronic device includes:

At least one processor; and,

A memory communicatively connected with the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the cell image segmentation method as described below:

Perform a down-sampling operation on the original cell image to obtain a down-sampled image;

Input the down-sampled image into a pre-built convolutional segmentation network model for segmentation to obtain a first segmentation map;

According to a pre-built pixel coordinate conversion model and a bilinear interpolation algorithm, up-sampling the first segmentation image to the same resolution size as the original cell image to obtain a second segmentation image;

Combining the second segmentation map and the original cell image on the corresponding color channel by a preset geometric constraint and image feature matching method to obtain an up-sampled image;

The up-sampled image is preprocessed by a morphological algorithm and input into the convolutional segmentation network model for segmentation to obtain a segmented image.
8. The electronic device of claim 7, wherein the performing down-sampling operation on the original cell image to obtain the down-sampled image comprises:

Perform a down-sampling operation on the original cell image with a size of M×N according to the preset down-sampling ratio s, and the size is
The down-sampled image of;

Among them, s is the common divisor of M and N.
8. The electronic device according to claim 7, wherein the construction process of the pre-built convolutional segmentation network model comprises:

According to the cascade rules set in the preview, cascade the fully connected layers in the fully convolutional neural network;

The convolutional segmentation network model is obtained by cascading the multi-layer convolutional neural network and the fully-convolutional neural network to which the fully-connected layer has been added.
9. The electronic device according to claim 9, wherein said inputting said down-sampled image into a pre-built convolutional segmentation network model for segmentation to obtain a first segmentation map comprises:

Performing a convolution operation on the down-sampled image by using the convolutional segmentation network model to generate a down-sampled convolution feature map;

Performing a deconvolution operation on the down-sampled convolution feature map to obtain a deconvolution feature map;

Input the deconvolution feature map into the classification function of the convolution segmentation network model, and calculate the probability value of each segmentation area of the downsampled image;

The down-sampled image is segmented according to the probability value of each segmented region to generate the first segmentation map.
The electronic device according to claim 7, wherein the up-sampled image is stored in a blockchain, and the second segmentation map is compared with the original cell through a preset geometric constraint and image feature matching method. The images are merged on the corresponding color channels to obtain an up-sampled image, including:

According to a preset matching rule, select SIFT feature points from the second segmentation map, and sequentially match them with the SIFT feature points of the original cell image to obtain an original matching pair set;

Calculate the interior point rate of each matched pair in the original matching pair set, and remove the matching pairs whose interior point rate is less than the preset value α to obtain the primary matching pair set;

Calculating a basic matrix of the primary matching pair set according to the primary matching pair set, calculating the rank of the basic matrix, and removing the matching pairs whose rank is greater than a preset rank threshold to obtain a standard matching pair set;

According to a preset insertion rule, the standard matching pair set is inserted into the second segmentation image to obtain the up-sampled image.
11. The electronic device according to any one of claims 7 to 11, wherein the preprocessing of the up-sampled image with a morphological algorithm comprises:

Performing denoising filtering preprocessing on the upsampled image to obtain a first preprocessed image;

Performing a binarization operation and a reversal operation on the first preprocessed image by using a preset large law threshold to generate a second preprocessed image;

The boundary of the second pre-processed image is smoothed through a morphological erosion operation, and the hole formed by the second pre-processed image during the smoothing process is filled in through a morphological expansion operation to obtain the pre-processing operation. Upsample the image.
A computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the cell image segmentation method as described below:

Perform a down-sampling operation on the original cell image to obtain a down-sampled image;

Input the down-sampled image into a pre-built convolutional segmentation network model for segmentation to obtain a first segmentation map;

According to a pre-built pixel coordinate conversion model and a bilinear interpolation algorithm, up-sampling the first segmentation image to the same resolution size as the original cell image to obtain a second segmentation image;

Combining the second segmentation map and the original cell image on the corresponding color channel by a preset geometric constraint and image feature matching method to obtain an up-sampled image;

The up-sampled image is preprocessed by a morphological algorithm and input into the convolutional segmentation network model for segmentation to obtain a segmented image.
15. The computer-readable storage medium of claim 13, wherein the down-sampling operation on the original cell image to obtain the down-sampled image comprises:

Perform a down-sampling operation on the original cell image with a size of M×N according to the preset down-sampling ratio s, and the size is
The down-sampled image of;

Among them, s is the common divisor of M and N.
15. The computer-readable storage medium of claim 13, wherein the process of constructing the pre-built convolutional segmentation network model comprises:

According to the cascade rules set in the preview, cascade the fully connected layers in the fully convolutional neural network;

The convolutional segmentation network model is obtained by cascading the multi-layer convolutional neural network and the fully-convolutional neural network to which the fully-connected layer has been added.
15. The computer-readable storage medium of claim 15, wherein the inputting the downsampled image into a pre-built convolutional segmentation network model for segmentation to obtain the first segmentation map comprises:

Performing a convolution operation on the down-sampled image by using the convolutional segmentation network model to generate a down-sampled convolution feature map;

Performing a deconvolution operation on the down-sampled convolution feature map to obtain a deconvolution feature map;

Input the deconvolution feature map into the classification function of the convolution segmentation network model, and calculate the probability value of each segmentation area of the downsampled image;

The down-sampled image is segmented according to the probability value of each segmented region to generate the first segmentation map.
The computer-readable storage medium according to claim 13, wherein the up-sampled image is stored in a blockchain, and the second segmentation image is compared with the second segmentation image through a preset geometric constraint and image feature matching method. The original cell images are merged on the corresponding color channels to obtain an up-sampled image, including:

According to a preset matching rule, select SIFT feature points from the second segmentation map, and sequentially match them with the SIFT feature points of the original cell image to obtain an original matching pair set;

Calculate the interior point rate of each matching pair in the original matching pair set, and eliminate the matching pairs whose interior point rate is less than the preset value α to obtain the primary matching pair set;

Calculating a basic matrix of the primary matching pair set according to the primary matching pair set, calculating the rank of the basic matrix, and removing the matching pairs whose rank is greater than a preset rank threshold to obtain a standard matching pair set;

According to a preset insertion rule, the standard matching pair set is inserted into the second segmentation image to obtain the up-sampled image.
17. The computer-readable storage medium according to any one of claims 13 to 17, wherein the preprocessing of the up-sampled image with a morphological algorithm comprises:

Performing denoising filtering preprocessing on the upsampled image to obtain a first preprocessed image;

Performing a binarization operation and a reversal operation on the first preprocessed image by using a preset large law threshold to generate a second preprocessed image;

The boundary of the second pre-processed image is smoothed through a morphological erosion operation, and the hole formed by the second pre-processed image during the smoothing process is filled in through a morphological expansion operation to obtain the pre-processed image Upsample the image.
A cell image segmentation device, wherein the device includes:

The down-sampling module is used to down-sample the original cell image to obtain the down-sampled image;

A low-resolution segmentation module, configured to input the down-sampled image into a pre-built convolutional segmentation network model for segmentation to obtain a first segmentation map;

The up-sampling segmentation module is used to up-sample the first segmentation image to the same resolution size as the original cell image according to a pre-built pixel coordinate conversion model and a bilinear interpolation algorithm to obtain a second segmentation image;

The cell image segmentation module is used to combine the second segmentation map and the original cell image on the corresponding color channel through a preset geometric constraint and image feature matching method to obtain an up-sampled image, and the up-sampling The image is preprocessed by a morphological algorithm, and is input to the convolutional segmentation network model for segmentation to obtain a segmented image.
The cell image segmentation device of claim 19, wherein the low-resolution segmentation module comprises:

A convolution operation module, configured to perform a convolution operation on the down-sampled image through the convolution segmentation network model to generate a down-sampled convolution feature map;

A deconvolution operation module, configured to perform a deconvolution operation on the down-sampled convolution feature map to obtain a deconvolution feature map;

A probability calculation module, configured to input the deconvolution feature map into the classification function of the convolution segmentation network model, and calculate the probability value of each segmentation area of the downsampled image;

The image segmentation module is configured to segment the down-sampled image according to the probability value of each segmentation area to generate the first segmentation map.