CN113436160B - Pathological image processing and displaying system, client, server and medium - Google Patents

Pathological image processing and displaying system, client, server and medium Download PDF

Info

Publication number
CN113436160B
CN113436160B CN202110691405.4A CN202110691405A CN113436160B CN 113436160 B CN113436160 B CN 113436160B CN 202110691405 A CN202110691405 A CN 202110691405A CN 113436160 B CN113436160 B CN 113436160B
Authority
CN
China
Prior art keywords
image
pathology
pathological
identification
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110691405.4A
Other languages
Chinese (zh)
Other versions
CN113436160A (en
Inventor
王本刚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Xingmai Information Technology Co ltd
Original Assignee
Shanghai Xingmai Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Xingmai Information Technology Co ltd filed Critical Shanghai Xingmai Information Technology Co ltd
Priority to CN202110691405.4A priority Critical patent/CN113436160B/en
Publication of CN113436160A publication Critical patent/CN113436160A/en
Priority to PCT/CN2021/137331 priority patent/WO2022267379A1/en
Application granted granted Critical
Publication of CN113436160B publication Critical patent/CN113436160B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0012Biomedical image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10056Microscopic image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20021Dividing image into blocks, subimages or windows
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30096Tumor; Lesion

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Medical Informatics (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Quality & Reliability (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a pathological image processing and displaying system, a client, a server and a medium, wherein the system comprises: the pathology image acquisition module is used for acquiring pathology images; the image preprocessing module is used for preprocessing the pathological images to obtain a plurality of pathological image block sets; the pathology image identification module is used for identifying specific parts in the pathology image; the display instruction acquisition module is used for receiving a display instruction; the image acquisition module is used for acquiring a plurality of pathological image blocks with corresponding resolution according to the zoom degree and the display area, and splicing the pathological image blocks acquired according to the image acquisition module to form a pathological image corresponding to the display area; the pattern acquisition module is used for acquiring the identification pattern corresponding to the specific part; and the display module is used for displaying the pathological image corresponding to the display area and the identification graph corresponding to the specific part in a real-time superposition mode. The system can improve the loading speed of the image, thereby improving the user experience and reducing the waiting time of the user.

Description

Pathological image processing and displaying system, client, server and medium
Technical Field
The present invention relates to an image processing system, and more particularly, to a pathological image processing and displaying system, a client, a server, and a medium.
Background
In the prior art, the implementation of many treatment protocols has relied on the observation of pathology microscopy images by a pathologist via a display device. However, pathology microscopic images often contain tens of thousands or even hundreds of thousands of cells, which are limited by factors such as the video memory and the display screen size of the terminal device, and when a doctor needs to adjust the zoom degree and the display area of the image display, there is a problem that the image loading is slow.
Disclosure of Invention
In view of the above-mentioned drawbacks of the prior art, an object of the present invention is to provide a pathological image processing and displaying system, a client, a server and a medium, which are used for solving the problem of slow image loading when adjusting the zoom level and the display area of image display in the prior art.
To achieve the above and other related objects, a first aspect of the present invention provides a pathology image processing and display system, the system comprising: the pathological image acquisition module is used for acquiring pathological images, wherein the pathological images are pathological microscopic images of the stained tissue samples; the image preprocessing module is connected with the pathological image acquisition module and is used for preprocessing the pathological image to obtain a plurality of pathological image block sets, wherein each pathological image block set corresponds to different resolutions, and all pathological image blocks contained in each pathological image block set can be spliced into the pathological image with the corresponding resolution; the pathology image identification module is connected with the pathology image acquisition module and used for identifying specific parts in the pathology image, wherein the specific parts comprise parts where tumor cells in the pathology image are located, focus parts, parts which are colored and present specific colors and/or other interested parts; the display instruction acquisition module is used for receiving a display instruction, wherein the display instruction is used for designating a zoom degree and a display area; the image acquisition module is connected with the display instruction acquisition module and the image preprocessing module and is used for acquiring a plurality of pathological image blocks with corresponding resolution according to the zoom degree and the display area and splicing the plurality of pathological image blocks acquired according to the zoom degree and the display area to form a pathological image corresponding to the display area; the image acquisition module is connected with the display instruction acquisition module and the pathological image recognition module and is used for acquiring an identification image corresponding to the specific part; and the display module is connected with the image acquisition module and is used for responding to the display instruction to display the pathological image corresponding to the display area and the identification image corresponding to the specific part in a real-time superposition mode.
In an embodiment of the first aspect, the system further includes a graphics preprocessing module, and the graphics acquisition module is connected to the pathological image recognition module through the graphics preprocessing module; the pattern preprocessing module is used for preprocessing the recognition result of the specific part to obtain a plurality of mark pattern block sets, wherein mark pattern blocks in the mark pattern block sets correspond to pathological image blocks in the pathological image block sets; the graph acquisition module acquires a plurality of corresponding identification graph blocks according to the zoom degree and the display area, and splices the acquired identification graph blocks to form an identification graph corresponding to the specific part.
In an embodiment of the first aspect, when the display instruction obtaining module receives a new display instruction, the image obtaining module obtains at least one first pathology image block from a pathology image block subset according to a scaling degree and a display area specified by the new display instruction, obtains at least one second pathology image block from the pathology image block set, and splices the obtained first pathology image block and second pathology image block to form a pathology image corresponding to the display area specified by the new display instruction, wherein the pathology image block subset includes all pathology image blocks obtained by the image obtaining module according to the scaling degree and the display area specified by the previous display instruction; and/or when the display instruction acquisition module receives a new display instruction, the image acquisition module acquires at least one first identification image block from an identification image block subset according to the zoom degree and the display area designated by the new display instruction, acquires at least one second identification image block from the identification image block set, and splices the acquired first identification image block and second identification image block to form an identification image corresponding to the specific part, wherein the identification image block subset comprises all the identification image blocks acquired by the image acquisition module according to the zoom degree and the display area designated by the previous display instruction.
In an embodiment of the first aspect, the image preprocessing module includes a resolution adjustment unit and an image segmentation unit, wherein: the resolution adjustment unit is connected with the pathology image acquisition module and used for adjusting the resolution of the pathology microscopic image, and the image segmentation unit is connected with the resolution adjustment unit and used for segmenting the pathology microscopic image after resolution adjustment to obtain a plurality of pathology image block sets; and/or the image segmentation unit is connected with the pathology image acquisition module and used for segmenting the pathology image to obtain a plurality of image blocks, and the resolution adjustment unit is connected with the image segmentation unit and used for adjusting the resolutions of the plurality of image blocks to obtain a plurality of pathology image block sets.
In an embodiment of the first aspect, the pathology image recognition module uses a trained neural network model to recognize specific locations in the pathology image.
In an embodiment of the first aspect, the pathology image recognition module stores a result of the pathology image recognition on the specific part into a data file, and the graph acquisition module acquires an identification graph corresponding to the specific part according to the data file, wherein the data file includes a position of the specific part in the pathology image and/or a color of the specific part in the pathology image after being dyed; and/or the identification graph corresponding to the specific part comprises a first identification graph and/or a second identification graph, wherein the first identification graph corresponds to the part where the tumor cells in the pathological image are located, and the second identification graph corresponds to the part which presents the specific color after being dyed.
In an embodiment of the first aspect, when the display instruction obtaining module receives a new display instruction, the image obtaining module obtains at least one first pathology image block from a pathology image block subset according to a scaling degree and a display area specified by the new display instruction, obtains at least one second pathology image block from the pathology image block set, and splices the obtained first pathology image block and second pathology image block to form a pathology image corresponding to the display area specified by the new display instruction, wherein the pathology image block subset includes all pathology image blocks obtained by the image obtaining module according to the scaling degree and the display area specified by the previous display instruction; the display module is used for displaying pathological images corresponding to the display area designated by the new display instruction and identification graphics corresponding to the specific part in a superposition mode.
A second aspect of the present invention provides a client having a pathology image processing and displaying device, the pathology image processing and displaying device comprising: the pathological image acquisition module is used for acquiring pathological images, wherein the pathological images are pathological microscopic images of the stained tissue samples; the pathological image sending module is connected with the pathological image acquisition module and a service end and is used for sending the pathological image to the service end so that the service end can identify a specific part in the pathological image, wherein the specific part comprises a part where a tumor cell in the pathological image is located and/or a part which presents a specific color after being dyed; the image preprocessing module is connected with the pathological image acquisition module and is used for preprocessing the pathological image to obtain a plurality of pathological image block sets, wherein each pathological image block set corresponds to different resolutions, and all pathological image blocks contained in each pathological image block set can be spliced into the pathological image with the corresponding resolution; the display instruction acquisition module is used for receiving a display instruction, wherein the display instruction is used for designating a zoom degree and a display area; the image acquisition module is connected with the display instruction acquisition module and the image preprocessing module and is used for acquiring a plurality of pathological image blocks with corresponding resolution according to the zoom degree and the display area and splicing the plurality of pathological image blocks acquired according to the zoom degree and the display area to form a pathological image corresponding to the display area; the image acquisition module is connected with the display instruction acquisition module and the server and is used for acquiring an identification image corresponding to the specific part according to the display instruction and the identification result of the specific part by the server; and the display module is connected with the image acquisition module and is used for displaying the pathological image corresponding to the display area and the identification image corresponding to the specific part in a superposition way.
A third aspect of the present invention provides a server including a pathology image processing apparatus, the pathology image processing apparatus including: the pathological image receiving module is used for receiving pathological images sent by a client, wherein the pathological images are pathological microscopic images of stained tissue samples; the pathology image identification module is connected with the pathology image receiving module and is used for identifying specific parts in the pathology image, wherein the specific parts comprise parts where tumor cells in the pathology image are located and/or parts which are colored and present specific colors; and the identification result sending module is connected with the pathological image identification module and is used for sending the identification result of the pathological image identification module on the specific part to the client.
A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a first pathology image processing and displaying method, a second pathology image processing and displaying method, or a pathology image processing method, wherein: the first pathological image processing and displaying method comprises the following steps: acquiring a pathology image, wherein the pathology image is a pathology microscopic image of a stained tissue sample; preprocessing the pathology image to obtain a plurality of pathology image block sets, wherein each pathology image block set corresponds to different resolutions, and all pathology image blocks contained in each pathology image block set can be spliced into a pathology image with the corresponding resolution; identifying a specific part in the pathological image, wherein the specific part comprises a part where tumor cells in the pathological image are located and/or a part which presents a specific color after being dyed; receiving a display instruction, wherein the display instruction is used for designating a zoom degree and a display area; acquiring a plurality of pathological image blocks with corresponding resolution according to the zoom degree and the display area, and splicing the acquired pathological image blocks to form a pathological image corresponding to the display area; acquiring an identification graph corresponding to the specific part; superposing and displaying a pathological image corresponding to the display area and an identification graph corresponding to the specific part; the second pathological image processing and displaying method is applied to a client and comprises the following steps: acquiring a pathology image, wherein the pathology image is a pathology microscopic image of a stained tissue sample; the pathological image is sent to a service end, so that the service end identifies a specific part in the pathological image, wherein the specific part comprises a part where tumor cells in the pathological image are located and/or a part which presents a specific color after being dyed; preprocessing the pathology image to obtain a plurality of pathology image block sets, wherein each pathology image block set corresponds to different resolutions, and all pathology image blocks contained in each pathology image block set can be spliced into a pathology image with the corresponding resolution; receiving a display instruction, wherein the display instruction is used for designating a zoom degree and a display area; acquiring a plurality of pathological image blocks with corresponding resolution according to the zoom degree and the display area, and splicing the pathological image blocks acquired according to the pathological image blocks to form a pathological image corresponding to the display area; acquiring an identification graph corresponding to the specific part according to the display instruction and the identification result of the specific part by the server; superposing and displaying a pathological image corresponding to the display area and an identification graph corresponding to the specific part; the pathological image processing method is applied to a server and comprises the following steps: receiving a pathology image sent by a client, wherein the pathology image is a pathology microscopic image of a stained tissue sample; identifying a specific part in the pathological image, wherein the specific part comprises a part where tumor cells in the pathological image are located and/or a part which presents a specific color after being dyed; and sending the identification result of the pathological image identification module on the specific part to the client.
As described above, one technical solution of the pathological image processing and displaying system, the client, the server and the medium of the present invention has the following beneficial effects:
according to the pathological image processing and displaying system, the pathological images are preprocessed into different pathological image block sets, and only the scaling degree designated by the display instruction and the pathological image block corresponding to the display area are loaded in the displaying process, so that the whole pathological image is not required to be loaded, the loading speed of the image is improved, the user experience is improved, and the waiting time of the user is reduced.
Drawings
Fig. 1A is a schematic structural diagram of a pathological image processing and displaying system according to an embodiment of the invention.
FIG. 1B is a diagram showing an example of a specific portion of a pathological image processing and displaying system according to an embodiment of the present invention.
FIG. 1C is a diagram showing an example of the display contents of a display module of the pathological image processing and display system according to an embodiment of the invention.
FIG. 1D is a diagram showing another example of the display contents of the display module of the pathological image processing and display system according to the present invention in one embodiment.
Fig. 1E is a schematic diagram of a pathological image processing and display system according to another embodiment of the invention.
FIG. 2 is a flow chart of training a neural network model in an embodiment of the pathological image processing and display system according to the present invention.
Fig. 3A is a schematic structural diagram of a representation level acquisition module of the pathological image processing and display system according to an embodiment of the invention.
Fig. 3B is a schematic diagram showing another configuration of the expression level acquisition module in an embodiment of the pathological image processing and display system according to the present invention.
Fig. 3C is a schematic diagram showing another configuration of the expression level acquisition module in an embodiment of the pathological image processing and display system according to the present invention.
Fig. 4 is a schematic structural diagram of a pathological image processing and displaying device of the client according to an embodiment of the invention.
Fig. 5 is a schematic structural diagram of a pathological image processing device according to an embodiment of the present invention.
Fig. 6 is a flowchart of a first pathological image processing and displaying method of the computer readable storage medium according to an embodiment of the invention.
Fig. 7 is a flowchart of a second pathological image processing and displaying method of the computer readable storage medium according to an embodiment of the invention.
Fig. 8 is a flowchart of a pathological image processing method of the computer readable storage medium according to an embodiment of the invention.
Description of element reference numerals
1. Image processing and display system
11. Pathological image acquisition module
12. Image preprocessing module
13. Pathological image recognition module
14. Display instruction acquisition module
15. Image acquisition module
16. Graph acquisition module
17. Display module
18. Expression level acquisition module
181a first cell number acquisition unit
182a second cell number acquisition unit
183a expression level acquisition unit
181b first cell number acquisition unit
182b second cell number acquisition unit
183b third cell number acquisition Unit
184b expression level acquisition unit
181c third cell number acquisition unit
182c fourth cell number acquisition unit
183c expression level acquisition unit
19. Graphics preprocessing module
4. Pathological image processing and displaying device
41. Pathological image acquisition module
42. Pathological pattern transmitting module
43. Image preprocessing module
44. Display instruction acquisition module
45. Image acquisition module
46. Graph acquisition module
47. Display module
5. Pathological image processing device
51. Pathological image receiving module
52. Pathological image recognition module
53. Recognition result transmitting module
S21 to S24 steps
S61 to S67 steps
S71 to S77 steps
S81 to S83 steps
Detailed Description
Other advantages and effects of the present invention will become apparent to those skilled in the art from the following disclosure, which describes the embodiments of the present invention with reference to specific examples. The invention may be practiced or carried out in other embodiments that depart from the specific details, and the details of the present description may be modified or varied from the spirit and scope of the present invention. It should be noted that the following embodiments and features in the embodiments may be combined with each other without conflict.
It should be noted that the illustrations provided in the following embodiments merely illustrate the basic concept of the present invention by way of illustration, and only the components related to the present invention are shown in the illustrations, not according to the number, shape and size of the components in actual implementation, and the form, number and proportion of each component in actual implementation may be arbitrarily changed, and the layout of the components may be more complex. Moreover, relational terms such as "first," "second," and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.
Pathology microscopic images often contain tens of thousands or even hundreds of thousands of cells, but in practical application, the pathology microscopic images are limited by factors such as the video memory and the display screen size of terminal equipment, and when a doctor needs to adjust the zoom degree and the display area of image display, the problems of slower image loading, clamping and the like often exist. Therefore, the invention provides a pathological image processing and displaying system, which is beneficial to improving the loading speed of images by preprocessing the pathological images into different pathological image block sets and only loading the pathological image blocks with the scaling degree designated by a display instruction and corresponding display areas in the display process without loading the whole pathological image, thereby improving the user experience and reducing the waiting time of users.
Referring to fig. 1A, in an embodiment of the invention, the pathology image processing and displaying system 1 includes a pathology image acquisition module 11, an image preprocessing module 12, a pathology image identification module 13, a display instruction acquisition module 14, an image acquisition module 15, a graphics acquisition module 16 and a display module 17.
The pathology image acquisition module 11 is configured to acquire pathology images, wherein the pathology images refer to pathology microscopic images of stained tissue samples, in particular, pathology images refer to pathology microscopic images of tissue samples stained with immunohistochemical detection reagents, including cytopathology images, histopathology images, immunohistochemical images, and the like, which can be obtained, for example, by staining tissue samples using PD-L1 IHC22C3 pharmDx (PD-L1 detection kit (immunohistochemistry)).
The image preprocessing module 12 is connected to the pathology image acquisition module 11, and is configured to perform preprocessing on the pathology image to obtain a plurality of pathology image block sets. Each pathological image block set corresponds to different resolutions, namely, each pathological image block set corresponds to one resolution, and the resolutions corresponding to the pathological image block sets are different. Each pathology image block set comprises at least one pathology image block, and the resolution ratio of all pathology image blocks contained in any pathology image block set is the same. The resolution corresponding to any of the pathological image block sets refers to the resolution of any pathological image block in the pathological image block set, for example, for the pathological image block set a, the pathological image block set a includes a plurality of pathological image blocks with the same resolution res_a, and the resolution corresponding to the pathological image block set a is res_a.
In addition, all the pathological image blocks contained in each pathological image block set can be spliced into a complete pathological image with corresponding resolution. For example, if the resolution corresponding to the pathological image block set a is res_a and the resolution corresponding to the pathological image block set B is res_b, all the pathological image blocks in the pathological image block set a can be stitched into the pathological image with the resolution res_a, and all the pathological image blocks in the pathological image block set B can be stitched into the pathological image with the resolution res_b.
The pathology image recognition module 13 is connected to the pathology image acquisition module 11, and is configured to recognize a specific location in the pathology image, where the specific location includes a location where a tumor cell in the pathology image is located and/or a location that presents a specific color after being stained.
Specifically, after staining, all tumor cells are stained, and specific ligands, such as PD-L1, positive cells appear a specific color different from that of other cells, so that the location of the tumor cells in the pathological image can be obtained according to the staining result, and then the tumor region in the pathological image can be obtained, and the location which presents the specific color after staining can be obtained.
The display instruction obtaining module 14 is configured to receive a display instruction, where the display instruction may be input by a user through an input device such as a mouse or a keyboard, or may be automatically generated by an electronic device according to a preset computer program. The display instruction is used for designating the zoom degree and the display area when the pathological image is displayed, wherein different zoom degrees correspond to different resolutions. For example, if the scaling degree is 100% corresponding to the original resolution res_0 of the pathological image, the scaling degree is 50% corresponding to the resolution of 0.5×res_0.
The image obtaining module 15 is connected to the display instruction obtaining module 14 and the image preprocessing module 12, and is configured to obtain a pathological image block set C with a corresponding resolution according to the zoom level, and obtain pathological image blocks corresponding to the display area and in m rows and n columns from the pathological image block set C, where the value of m is the height of the display area divided by the height of the pathological image blocks in the pathological image block set C, and is added by 1 after rounding, and the value of n is the width of the display area divided by the width of the pathological image blocks in the pathological image block set C, and is added by 1 after rounding. After m rows and n columns of pathological image blocks are acquired, the image acquisition module 15 is further configured to stitch and form a pathological image corresponding to the display area according to the acquired m rows and n columns of pathological image blocks. For example, when the zoom level is 50%, the display area is the upper left area, and m=4, n=3, the image acquisition module 15 acquires a pathology image block set D corresponding to a resolution of 0.5×res_0, acquires 12 pathology image blocks in total of 4 rows and 3 columns corresponding to the upper left area from the pathology image block set D, and stitches the 12 pathology image blocks into a pathology image corresponding to the upper left area.
The pattern acquisition module 16 is connected to the display instruction acquisition module 14 and the pathological image recognition module 13, and is configured to acquire an identification pattern corresponding to the specific portion. The identification pattern is used for identifying the specific part, and the identification pattern can be represented by an image shown in fig. 1B, or can be represented by other forms such as text.
The display module 17 is connected to the image acquisition module 15 and the image acquisition module 16, and is configured to superimpose and display, in real time, a pathology image corresponding to the display area and an identification image corresponding to the specific portion in response to the display instruction. For example, referring to fig. 1C, an exemplary display result diagram of the display module 17 in response to a display instruction is shown, wherein curves with different colors are used to identify the outline of a specific portion. For another example, referring to fig. 1D, an exemplary diagram of a display result of the display module 17 in response to another display instruction is shown, in which a location of a specific portion is identified by a dot of a different color.
As can be seen from the above description, in the pathological image processing and displaying system 1 according to the present embodiment, the pathological image is preprocessed into different pathological image block sets, and only the scaling degree designated by the display instruction and the pathological image block corresponding to the display area are loaded in the displaying process, so that the whole pathological image is not required to be loaded, which is beneficial to improving the loading speed of the image, thereby improving the user experience and reducing the waiting time of the user.
In addition, in the pathological image processing and displaying system 1 according to the present embodiment, the display module 17 can simultaneously display the pathological image corresponding to the display area and the identification pattern corresponding to the specific portion in a superimposed display manner, so that a doctor can intuitively and intuitively understand the condition of the specific portion of the pathological image in the current display area.
Referring to fig. 1E, in an embodiment of the invention, the pathological image processing and displaying system 1 further includes a graphics preprocessing module 19, and the graphics acquisition module 16 is connected to the pathological image recognition module 13 through the graphics preprocessing module 19.
Specifically, the graphics preprocessing module 19 is configured to preprocess the recognition result of the specific location to obtain a plurality of identification graphics block sets, where identification graphics blocks in the identification graphics block sets correspond to pathology image blocks in the pathology image block set, for example, the identification graphics blocks in the identification graphics block sets may correspond to pathology image blocks in the pathology image block set one by one. The graphic acquisition module 16 acquires a plurality of corresponding identification graphic blocks according to the zoom level and the display area, and splices the acquired identification graphic blocks to form an identification graphic corresponding to the specific part. For example, the image acquisition module 16 may acquire a plurality of pathological image blocks with corresponding resolutions according to the zoom level and the display area, or may directly acquire the plurality of pathological image blocks with corresponding resolutions from the image acquisition module 15, and acquire the corresponding plurality of identification image blocks according to the correspondence between the pathological image blocks and the identification image blocks, so as to splice the plurality of identification image blocks acquired according to the corresponding plurality of identification image blocks to form the identification image corresponding to the specific portion.
As can be seen from the above description, the pathological image processing and displaying system according to the present embodiment can not only increase the image loading speed, but also increase the loading speed of the logo graphics, which is beneficial to further improving the user experience.
In an embodiment of the present invention, when the display instruction obtaining module 14 receives a new display instruction, the image obtaining module 15 obtains at least one first pathology image block from a pathology image block subset according to a scaling degree and a display area specified by the new display instruction, and obtains at least one second pathology image block from the pathology image block set, and splices the obtained first pathology image block and second pathology image block to form a pathology image corresponding to the display area specified by the new display instruction, wherein the pathology image block subset includes all pathology image blocks obtained by the image obtaining module 15 according to the scaling degree and the display area specified by the previous display instruction. At this time, the display module 17 is configured to superimpose and display a pathology image corresponding to the display area specified by the new display instruction and an identification pattern corresponding to the specific portion.
For example, for any display instruction 1, if the image acquisition module 15 acquires a set of all the pathological image blocks according to the zoom level and the display area specified by the display instruction 1 as a pathological image block subset E, when a user inputs a new display instruction 2, the image acquisition module 15 acquires one or more first pathological image blocks corresponding to the zoom level and the display area specified by the display instruction 2 from the pathological image block subset E, acquires one or more second pathological image blocks corresponding to the zoom level and the display area specified by the display instruction 2 from the pathological image block set, and splices the acquired first pathological image blocks and second pathological image blocks into a pathological image corresponding to the display area specified by the display instruction 2. Thereafter, the display module 17 superimposes and displays the pathology image corresponding to the display area specified by the display instruction 2 and the identification pattern corresponding to the specific part.
Optionally, when the display instruction obtaining module 14 receives a new display instruction, the graphics obtaining module 16 obtains at least one first identification graphics block from a subset of identification graphics blocks according to a zoom level and a display area specified by the new display instruction, and obtains at least one second identification graphics block from the set of identification graphics blocks, and splices the obtained first identification graphics block and second identification graphics block to form an identification graphic corresponding to the specific part, where the subset of identification graphics blocks includes all the identification graphics blocks obtained by the graphics obtaining module 16 according to the zoom level and the display area specified by the previous display instruction. This procedure is similar to the procedure of the above-described image acquisition module 15 for acquiring a pathology image corresponding to the display area designated by the new display instruction, and will not be described in detail here.
As can be seen from the above description, in this embodiment, the image obtaining module 15 only needs to obtain the second pathological image block from the pathological image block set, so that data transmitted between the image preprocessing module 12 and the image obtaining module 15 can be reduced, which is beneficial to further improving the loading speed of the image, and particularly, the advantages of this embodiment are more obvious when the image preprocessing module 12 and the image obtaining module 15 are disposed in different devices. In addition, the graphics acquisition module 16 may acquire only the second identification graphics block from the graphics preprocessing module 19, so that data transmitted between the graphics preprocessing module 19 and the graphics acquisition module 16 can be reduced, which is beneficial to further improving the loading speed of the identification graphics.
In an embodiment of the invention, the image preprocessing module includes a resolution adjustment unit and an image segmentation unit.
Optionally, the resolution adjustment unit is connected to the pathology image acquisition module, and is used for adjusting the resolution of the pathology microscopic image, and the image segmentation unit is connected to the resolution adjustment unit, and is used for segmenting the pathology microscopic image after resolution adjustment, so as to obtain a plurality of pathology image block sets. Specifically, the resolution adjustment unit may adjust the resolution of the pathology microscope image a plurality of times. And, the resolution adjustment unit adjusts the resolution of the pathology microscopic image once, and the image segmentation unit may segment the pathology microscopic image once to obtain a pathology image block set.
Optionally, the image segmentation unit is connected to the pathology image acquisition module, and is configured to segment the pathology image to obtain a plurality of image blocks, and the resolution adjustment unit is connected to the image segmentation unit, and is configured to adjust resolutions of the plurality of image blocks to obtain a plurality of pathology image block sets. Specifically, the resolution adjustment unit may adjust the resolutions of the plurality of image blocks a plurality of times, and each time the resolution adjustment unit adjusts the resolutions of the plurality of image blocks, one pathological image block set may be obtained.
In an embodiment of the invention, the pathology image recognition module uses a trained neural network model to recognize specific locations in the pathology image. Specifically, referring to fig. 2, the training method of the neural network model in this embodiment includes:
s21, constructing a neural network model, for example, the neural network model can be constructed by adopting a Unet deep learning image segmentation model as a basis.
S22, acquiring training data, wherein the training data is a training pathology image marked with a specific ligand, such as PD-L1, cells, and the training pathology image is a pathology microscopic image of a stained tissue sample.
And S23, training the neural network model by using the training data, wherein the step S23 can be realized by adopting the existing neural network training method, and the detailed implementation method is not repeated here.
S24, test data are obtained, and the neural network model is tested by using the test data. Wherein the test data is a test pathology image labeled with the specific ligand, e.g., PD-L1, cells, and the test pathology image is a pathology microscopy image of a stained tissue sample.
In this embodiment, after training the neural network in the steps S21 to S24, the neural network model may be used to process the pathology image to obtain the specific portion in the pathology image.
In an embodiment of the present invention, the pathology image recognition module stores a recognition result of the specific location in a data file. The pattern acquisition module acquires an identification pattern corresponding to the specific part according to the data file, wherein the data file comprises the position of the tumor cells in the pathological image and/or the color of the tumor cells in the pathological image after being dyed. For example, the data file may record the location of the tumor cells using the coordinates of the center point of the tumor cells, and record whether the tumor cells appear the specific color after being stained using a numerical code of 0 or 1.
In this embodiment, the identification result of the specific portion by the pathological image identification module is stored in the data file, so that only the data file needs to be transmitted between the pathological image identification module and the graph acquisition module, and therefore, a pathological image containing tens of thousands or even hundreds of thousands of cells does not need to be transmitted between the pathological image identification module and the graph acquisition module, which is beneficial to reducing the data volume transmitted between the pathological image identification module and the graph acquisition module.
In an embodiment of the present invention, the identification pattern corresponding to the specific portion includes a first identification pattern and/or a second identification pattern, where the first identification pattern corresponds to a portion where the tumor cells in the pathological image are located, for example, a point or a curve having a first color, and the second identification pattern corresponds to a portion where the specific color appears after being stained, for example, a point or a curve having a second color.
In an embodiment of the invention, the identification pattern corresponding to the specific location comprises points for identifying the location of the cell and/or curves for identifying the contour of the cell.
In an embodiment of the invention, the pathological image processing and displaying system further includes an expression level obtaining module, where the expression level obtaining module is connected to the pathological image identifying module, and is configured to obtain the PD-L1 expression level of the pathological image according to the identification result of the specific portion by the pathological image identifying module.
Alternatively, referring to fig. 3A, the expression level acquisition module 18 includes a first cell number acquisition unit 181a, a second cell number acquisition unit 182a, and an expression level acquisition unit 183A.
The first cell number obtaining unit 181a is connected to the pathological image recognition module, and is configured to obtain a first cell number, where the first cell number refers to a specific ligand, such as PD-L1, in the pathological image, and the number of positive tumor cells may be obtained according to a recognition result of the pathological image recognition module on the specific location.
The second cell number obtaining unit 182a is connected to the pathological image recognition module, and is configured to obtain a second cell number, where the second cell number is the total number of tumor cells in the pathological image, and may be obtained according to the recognition result of the pathological image recognition module on the specific part.
The expression level obtaining unit 183a is connected to the first cell number obtaining unit 181a and the second cell number obtaining unit 182a, and is configured to obtain a PD-L1 expression score and/or a PD-L1 expression level of the pathological image according to the first cell number and the second cell number. The PD-L1 expression score and/or expression level of the pathological image is the expression level of the pathological image in this embodiment.
Alternatively, the PD-L1 expression fraction of the pathology image is, for example, a value obtained by dividing the first cell number by the second cell number and multiplying the divided value by 100%. If the PD-L1 expression score of the pathological image is larger than a first preset threshold, the PD-L1 expression grade of the pathological image is low, otherwise, the PD-L1 expression grade of the pathological image is high.
Alternatively, referring to fig. 3B, the expression level obtaining module 18 includes a first cell number obtaining unit 181B, a second cell number obtaining unit 182B, a third cell number obtaining unit 183B, and an expression level obtaining unit 184B.
The first cell number obtaining unit 181b is connected to the pathological image recognition module, and is configured to obtain a first cell number, where the first cell number is the number of tumor cells positive for PD-L1 in the pathological image, and may be obtained according to a recognition result of the pathological image recognition module on the specific part.
The second cell number obtaining unit 182b is connected to the pathological image recognition module, and is configured to obtain a second cell number, where the second cell number is the total number of tumor cells in the pathological image, and may be obtained according to the recognition result of the pathological image recognition module on the specific part.
The third cell number obtaining unit 183b is connected to the pathological image recognition module, and is configured to obtain a third cell number, where the third cell number is the number of tumor immune cells positive for PD-L1 in the pathological image, and the third cell number can be obtained according to the recognition result of the pathological image recognition module on the specific part, and the tumor immune cells are immune cells related to tumor such as lymphocytes and macrophages.
The expression level obtaining unit 184b is connected to the first cell number obtaining unit 181b, the second cell number obtaining unit 182b, and the third cell number obtaining unit 183b, and is configured to obtain a PD-L1 expression score and/or a PD-L1 expression level of the pathological image according to the first cell number, the second cell number, and the third cell number. The PD-L1 expression score and/or expression level of the pathological image is the expression level of the pathological image in this embodiment.
Alternatively, the PD-L1 expression fraction of the pathology image is, for example, a value obtained by dividing the sum of the first cell number and the third cell number by the second cell number and multiplying the product by 100. If the PD-L1 expression score of the pathological image is greater than a second preset threshold, for example, 10, the PD-L1 expression grade of the pathological image is low, otherwise, the PD-L1 expression grade of the pathological image is high.
Alternatively, referring to fig. 3C, the expression level acquisition module 18 includes a third cell number acquisition unit 181C, a fourth cell number acquisition unit 182C, and an expression level acquisition unit 183C.
The third cell number obtaining unit 181c is connected to the pathological image recognition module, and is configured to obtain a third cell number, where the third cell number refers to the number of tumor immune cells positive for PD-L1 in the pathological image, and may be obtained according to the recognition result of the pathological image recognition module on the specific part, and the tumor immune cells refer to immune cells such as lymphocytes and macrophages related to tumor.
The fourth cell number obtaining unit 182c is connected to the pathology image obtaining module, and is configured to obtain a fourth cell number, where the fourth cell number refers to a total number of tumor immune cells in the pathology image.
The expression level obtaining unit 183c is connected to the third cell number obtaining unit 181c and the fourth cell number obtaining unit 182c, and is configured to obtain a PD-L1 expression score and/or a PD-L1 expression level of the pathological image according to the third cell number and the fourth cell number. The PD-L1 expression score and/or expression level of the pathological image is the expression level of the pathological image in this embodiment.
Alternatively, the PD-L1 expression fraction of the pathology image is, for example, a value obtained by dividing the third cell number by the fourth cell number and multiplying the divided value by 100%. If the PD-L1 expression score of the pathological image is larger than a third preset threshold, the PD-L1 expression grade of the pathological image is low, otherwise, the PD-L1 expression grade of the pathological image is high.
In an embodiment of the invention, the display module is further connected to the expression level acquisition module, and the display module is further configured to display the PD-L1 expression level of the pathological image.
In an embodiment of the present invention, the pathological image processing and displaying system further includes a treatment plan generating module, where the treatment plan generating module is connected to the expression level obtaining module, and is configured to generate a treatment plan according to the PD-L1 expression level of the pathological image, and a doctor may refer to the treatment plan to treat the patient. For example, if the PD-L1 expression level of the pathological image is high, then the treatment regimen is to recommend treatment of the patient with a PD-L1 inhibitor, otherwise treatment of the patient with a PD-L1 inhibitor is not recommended.
The invention also provides a client which is provided with a pathological image processing and displaying device. Specifically, referring to fig. 4, in an embodiment of the present invention, the pathological image processing and displaying device 4 includes a pathological image obtaining module 41, a pathological image sending module 42, an image preprocessing module 43, a display instruction obtaining module 44, an image obtaining module 45, a graphics obtaining module 46 and a display module 47.
The pathology image acquisition module 41 is configured to acquire a pathology image, which is a pathology microscopic image of a stained tissue sample.
The pathology image sending module 42 is connected to the pathology image obtaining module 41 and a service end, and is configured to send the pathology image to the service end, so that the service end identifies a specific location in the pathology image, where the specific location includes a location where a tumor cell in the pathology image is located and/or a location that presents a specific color after being stained.
The image preprocessing module 43 is connected to the pathological image obtaining module 41, and is configured to preprocess the pathological image to obtain a plurality of pathological image block sets, where each pathological image block set corresponds to a different resolution, and all pathological image blocks included in each pathological image block set can be spliced into a complete pathological image with a corresponding resolution.
The display instruction acquisition module 44 is configured to receive a display instruction, where the display instruction is used to specify a zoom level and a display area.
The image obtaining module 45 is connected to the display instruction obtaining module 44 and the image preprocessing module 43, and is configured to obtain a plurality of pathological image blocks with corresponding resolutions according to the zoom level and the display area, and splice the plurality of pathological image blocks obtained according to the zoom level and the display area to form a pathological image corresponding to the display area.
The graphic obtaining module 46 is connected to the display instruction obtaining module 44 and the server, and is configured to obtain an identification graphic corresponding to the specific location according to the display instruction and the identification result of the specific location by the server.
The display module is connected to the image acquisition module 45 and the image acquisition module 46, and is configured to superimpose and display a pathological image corresponding to the display area and an identification image corresponding to the specific portion.
The above-mentioned pathological image obtaining module 41, image preprocessing module 43, display instruction obtaining module 44, image obtaining module 45, image obtaining module 46 and display module 47 are the same as the corresponding modules in the pathological image processing and displaying system 1 shown in fig. 1A, and are not repeated here for saving the description space.
Preferably, the server stores the identification result of the specific location in a data file, and the graph obtaining module 46 obtains an identification graph corresponding to the specific location according to the data file, where the data file includes the position of the tumor cells in the pathological image and/or the color of the tumor cells in the pathological image after being stained. At this time, the server does not need to send the pathological image back to the client, which is beneficial to reducing the data volume transmitted between the two, thereby improving the graphic loading speed of the client.
In practical applications, the data processing capability of the client is relatively weak, so if the client is used to identify the specific part in the pathological image, it may take a lot of time, and for this problem, the client 4 in this embodiment sends the pathological image to the server, and the server with higher data processing capability identifies the specific part in the pathological image, which is beneficial to reducing the waiting time of the user and improving the user experience.
The invention also provides a server, which comprises a pathological image processing device. Specifically, referring to fig. 5, in an embodiment of the present invention, the pathology image processing apparatus 5 includes a pathology image receiving module 51, a pathology image identification module 52, and an identification result transmitting module 53.
The pathology image receiving module 51 is configured to receive a pathology image sent by a client, where the pathology image is a pathology microscopic image of a stained tissue sample.
The pathology image recognition module 52 is connected to the pathology image receiving module 51, and is configured to recognize a specific location in the pathology image, where the specific location includes a location where a tumor cell in the pathology image is located and/or a location that presents a specific color after being stained.
The recognition result sending module 53 is connected to the pathology image recognition module 52, and is configured to send a recognition result of the pathology image recognition module on the specific location to the client.
In this embodiment, the pathological image recognition module 52 is the same as the pathological image recognition module 13 in the pathological image processing and displaying system 1 shown in fig. 1A, and is not described in detail here for saving the description space.
The present invention also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a first pathology image processing and displaying method. Specifically, referring to fig. 6, in an embodiment of the present invention, the method includes:
s61, acquiring a pathological image, wherein the pathological image is a pathological microscopic image of a stained tissue sample.
S62, preprocessing the pathology image to obtain a plurality of pathology image block sets, wherein each pathology image block set corresponds to different resolutions, and all pathology image blocks contained in each pathology image block set can be spliced into a complete pathology image with the corresponding resolution.
S63, identifying a specific part in the pathological image, wherein the specific part comprises a part where tumor cells in the pathological image are located and/or a part which presents a specific color after being dyed.
S64, receiving a display instruction, wherein the display instruction is used for designating the zoom degree and the display area.
S65, acquiring a plurality of pathological image blocks with corresponding resolutions according to the zoom degree and the display area, and splicing the pathological image blocks to form a pathological image corresponding to the display area.
S66, acquiring the identification graph corresponding to the specific part.
S67, overlaying and displaying the pathological image corresponding to the display area and the identification graph corresponding to the specific part.
The steps S61 to S67 correspond to the functions of the corresponding modules in the pathological image processing and displaying system 1 shown in fig. 1A, and are not repeated here for saving the description space.
In addition, it should be noted that, the above reference numerals S61 to S67 are only used to designate different steps, and are not used to limit the execution sequence between different steps, and the sequence of each step may be determined according to actual requirements in specific applications. For example, step S66 may be performed before step S65 is performed, or steps S66 and S65 may be performed simultaneously.
The present invention also provides another computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a second pathology image processing and displaying method, said method being applied to a client. Specifically, referring to fig. 7, in an embodiment of the present invention, the method includes:
S71, acquiring a pathological image, wherein the pathological image is a pathological microscopic image of the stained tissue sample.
And S72, the pathological image is sent to a service end, so that the service end identifies a specific part in the pathological image, wherein the specific part comprises a part where tumor cells in the pathological image are located and/or a part which presents a specific color after being dyed.
S73, preprocessing the pathology image to obtain a plurality of pathology image block sets, wherein each pathology image block set corresponds to different resolutions, and all pathology image blocks contained in each pathology image block set can be spliced into a complete pathology image with the corresponding resolution.
S74, receiving a display instruction, wherein the display instruction is used for designating the zoom degree and the display area.
And S75, acquiring a plurality of pathological image blocks with corresponding resolutions according to the zoom degree and the display area, and splicing the pathological image blocks acquired according to the zoom degree and the display area to form a pathological image corresponding to the display area.
S76, acquiring an identification graph corresponding to the specific part according to the display instruction and the identification result of the server side on the specific part.
And S77, overlaying and displaying the pathological image corresponding to the display area and the identification graph corresponding to the specific part.
The above steps S71 to S77 correspond to the functions of the corresponding modules in the pathological image processing and displaying device 4 shown in fig. 4 one by one, and are not repeated here for saving the description space.
The present invention also provides a further computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a pathology image processing method, the method being applied to a server. Specifically, referring to fig. 8, in an embodiment of the present invention, the method includes:
s81, receiving a pathology image sent by a client, wherein the pathology image is a pathology microscopic image of a stained tissue sample.
S82, identifying specific parts in the pathological image, wherein the specific parts comprise parts where tumor cells in the pathological image are located and/or parts which are stained and present specific colors.
S83, sending the identification result of the pathological image identification module on the specific part to the client.
The steps S81 to S83 correspond to the functions of the corresponding modules in the pathological image processing device 5 shown in fig. 5 one by one, and are not repeated here for saving the description space.
According to the pathological image processing and displaying system, the pathological images are preprocessed into different pathological image block sets, and only the scaling degree designated by the display instruction and the pathological image block corresponding to the display area are loaded in the displaying process, so that the whole pathological image is not required to be loaded, the loading speed of the image is improved, the user experience is improved, and the waiting time of the user is reduced.
In summary, the present invention effectively overcomes the disadvantages of the prior art and has high industrial utility value.
The above embodiments are merely illustrative of the principles of the present invention and its effectiveness, and are not intended to limit the invention. Modifications and variations may be made to the above-described embodiments by those skilled in the art without departing from the spirit and scope of the invention. Accordingly, it is intended that all equivalent modifications and variations of the invention be covered by the claims, which are within the ordinary skill of the art, be within the spirit and scope of the present disclosure.

Claims (9)

1. A pathology image processing and display system, the system comprising:
the pathological image acquisition module is used for acquiring pathological images, wherein the pathological images are pathological microscopic images of the stained tissue samples;
The image preprocessing module is connected with the pathological image acquisition module and is used for preprocessing the pathological image to obtain a plurality of pathological image block sets, wherein each pathological image block set corresponds to different resolutions, and all pathological image blocks contained in each pathological image block set can be spliced into the pathological image with the corresponding resolution;
the pathology image identification module is connected with the pathology image acquisition module and is used for identifying a specific part in the pathology image;
the display instruction acquisition module is used for receiving a display instruction, wherein the display instruction is used for designating a zoom degree and a display area;
the image acquisition module is connected with the display instruction acquisition module and the image preprocessing module and is used for acquiring a plurality of pathological image blocks with corresponding resolution according to the zoom degree and the display area and splicing the plurality of pathological image blocks acquired according to the zoom degree and the display area to form a pathological image corresponding to the display area;
the image acquisition module is connected with the display instruction acquisition module and the pathological image recognition module and is used for acquiring an identification image corresponding to the specific part, wherein the identification image comprises outlines of the specific part marked by curves with different colors;
The image acquisition module is connected with the pathological image recognition module through the image preprocessing module; the pattern preprocessing module is used for preprocessing the recognition result of the specific part to obtain a plurality of mark pattern block sets, wherein mark pattern blocks in the mark pattern block sets correspond to pathological image blocks in the pathological image block sets; the graph acquisition module acquires a plurality of corresponding identification graph blocks according to the zoom degree and the display area, and splices the acquired identification graph blocks to form an identification graph corresponding to the specific part;
and the display module is connected with the image acquisition module and is used for responding to the display instruction to display the pathological image corresponding to the display area and the identification image corresponding to the specific part in a real-time superposition mode.
2. The system according to claim 1, wherein:
when the display instruction acquisition module receives a new display instruction, the image acquisition module acquires at least one first pathology image block from a pathology image block subset according to the scaling degree and the display area appointed by the new display instruction, acquires at least one second pathology image block from the pathology image block set, and splices the acquired first pathology image block and second pathology image block to form a pathology image corresponding to the display area appointed by the new display instruction, wherein the pathology image block subset comprises all pathology image blocks acquired by the image acquisition module according to the scaling degree and the display area appointed by the previous display instruction; and/or
When the display instruction acquisition module receives a new display instruction, the image acquisition module acquires at least one first identification image block from an identification image block subset according to the zoom degree and the display area appointed by the new display instruction, acquires at least one second identification image block from the identification image block set, and splices the acquired first identification image block and second identification image block to form an identification image corresponding to the specific part, wherein the identification image block subset comprises all the identification image blocks acquired by the image acquisition module according to the zoom degree and the display area appointed by the previous display instruction.
3. The system of claim 1, wherein the image preprocessing module comprises a resolution adjustment unit and an image segmentation unit, wherein:
the resolution adjustment unit is connected with the pathology image acquisition module and used for adjusting the resolution of the pathology microscopic image, and the image segmentation unit is connected with the resolution adjustment unit and used for segmenting the pathology microscopic image after resolution adjustment to obtain a plurality of pathology image block sets; and/or
The image segmentation unit is connected with the pathological image acquisition module and is used for segmenting the pathological image to obtain a plurality of image blocks, and the resolution adjustment unit is connected with the image segmentation unit and is used for adjusting the resolutions of the plurality of image blocks to obtain a plurality of pathological image block sets.
4. The system according to claim 1, wherein: the pathology image recognition module utilizes a trained neural network model to recognize specific parts in the pathology image.
5. The system according to claim 1, wherein: the pathology image recognition module stores the recognition result of the specific part into a data file, and the pattern acquisition module acquires an identification pattern corresponding to the specific part according to the data file, wherein the data file comprises the position of the specific part in the pathology image and/or the color of the specific part in the pathology image after being dyed.
6. The system according to claim 1, wherein: when the display instruction acquisition module receives a new display instruction, the image acquisition module acquires at least one first pathology image block from a pathology image block subset according to the scaling degree and the display area appointed by the new display instruction, acquires at least one second pathology image block from the pathology image block set, and splices the acquired first pathology image block and second pathology image block to form a pathology image corresponding to the display area appointed by the new display instruction, wherein the pathology image block subset comprises all pathology image blocks acquired by the image acquisition module according to the scaling degree and the display area appointed by the previous display instruction; the display module is used for displaying pathological images corresponding to the display area designated by the new display instruction and identification graphics corresponding to the specific part in a superposition mode.
7. A client having a pathology image processing and displaying device, the pathology image processing and displaying device comprising:
the pathological image acquisition module is used for acquiring pathological images, wherein the pathological images are pathological microscopic images of the stained tissue samples;
the pathological image sending module is connected with the pathological image acquisition module and a service end and is used for sending the pathological image to the service end so that the service end can identify a specific part in the pathological image;
the image preprocessing module is connected with the pathological image acquisition module and is used for preprocessing the pathological image to obtain a plurality of pathological image block sets, wherein each pathological image block set corresponds to different resolutions, and all pathological image blocks contained in each pathological image block set can be spliced into the pathological image with the corresponding resolution;
the display instruction acquisition module is used for receiving a display instruction, wherein the display instruction is used for designating a zoom degree and a display area;
the image acquisition module is connected with the display instruction acquisition module and the image preprocessing module and is used for acquiring a plurality of pathological image blocks with corresponding resolution according to the zoom degree and the display area and splicing the plurality of pathological image blocks acquired according to the zoom degree and the display area to form a pathological image corresponding to the display area;
The image acquisition module is connected with the display instruction acquisition module and the server and is used for acquiring an identification image corresponding to the specific part according to the display instruction and the identification result of the server on the specific part, wherein the identification image comprises the outline of the specific part marked by curves with different colors;
the image acquisition module is connected with the pathological image recognition module through the image preprocessing module; the pattern preprocessing module is used for preprocessing the recognition result of the specific part to obtain a plurality of mark pattern block sets, wherein mark pattern blocks in the mark pattern block sets correspond to pathological image blocks in the pathological image block sets; the graph acquisition module acquires a plurality of corresponding identification graph blocks according to the zoom degree and the display area, and splices the acquired identification graph blocks to form an identification graph corresponding to the specific part;
and the display module is connected with the image acquisition module and is used for displaying the pathological image corresponding to the display area and the identification image corresponding to the specific part in a superposition way.
8. A server, wherein the server comprises a pathology image processing apparatus, the pathology image processing apparatus comprising:
a pathology image receiving module for receiving a pathology image sent by the client according to claim 7, the pathology image being a pathology microscopic image of a stained tissue sample;
the pathology image identification module is connected with the pathology image receiving module and is used for identifying a specific part in the pathology image;
and the identification result sending module is connected with the pathological image identification module and is used for sending the identification result of the pathological image identification module on the specific part to the client.
9. A computer readable storage medium having stored thereon a computer program, characterized in that the computer program when executed by a processor implements a first pathology image processing and displaying method, a second pathology image processing and displaying method or a pathology image processing method, wherein:
the first pathological image processing and displaying method comprises the following steps:
acquiring a pathology image, wherein the pathology image is a pathology microscopic image of a stained tissue sample;
preprocessing the pathology image to obtain a plurality of pathology image block sets, wherein each pathology image block set corresponds to different resolutions, and all pathology image blocks contained in each pathology image block set can be spliced into a pathology image with the corresponding resolution;
Identifying a specific location in the pathology image;
receiving a display instruction, wherein the display instruction is used for designating a zoom degree and a display area;
acquiring a plurality of pathological image blocks with corresponding resolution according to the zoom degree and the display area, and splicing the acquired pathological image blocks to form a pathological image corresponding to the display area;
acquiring the identification graph corresponding to the specific part, which comprises the following steps: preprocessing the recognition result of the specific part to obtain a plurality of identification graph block sets, wherein the identification graph blocks in the identification graph block sets correspond to the pathological image blocks in the pathological image block sets, and the identification graph comprises outlines of the specific part marked by curves with different colors; acquiring a plurality of corresponding identification graph blocks according to the zoom degree and the display area, and splicing the acquired identification graph blocks to form an identification graph corresponding to the specific part;
superposing and displaying a pathological image corresponding to the display area and an identification graph corresponding to the specific part;
the second pathological image processing and displaying method is applied to a client and comprises the following steps:
Acquiring a pathology image, wherein the pathology image is a pathology microscopic image of a stained tissue sample;
transmitting the pathological image to a service end so that the service end can identify a specific part in the pathological image;
preprocessing the pathology image to obtain a plurality of pathology image block sets, wherein each pathology image block set corresponds to different resolutions, and all pathology image blocks contained in each pathology image block set can be spliced into a pathology image with the corresponding resolution;
receiving a display instruction, wherein the display instruction is used for designating a zoom degree and a display area;
acquiring a plurality of pathological image blocks with corresponding resolution according to the zoom degree and the display area, and splicing the pathological image blocks acquired according to the pathological image blocks to form a pathological image corresponding to the display area;
acquiring an identification graph corresponding to the specific part according to the display instruction and the identification result of the specific part by the server side, wherein the identification graph comprises the following steps: preprocessing the recognition result of the specific part to obtain a plurality of identification graph block sets, wherein the identification graph blocks in the identification graph block sets correspond to the pathological image blocks in the pathological image block sets; acquiring a plurality of corresponding identification graph blocks according to the zoom degree and the display area, and splicing the acquired identification graph blocks to form an identification graph corresponding to the specific part;
Superposing and displaying a pathological image corresponding to the display area and an identification graph corresponding to the specific part;
the pathological image processing method is applied to a server and comprises the following steps:
receiving a pathology image sent by a client, wherein the pathology image is a pathology microscopic image of a stained tissue sample;
identifying a specific location in the pathology image;
and sending the identification result of the pathological image identification module on the specific part to the client.
CN202110691405.4A 2021-06-22 2021-06-22 Pathological image processing and displaying system, client, server and medium Active CN113436160B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202110691405.4A CN113436160B (en) 2021-06-22 2021-06-22 Pathological image processing and displaying system, client, server and medium
PCT/CN2021/137331 WO2022267379A1 (en) 2021-06-22 2021-12-13 Pathological image processing and display system, client, server and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110691405.4A CN113436160B (en) 2021-06-22 2021-06-22 Pathological image processing and displaying system, client, server and medium

Publications (2)

Publication Number Publication Date
CN113436160A CN113436160A (en) 2021-09-24
CN113436160B true CN113436160B (en) 2023-07-25

Family

ID=77756928

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110691405.4A Active CN113436160B (en) 2021-06-22 2021-06-22 Pathological image processing and displaying system, client, server and medium

Country Status (2)

Country Link
CN (1) CN113436160B (en)
WO (1) WO2022267379A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113436160B (en) * 2021-06-22 2023-07-25 上海杏脉信息科技有限公司 Pathological image processing and displaying system, client, server and medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110727097A (en) * 2019-12-19 2020-01-24 上海兰脉信息科技有限公司 Pathological microscopic image real-time acquisition and analysis system, method, device and medium
CN110956584A (en) * 2019-11-05 2020-04-03 上海杏脉信息科技有限公司 Pathological microscopic image interactive display method, medium and equipment
CN111260677A (en) * 2020-02-20 2020-06-09 腾讯科技(深圳)有限公司 Cell analysis method, device, equipment and storage medium based on microscopic image
CN112035691A (en) * 2020-09-23 2020-12-04 上海商汤智能科技有限公司 Method, device, equipment and medium for displaying cell labeling data of slice image

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130305138A1 (en) * 2012-05-14 2013-11-14 Pacsthology Ltd. Systems and methods for acquiring and transmitting high-resolution pathology images
US20220138939A1 (en) * 2019-02-15 2022-05-05 The Regents Of The University Of California Systems and Methods for Digital Pathology
CN111784711A (en) * 2020-07-08 2020-10-16 麦克奥迪(厦门)医疗诊断系统有限公司 Lung pathology image classification and segmentation method based on deep learning
CN111915622B (en) * 2020-07-09 2024-01-23 沈阳先进医疗设备技术孵化中心有限公司 Training of image segmentation network model and image segmentation method and device
CN112435243A (en) * 2020-11-26 2021-03-02 山东第一医科大学附属省立医院(山东省立医院) Automatic analysis system and method for full-slice digital pathological image
CN113436160B (en) * 2021-06-22 2023-07-25 上海杏脉信息科技有限公司 Pathological image processing and displaying system, client, server and medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110956584A (en) * 2019-11-05 2020-04-03 上海杏脉信息科技有限公司 Pathological microscopic image interactive display method, medium and equipment
CN110727097A (en) * 2019-12-19 2020-01-24 上海兰脉信息科技有限公司 Pathological microscopic image real-time acquisition and analysis system, method, device and medium
CN111260677A (en) * 2020-02-20 2020-06-09 腾讯科技(深圳)有限公司 Cell analysis method, device, equipment and storage medium based on microscopic image
CN112035691A (en) * 2020-09-23 2020-12-04 上海商汤智能科技有限公司 Method, device, equipment and medium for displaying cell labeling data of slice image

Also Published As

Publication number Publication date
CN113436160A (en) 2021-09-24
WO2022267379A1 (en) 2022-12-29

Similar Documents

Publication Publication Date Title
CN109508681B (en) Method and device for generating human body key point detection model
CN111563502B (en) Image text recognition method and device, electronic equipment and computer storage medium
CN108197618B (en) Method and device for generating human face detection model
CN117456023A (en) Virtual staining of tissue slice images
US7931602B2 (en) Gaze guidance degree calculation system, gaze guidance degree calculation program, storage medium, and gaze guidance degree calculation method
CN108734078B (en) Image processing method, image processing apparatus, electronic device, storage medium, and program
CN104658030B (en) The method and apparatus of secondary image mixing
Fu et al. A computational cognition model of perception, memory, and judgment
CN113411550B (en) Video coloring method, device, equipment and storage medium
CN110298569B (en) Learning evaluation method and device based on eye movement recognition
US11238623B2 (en) Automatic line drawing coloring program, automatic line drawing coloring apparatus, and graphical user interface program
CN110490882B (en) Cell membrane staining image analysis method, device and system
CN113706562B (en) Image segmentation method, device and system and cell segmentation method
CN109359517A (en) Image-recognizing method and device, electronic equipment, storage medium, program product
CN113436160B (en) Pathological image processing and displaying system, client, server and medium
CN114168052A (en) Multi-graph display method, device, equipment and storage medium
CN110427915A (en) Method and apparatus for output information
CN110473176B (en) Image processing method and device, fundus image processing method and electronic equipment
JP2023131117A (en) Joint perception model training, joint perception method, device, and medium
CN114842035A (en) License plate desensitization method, device and equipment based on deep learning and storage medium
CN108921138B (en) Method and apparatus for generating information
CN113313705B (en) Pathological image processing system, method and medium
CN113781387A (en) Model training method, image processing method, device, equipment and storage medium
CN108764248A (en) The extracting method and device of image characteristic point
CN114283178A (en) Image registration method and device, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant