TWI778718B - Specimen slide scrape guide system and method thereof - Google Patents

Abstract

A specimen slide scrape guide system and method thereof are provided by the disclosure. The disclosure provides an electronic image of a specimen by an interaction interface, the user is allowable to adjust the parameter to instantly change the analysis result marked on the electronic image. The disclosure divides the electronic image into multiple regions, selects at least one of regions according to the analysis result, labels them, generates and outputs a scrape guide. Via superimposing the physical specimen slide on the scrape guide provided by the disclosure, an obvious mark effect can be implemented on the physical specimen slide, and assist the user to scrape the specimen effectively.

Description

Scraping guide system and method for specimen slides

The present invention is related to guiding the scraping operation, and in particular, to guiding the scraping of the specimen slide.

In the existing methods for scraping a specimen, professional clinical personnel mostly read the solid glass slide of the specimen, and then manually determine the area that needs to be scraped, and then perform the scraping operation.

The accuracy and efficiency of the aforementioned method of manually judging the scraped area completely depend on the professionalism of the clinical staff, but this method cannot ensure that the interpretation standards or circle selection methods of different clinical staff are consistent, resulting in uneven quality of the specimens.

Therefore, the existing scraping samples have the above problems, and more effective solutions are urgently needed.

The main purpose of the present invention is to provide a scraping system and scraping method for guiding specimen slides, which can generate and output scraping guidelines for assisting scraping of specimens.

In one embodiment, a scraping guide method for specimen slides includes: a displaying step including displaying an electronic image of a specimen on an interactive interface, wherein the electronic image is marked A first analysis result of a first parameter value; a changing step includes displaying the electronic image marked with a second analysis result of a changed second parameter value when the parameter value changing operation is accepted, wherein the The first analysis result and the second analysis result represent the distribution of target cells with different smoothness; a segmentation step includes superimposing the electronic image with a region segmentation map to segment the electronic image into a plurality of sub-regions , wherein an output size of the region segmentation map is the size of a solid slide corresponding to the specimen; a selecting step includes selecting at least a part of the subregion based on the analysis result; and, a guiding step includes segmenting the region Mark the selected sub-area to generate a scratching guideline, and output the scratching guideline based on the output size, wherein after the outputted scratching guideline is overlapped with the solid glass slide, the scratching guideline is output. The marker in the guide has the effect of marking the solid slide.

In one embodiment, a scraping guide system for specimen slides includes: an image library for storing an electronic image of a specimen; and a platform end connected to the image library, the platform end It is used to establish a network connection with a client through the network, and provide an interactive interface to the client, and the platform is configured to display a first analysis result marked with a first parameter value on the interactive interface the electronic image, and when a parameter value change operation is accepted through the interactive interface, the electronic image marked with a second analysis result of the changed second parameter value is displayed on the interactive interface, and the platform is It is configured to superimpose the electronic image and a region segmentation map on the interactive interface to divide the electronic image into a plurality of sub-regions. An output size of the region segmentation map is the size of a physical slide corresponding to the specimen. The platform side is configured to select at least a part of the sub-region based on the analysis result, mark the selected sub-region in the region segmentation map to generate a scraping guideline, and output the scraping guideline based on the output size , wherein after the output scraping guide is superimposed with the solid glass slide, the mark in the scraping guide has a marking effect on the solid glass slide.

By superimposing the scraping guide outputted by the present invention and the solid glass slide, the solid glass slide can be marked, which can assist the user to efficiently perform the sample scraping, and improve the standard of the sample scraping standard. Consistency.

1: Platform side

10: Processing modules

100: web module

101: Interactive Control Module

102: Output control module

11: Storage Module

110: Storage Array

12: Communication module

13: Human-Machine Interface

2: Client

20: Processor

21: Input device

22: Display

220: Electronic Scraping Guidelines

23: Network Devices

24: Printing device

25: Storage device

3: Identification and analysis module

30: Learning Models

31: Algorithm module

4: Database

40: Image Library

41: Results library

50: Physical Scraping Guidelines

51: Solid slides

60: Parameter value transformation module

61: Split Module

62: Manual selection of modules

63:Automatically select modules

630: Purity calculation module

631: External computing module

632: Recommended regional decision module

633: Area Restriction Module

64: Rendering Mods

65: Guide to generate modules

66: Image adjustment module

70-71, 80-81: Cluster

72, 82: Grid area

S10-S15: Generate scraping guide steps

S20-S24: Steps for changing parameter values

S30-S32: Manual selection steps

S40-S47: Automatic selection steps

S50-S54: Calibration Output Size Steps

S60-S64: Steps for generating analysis results

S70-S72: Target recognition and analysis steps

S80-S73: Cell identification and analysis steps

S90-93: Analysis and Filtering Steps

Fig. 1 shows the current scraping and inspection procedure of the specimen.

FIG. 2 is a structural diagram of a scraping guide system according to an embodiment of the present invention.

FIG. 3 is a structural diagram of a processing module on a platform side according to an embodiment of the present invention.

FIG. 4 is a flowchart of a scraping guide method according to an embodiment of the present invention.

FIG. 5 is a flowchart of changing parameter values according to an embodiment of the present invention.

FIG. 6 is a flowchart of selecting a sub-region according to an embodiment of the present invention.

FIG. 7 is a flow chart of calibrating output size according to an embodiment of the present invention.

FIG. 8 is a flow chart of generating analysis results according to an embodiment of the present invention.

FIG. 9 is a flowchart of target identification and analysis according to an embodiment of the present invention.

FIG. 10 is a flowchart of cell identification and analysis according to an embodiment of the present invention.

FIG. 11 is a flowchart of generating an analysis result of parameter values according to an embodiment of the present invention.

FIG. 12 is a schematic diagram of an interface for selecting a sub-region according to an embodiment of the present invention.

FIG. 13 is a schematic diagram of an interface for marking sub-regions according to an embodiment of the present invention.

FIG. 14 is a schematic diagram of the use of the scraping guide according to an embodiment of the present invention.

FIG. 15 is a schematic diagram of marking an electronic image based on a first parameter value according to an embodiment of the present invention.

16 is a schematic diagram of marking an electronic image based on a second parameter value according to an embodiment of the present invention.

FIG. 17 is a schematic diagram of scale measurement according to an embodiment of the present invention.

FIG. 18 is a schematic diagram of the calibration output size based on FIG. 17 .

FIG. 19 is a schematic diagram of the scraping guide generated based on FIG. 18 .

Hereinafter, a preferred embodiment of the present invention will be described in detail in conjunction with the drawings.

Please refer to Fig. 1 for the current scraping and inspection procedure of the specimen. At present, there is a computer-assisted method for scraping a specimen, which can assist clinical personnel to identify the position of the specimen that needs to be scraped.

Specifically, after obtaining a specimen (eg, surgically excised diseased tissue or other tissues to be tested), the specimen is cut into multiple thin sections, and multiple solid glass slides are used as carriers. Thin sections of the specimen on these solid slides, because they are taken from the same tissue location, usually have the same or similar appearance and cellular composition.

Also, one of these solid slides will be used to scan into an electronic image, ie an electronic image of a thin section of the specimen. Performing feature analysis on the electronic image through a computer can generate analysis results, such as the location of the diseased tissue in the electronic image.

Then, the clinician can manually mark the target cells (such as tumor cells or other diseased cells) on the remaining solid slides by watching the analysis results displayed on the computer screen to mark the location to be scraped.

Finally, the clinical staff scrapes the manually marked position, and sends the scraped tissue to a special inspection unit for inspection and analysis, such as Next Generation Sequencing (NGS).

However, when clinical personnel look at the computer screen to manually mark solid slides, due to the difference between the virtual and the real, it is difficult for the clinical personnel to accurately circle the correct position, which may easily lead to labeling errors.

In addition, the above-mentioned computer-assisted specimen scraping method can only display the analysis results of a set of preset criteria (amplified and fixed tumor boundaries), and cannot provide the analysis results of different preset criteria for clinical personnel to compare or select, and cannot Meet the changing needs of the clinic.

In addition, since the production of solid glass slides is easily affected by cutting or scraping (knife marks, tissue cracks or position deviation, etc.), even two adjacent tissue slides are not necessarily the same in shape and characteristics, which makes The analysis results displayed on the computer screen may not completely correspond to the solid glass slides, and therefore cannot be accurately compared, that is, there may be differences in the shape of the samples of the solid glass slides used for feature analysis and the samples of the solid glass slides used for scraping.

In order to solve the above problems, the present invention proposes a scraping guide system and a scraping guide method for specimen slides, which can automatically suggest scraping areas and generate scraping guides. Moreover, after the user (eg, clinical staff) superimposes the output scraping guide with the solid glass slide, the scraping guide can clearly mark the position suitable for scraping.

In addition, the scraping guidance system and scraping guidance method of the present invention can also allow clinical personnel to quickly switch between different analysis results with different rigours (parameter values), so as to select a scraping area suitable for the current analysis results.

Please also refer to FIG. 2 , which is a structural diagram of a scraping guide system according to an embodiment of the present invention. The scraping guide system of the present invention mainly includes a platform terminal 1 and a user terminal 2 connected through a network (eg, the Internet or a local area network).

The platform 1 (such as a server or a cloud service platform and other computer equipment that can provide network services) is used to set up a website (web service), and provide electronic images and scraping guide services through the website.

Client 2 (such as desktop computer, notebook computer, tablet computer, etc., computer equipment that can be operated by the user) is used to connect to the platform terminal 1 through the network, so that the user can operate the webpage to monitor the electronic data of the designated specimen. The image performs the generation and editing of scraping guidelines (electronic scraping guidelines 220).

Moreover, the user can operate the user terminal 2 to output the electronic scraping guide 220 to obtain the physical scraping guide 50 (eg paper), and superimpose the physical scraping guide 50 with the physical glass slide 51 of the specimen.

In this way, the physical scraping guide 50 can mark the solid glass slide 51 so that the user can scrape the specimen on the solid glass slide 51 according to the marking.

It is worth mentioning that the aforementioned electronic scraping guide 220 and the physical scraping guide 50 correspond to real and virtual ones, that is, the physical scraping guide 50 is the actualized electronic scraping guide 220 . The present invention can eliminate the gap between the virtual and the real, and improve the efficiency and accuracy of marking by scraping and guiding the material.

In one embodiment, the platform end 1 may include a storage module 11 , a communication module 12 , a human-machine interface 13 and a processing module 10 electrically connected to the above-mentioned components.

The storage module 11 (eg, a storage device such as RAM, EEPROM, solid state hard disk, magnetic disk hard disk, flash memory or any combination thereof) is used to store data. The communication module 12 (eg, network interface card, NIC) is used to connect to the network (eg, the Internet), and through the network and external devices (eg, client 2 , database 4 and/or identification and analysis module 3 ) )communication. The human-machine interface 13 (including input interface and output interface, such as mouse, keyboard, various keys, touch pad, display, touch screen, projection module, etc.) is used for the operator of the platform 1 (such as network management personnel) ) to interact. The processing module 10 (which can be a processor such as CPU, GPU, TPU, MCU, or any combination thereof) is used to control the platform terminal 1 and realize the functions proposed by the present invention.

In one embodiment, the processing module 10 may include a web module 100 , an interactive control module 101 and/or an output control module 102 .

The web module 100 (eg, a web server module) is configured to provide web services to connect with the client 2 through a web protocol (eg, HTTP or HTTPS).

In one embodiment, the web module 100 can verify the client 2 (eg, account password verification, one-time password verification, binding hardware verification, etc.), and allow the client 2 to use the service only after the client 2 passes the verification.

The interactive control module 101 is configured to generate an interactive interface (which can be a graphical user interface (GUI), such as the interfaces shown in FIG. 12 to FIG. 19 ) in the webpage displayed in the client terminal 2, and through the interactive interface Receive the operation of the client 2 and display information to the user.

The output control module 102 is configured to transmit the generated (edited) scraping guide to the client 2 .

In one embodiment, the client 2 may include an input device 21 , a display 22 , a network device 23 , a printing device 24 , a storage device 25 , and a processor 20 electrically connected to the above-mentioned devices.

The input device 21 (such as a mouse, a keyboard, a touch panel, etc.) is used for the user to input data or operate. The display 22 (eg, a liquid crystal display, a projector, a touch screen, etc.) is used for displaying information (eg, displaying a web page and generating an electronic scraping guide 220 ). The network device 23 (eg, a network interface card, NIC) is used to connect to a network (eg, the Internet), and communicate with an external device (eg, the platform 1 ) through the network. The printing device 24 (such as a printer) is used to print the electronic scraping guide 220 as the physical scraping guide 50, such as printing on paper or thin transparent plate, or directly printing on the physical glass slide 51, etc. be limited. The storage device 25 (such as RAM, EEPROM, solid state hard disk, magnetic disk hard disk, flash memory or any combination thereof) is used to store data. The processor 20 (such as CPU, GPU, TPU, MCU or any combination thereof) is used to control the client 2 .

In one embodiment, the platform end 1 can be connected with information (eg, through a network connection or through a local line connection) to identify and analyze the module 3 . The identification and analysis module 3 can be built in a server or a cloud service platform (such as Amazon Web Service, Google Cloud Platform, or Microsoft Azure, etc.), and is used for object identification and analysis of electronic images (details will be described later).

In one embodiment, the identification and analysis module 3 can be directly built in the platform side 1 .

In one embodiment, the recognition and analysis module 3 can perform automatic image recognition and analysis based on artificial intelligence (AI, Artificial Intelligence), and includes a learning model 30 and an algorithm module 31 .

A learning model 30 (eg, a classifier) may be configured to perform image classification (eg, image classification of cells) based on machine learning.

The algorithm module 31 is configured to use the learning model 30 to identify cells (eg, non-target cells and target cells), and can calculate the number of cells at the location of each target cell.

In one embodiment, the platform 1 can be connected to a database 4 (eg, a network database, a local database, a relational database, etc., or a combination thereof) for storing a large amount of data by information connection (eg, through a network). .

In one embodiment, the platform 1 may include an image library 40, and the image library 40 may store a plurality of electronic images of a plurality of specimens.

In one embodiment, the platform 1 may include a result library 41 . The result library 41 can store the analysis results of the electronic images of each specimen (such as the result storage array 110 described later).

Please also refer to FIG. 3 , which is a structural diagram of a processing module on a platform side according to an embodiment of the present invention. Processing module 10 may include modules 60-66, and automatic selection module 63 may include modules 630-633.

These modules 60-66, 630-633, the modules 100-102 of FIG. 2, and the identification and analysis module 3 are respectively configured to perform different functions (described in detail later).

The aforementioned modules are connected to each other (which can be electrical connection and information connection), and can be hardware modules (such as electronic circuit modules, integrated circuit modules, SoCs, etc.), software modules (such as firmware modules, etc.) system, operating system or application) or a mix of software and hardware modules, without limitation.

It is worth mentioning that when the aforementioned module is a software module (such as firmware, operating system or application), the storage module 11 of the platform 1 may include a non-transitory computer-readable recording medium, and the aforementioned non-transitory computer-readable recording medium may be included. The transient computer-readable recording medium stores a computer program, and the computer program records a computer-executable program code. After the processing module 10 executes the aforesaid program code, the function of the corresponding module can be realized.

Please also refer to FIG. 4 , which is a flowchart of a scraping guide method according to an embodiment of the present invention. The scraping and guiding methods of the various embodiments of the present invention can be implemented by the scraping and guiding systems shown in FIGS. 2-3 .

In the scraping guide method of this embodiment, the platform terminal 1 can provide web services to the client terminal 2 through the network and the web module 100 . The client 2 can send a service usage request to the platform 1 and perform identity verification, and after passing the verification, log in to the webpage (or without verification), and display the interactive interface provided by the platform 1 through the interactive control module 101 .

Step S10: The platform 1 can display the electronic image of the specimen on the interactive interface. In one embodiment, the user terminal 2 can select the electronic image of the specified specimen through the interactive interface, and the platform terminal 1 loads the specified electronic image (such as hematoxylin and eosin (H&E) stained glass) from the image library 40 . film images).

Step S11 : the platform terminal 1 can set parameter values (which can be preset by the system or set by the user terminal 2 ), obtain the analysis results of the parameter values, and mark the analysis results on the electronic image through the rendering module 64 .

In one embodiment, the result library 41 can pre-store different analysis results of the same electronic image (same specimen) using different parameter values, and read the corresponding analysis results according to the currently set parameter values.

In one embodiment, different parameter values represent different degrees of stringency or smoothness (described in detail later), and different analysis results of different parameter values represent the distribution state of target cells with different degrees of smoothness.

In one embodiment, different analysis results of different parameter values are markers corresponding to different rigor.

In one embodiment, the platform 1 can generate different analysis results with different parameter values immediately after loading the electronic image, and present the corresponding analysis results in the electronic image according to the parameter values set by the user.

In one embodiment, the client 2 can switch different parameter values in real time (such as switching from the first parameter value to the second parameter value), so that the platform 2 can display different analysis results on the electronic image in real time (such as switching from the first parameter value to the second parameter value). One analysis result is transformed to the second analysis result).

Step S12 : The platform end 1 can divide the electronic image into a plurality of sub-regions by superimposing the electronic image and the region segmentation map through the segmentation module 61 . The output size of the aforementioned region segmentation map is the size (eg, the same or a fixed scale) of the solid slide corresponding to the specimen.

Step S13: The platform 1 can automatically select at least a part of the sub-regions (eg, select a part or all of them, not limited) based on the analysis result through the automatic selection module 63, such as sub-regions where the target cells are located or are easily scraped.

In one embodiment, the platform side 1 can use the manual selection module 62 for the user terminal 2 to manually select the sub-areas.

In one embodiment, the platform side 1 can render the selected sub-region in the interactive interface in real time through the rendering module 64 .

Step S14 : The platform side 1 can mark the selected sub-areas in the area segmentation map through the guide generating module 65 to generate a scraping guide (electronic scraping guide 220 ).

In one embodiment, the platform side 1 can display the generated electronic scraping guide 220 in the interactive interface in real time through the rendering module 64 .

Step S15 : the platform terminal 1 outputs the scraping guide with the specified output size to the user terminal 2 through the output control module 102 .

In one embodiment, the platform side 1 may request the client side 2 to allow the printing device 24 to be controlled to print the generated scraping guide as a paper copy.

In one embodiment, the client 2 can actively print out the scraping guide (physical scraping guide 50 ) with a specified output size. In addition, after the printed scraping guide is superimposed with the solid glass slide 51 , the mark in the scraping guide has a marking effect on the suggested scraping position of the solid glass slide 51 .

In this way, by superimposing the scraping guide outputted by the present invention and the solid glass slide, the solid glass slide can be marked, and the user can be assisted to effectively perform the scraping of the specimen.

Please refer to FIG. 12 to FIG. 14 together. FIG. 12 is a schematic diagram of an interface for selecting a sub-region according to an embodiment of the present invention. FIG. 13 is a schematic diagram of an interface for marking a sub-area according to an embodiment of the present invention, and FIG. 14 is a schematic diagram of the use of a scraping guide according to an embodiment of the present invention. 12 to 14 are used to illustrate a specific implementation of the present invention.

In this example, as shown in FIGS. 12 and 13 , the region division map includes a plurality of grid lines, and the plurality of grid lines overlapping the electronic image divides the electronic image of the specimen image into a plurality of grid regions as a plurality of sub-regions. .

The platform 1 can present the analysis result of the current parameter value on the electronic image in the interactive interface, so as to display the circle mark of the target cell on the electronic image (eg, circle the position of the target cell). The smoothness of the aforementioned circle check marks varies with the parameter value.

In addition, the platform side 1 can display the electronic image divided into multiple sub-regions in the interactive interface, and automatically select the sub-regions that are recommended for scraping (eg, the purity of the target cells is high), such as clusters of multiple grid regions 70 , 71 .

In one embodiment, the platform 1 can also perform marking with different colors or patterns according to the purity of the target cells in each grid area (eg, tumor cell purity, Tumor Cell Purity, TCP), such as the color of the grid area with higher purity the deeper.

In one embodiment, the platform 1 can also display sub-regions, such as the grid region 72 , which are not recommended for scraping (eg, the purity of the target cells is too low) on the interactive interface.

In addition, the client 2 can manually select or deselect grid areas to change the extent of the clusters 70, 71

As shown in FIG. 13 , after the selection is completed, the user terminal 2 may request to generate an electronic scraping guide. The scraping guide in this example is a line marking (or, as shown in Figure 19, a color block marking) where the specimen is to be scraped.

Next, as shown in FIG. 14 , after the electronic scratching guide is printed as a physical scratching guide, it can be used to overlap with the physical glass slide, so that the marks on the physical scratching guide (such as the range marks of clusters 70 and 71 and the The range marking of the grid area 72 ) can clearly mark the specimen slices in the solid slide, so that the user can perform the scraping operation correctly and efficiently.

Please also refer to FIG. 5 , which is a flowchart of changing parameter values according to an embodiment of the present invention. Compared with the scraping guide method of FIG. 4 , the scraping guide method of this embodiment further provides a parameter value changing function implemented by steps S20 - S24 .

Step S20: The platform side 1 switches to the parameter value setting mode. In one embodiment, the platform terminal 1 can switch to the parameter value setting mode after the electronic image is loaded (step S10 ), so that the user terminal 2 can set appropriate parameter values.

Step S21: The platform terminal 1 identifies the currently set parameter value (eg, the first parameter value) through the parameter value transformation module 60, and obtains the analysis result (eg, the first analysis result) of the current parameter value.

In one embodiment, the platform 1 can obtain the quantity or purity of target cells (such as tumor cells and other diseased cells) at each image position of the electronic image of the specimen image, and the aforementioned information can be pre-analyzed and stored in the result database 41 ( Such as through the steps S60-S64 described later). Then, the platform terminal 1 calculates the distribution positions of target cells with different rigours according to all the parameter values, and adds these calculated positions to the analysis results of the corresponding parameter values, which are presented for subsequent user selection.

Furthermore, the aforementioned different parameter values correspond to different degrees of rigor respectively. The present invention mainly calculates distribution markers of target cells with different smoothness based on different parameter values. When the smoothness is higher, it means that the target cell identification range is wider, that is, the degree of rigour in identifying the target cell is lower; The lower the slip, the narrower the target cell identification range is, that is, the higher the degree of rigour in identifying the target cell. Thereby, the present invention can provide the user to obtain the distribution of the identified target cells under different rigor.

Step S22: The platform side 1 marks the analysis result of the current parameter value on the electronic image through the rendering module 64, such as displaying the analysis result in the electronic image, so as to use the mark with the corresponding smoothness to circle the distribution position of the target cells The image position of the target cell (the image position of the target cell shown in Figure 12).

Step S23: The platform terminal 1 detects through the parameter value transformation module 60 whether the parameter value change operation of the client terminal 2 is accepted through the interactive interface.

If the user terminal 2 changes the parameter value, it identifies the transformed new parameter value (such as the second parameter value), and executes steps S21-S22 again to obtain the analysis result of the new parameter value (such as the second analysis result), and Electronic images are tagged with new analysis results.

If the user terminal 2 does not change the parameter value or has determined the final parameter value, step S24 is executed: the platform terminal 1 determines whether to leave the parameter value setting mode, such as whether the user decides not to change the parameter value.

If the parameter value setting mode is not left, step S23 is performed again to detect again. Otherwise, continue with the other steps of the method.

Thereby, the present invention can provide analysis results with different rigor, that is, markers with different smoothness, and can meet the requirements of different rigor in clinical practice.

Please also refer to FIG. 11 , which is a flow chart of generating analysis results of parameter values. Compared with the scraping guide method of FIG. 4 and FIG. 5 , the scraping guide method of the present embodiment further provides the analysis results of target cells of different rigor achieved by steps S90-S93.

In this embodiment, different parameter values are used to represent different degrees of rigor, and the degree of image detail retained by the filtering is controlled according to the different parameter values.

Step S90: The platform side 1 reads all possible parameter values. For example, multiple severity levels are specified in advance, and multiple severity levels are respectively corresponding to multiple parameter values). For example, a parameter value of 50 corresponds to a lower stringency; a parameter value of 70 corresponds to a higher stringency.

Step S91: The platform terminal 1 calculates the corresponding filter mask according to different parameter values. Different filter masks corresponding to different parameter values have different filtering capabilities, for example, they are used to filter out signals in different frequency ranges.

Step S92: The platform terminal 1 uses different filter masks with different parameter values to perform filtering processing on the image frequency domain respectively, so as to obtain different filtering results of different filter masks.

In one embodiment, the platform 1 first converts the electronic image from the spatial domain to the frequency domain (eg, through Fourier transform, fast Fourier transform, etc.), and then performs the aforementioned filtering process.

In one embodiment, the aforementioned filter masks are low-pass filter masks, and each filter mask is used to filter high-frequency signals in different frequency ranges in the filtering process. For example, a wider low-pass filter mask will filter more high-frequency signals (less strictness), and a narrower low-pass filter mask will retain more high-frequency signals (higher strictness).

Step S93 : After filtering, the platform 1 can obtain the image positions of the target cells under different parameter values (different filter masks) as the analysis results of the parameter values, and store them in the database 4 .

In this way, the present invention can generate analytical results of varying degrees of stringency.

Please refer to FIG. 15 to FIG. 16 together. FIG. 15 is a schematic diagram of marking an electronic image based on a first parameter value according to an embodiment of the present invention, and FIG. 16 is a schematic diagram of marking an electronic image based on a second parameter value according to an embodiment of the present invention. Schematic.

In this example, the target cells are living tumor cells (Variable Cancer Cell, VCC). In this example, the parameter value represents the severity of rendering VCC, and the parameter value is proportional to the severity. proportional. That is, the higher the parameter value, the higher the degree of rigor (the more precise the marking boundary is, and the more difficult it is to circle selection); the lower the parameter value, the lower the degree of rigour (the smoother the marking boundary is, and the easier it is to circle selection), but not limited to this.

In another example, the parameter value can be set to be inversely proportional to the stringency. That is, the higher the parameter value, the lower the rigor; the lower the parameter value, the higher the rigor.

It is worth mentioning that the present invention can control the degree of retention of marking details in the filtering process (as shown in the step shown in FIG. 11 ) based on different parameter values, so as to provide different degrees of rigor.

In Fig. 15, the parameter value is 70, that is, a high degree of rigor is set. Under this setting, the filtering process will retain more high-frequency signals (ie, details), and generate a more detailed indication of the image position of the VCC, thereby providing a detection result of the VCC position with a higher degree of rigor.

In Fig. 16, the parameter value is 50, that is, a low degree of rigor is set. Under this setting, the filtering process will retain less high frequency signal, resulting in a rougher indication of the image position of the VCC, thereby providing a less rigorous detection result of the VCC position.

In this way, the present invention allows users to set different parameter values according to different rigor requirements, and view the corresponding analysis results in real time, and can provide analysis results of different standards for clinical personnel to compare or select, so as to avoid failing to meet the changing needs of the clinic. .

FIG. 6 is a flowchart of selecting a sub-region according to an embodiment of the present invention. Compared with the scraping guide method of FIG. 4 , the scraping guide method of this embodiment further provides a manual selection mode implemented by steps S30-S32 and an automatic selection mode implemented by steps S40-S47 in step S13.

Manual selection mode includes the following steps:

Step S30 : the platform terminal 1 accepts the sub-area selection operation of the user terminal 2 through the manual selection module 62 and the interactive interface. The aforementioned sub-area selection operation is to select or deselect at least one sub-area.

Step S31 : The platform side 1 changes the state of the sub-region selected in step S30 to selected, unselected or not scraped through the manual selection module 62 .

In one embodiment, when the sub-region is selected for the first time, its state is changed from unselected to selected; when the sub-region is selected for the second time, its state is changed from selected to unselected.

In one embodiment, the user terminal 2 can directly set the state of some sub-regions as not to be scraped (eg, a region with low purity of target cells or a region that is not easy to scrape), so as to avoid subsequent scraping of this region by mistake.

Step S32 : the platform side 1 presents the changed states of all the sub-regions on the interactive interface in real time through the rendering module 64 .

The automatic selection mode includes the following steps.

Step S40: The platform side 1 obtains the purity of each sub-region of the electronic image through the purity calculation module 630, and selects the sub-region whose purity is greater than the preset purity (eg, 30%, 50% or 70%).

Step S41 : The platform side 1 sets the adjacent multiple sub-region clusters as candidate regions with a wider range in the selected multiple sub-regions through the external computing module 631 .

In one embodiment, the platform end 1 can generate a minimum circumscribed rectangular area of a plurality of sub-areas as a candidate area.

It is worth mentioning that since the area of a single sub-region is too small, it is not easy to scrape. The present invention can effectively improve the correctness and efficiency of the scraping operation by clustering a plurality of adjacent sub-regions suitable for scraping into a larger suggested range.

Step S42 : The platform terminal 1 determines whether each candidate region meets the preset conditions one by one through the recommended region decision module 632 .

In one embodiment, the aforementioned predetermined condition may include that the purity of the candidate region is not less than the predetermined purity (eg, 30%, 50%, or 70%).

In one embodiment, the aforementioned predetermined condition may include that the number of target cells in the candidate region is not less than a predetermined number (eg, 5000, 10000, 15000, 20000, etc.).

If any candidate region meets the preset condition, step S43 is performed for the matching candidate region: the platform 1 sets the matching candidate region as a recommended region through the recommendation region decision module 632 .

If any candidate region does not meet the preset conditions, step S44 is executed for the unmatched candidate region: the platform side 1 reduces the range of the candidate region through the region reduction module 633 (eg, excludes one or more sub-regions) to The reduced candidate regions meet the preset conditions and are set as recommended regions.

In one embodiment, the platform side 1 can calculate the purity of the target cells on each side of the candidate region that does not meet the requirements through the region reduction module 633, and exclude the sub-regions on the side with the lowest purity, so as to improve the reduced candidate region. The overall purity of the area.

In one embodiment, the platform 1 can directly set the non-compliant candidate area as a non-recommended area, or set it as a non-recommended area after the non-compliant candidate area has been reduced to a blank area.

Step S45: The platform terminal 1 determines whether any recommended area is generated.

If any recommended area is generated, step S46 is executed: the platform 1 displays the recommended area in the electronic image of the interactive interface through the rendering module 64 .

If no recommended area is generated, a warning of no recommended area can be issued directly, or step S47 is executed: the platform side 1 presents the sub-area covering the target cell in the electronic image of the interactive interface through the rendering module 64 as a user terminal 2 Manually selected references.

In one embodiment, the platform 1 can further present all sub-regions in the electronic image of the interactive interface through the rendering module 64, and provide the target cell count and the non-target cell count of each sub-region.

Please refer to FIG. 7 and FIGS. 17-19 together. FIG. 7 is a flow chart of calibrating the output size according to an embodiment of the present invention. FIG. 17 is a schematic diagram of scale measurement according to an embodiment of the present invention. 17 is a schematic diagram of the calibration output size, and FIG. 19 is a schematic diagram of the scraping guide generated based on FIG. 18 .

Compared with the scraping guide method of FIG. 4 , the scraping guide method of this embodiment further provides the calibrated output size realized by steps S50 - S54 before outputting the scraping guide.

Step S50: The user can first align the specimen in the solid glass slide with the solid area segmentation map (in Fig. 17, the solid grid paper), and confirm the corresponding scale (in Fig. 17, the length of (12, 8)) wide scale).

The region segmentation map of the entity has a corresponding size to the region segmentation map used to segment the electronic image.

Step S51 : The platform side 1 accepts the scale input operation through the image adjustment module 66 through the interactive interface to obtain the input scale.

As shown in the upper figure of Fig. 18, the user terminal 2 can modify the current scale (X, Y)=(12, 9) to the actual scale measured in step S51 (X, Y)=(12, 8), that is, input scale.

Step S52: The platform side 1 adjusts the output size of the electronic image through the image adjustment module 66, so that the current scale of the electronic image matches the input scale.

In one embodiment, the platform 1 scales the electronic image and the scraping guide in equal proportions, so that the output size of the scaled electronic image and the scraping guide conforms to the input scale.

As shown in the lower part of Fig. 18, since the electronic image is zoomed, the range and position of the clusters 70-71 and the grid area 72 of the scraping guide will also be scaled and moved accordingly, and changed to the clusters 80-81 and the grid area 82.

Step S53: The platform side 1 outputs the adjusted scraping guide through the guide generating module 65, and the output size of the scraping guide has been adjusted in step S52.

Then, the client terminal 2 may print the adjusted scraping guide based on the adjusted output size. Thereby, the printed scraping guide can fit the size of the specimen of the solid slide.

As shown in Figure 19, due to the scaling, the clusters 80-81 and grid area 82 of the adjusted scraping guide (solid grid paper with color block marks) may not align with the grid lines, but are more in line The size of the solid slide.

Step S54: The platform end 1 can align the printed scraping guide with the mark with the solid glass slide and the solid glass slide to be scraped, and scrape according to the marking.

In this way, the present invention can compensate for the morphological difference between the specimen of the solid glass slide for feature analysis and the specimen of the solid glass slide for scraping, thereby improving the accuracy of the scraping operation.

Please also refer to FIG. 8 , which is a flowchart of generating an analysis result according to an embodiment of the present invention. Compared with the scraping guide method of FIG. 4 , the scraping guide method of this embodiment further provides the analysis result calculation function realized by steps S60 - S64 . The aforementioned analysis result calculation function can calculate the detailed analysis result of the specified electronic image, which can be used as the selected sub-region to provide the data acquisition source of step S21.

Step S60 : The platform end 1 creates the result storage array 110 of the currently selected electronic image in the storage module 11 through the identification and analysis module 3 .

Step S61: The platform 1 reads each sub-image (regional image) of the electronic image one by one through the identification and analysis module 3, and performs steps S62-63 for each sub-image until the entire electronic image is processed.

It is worth mentioning that due to the high resolution of electronic images, extremely high-standard hardware equipment is required to perform identification and analysis processing on the entire electronic image at the same time. Requirements for hardware specifications.

Step S62 : The platform 1 performs target recognition and analysis on each sub-image through the recognition and analysis module 3 to identify the number of target cells in each sub-image as the analysis result of the sub-image.

In one embodiment, the identification and analysis module 3 may use a classifier (eg, the learning model 30 ) based on machine learning (eg, trained based on image features of the target cells) to identify each target cell in the sub-image.

In one embodiment, the identification and analysis module 3 can use the learning model 30 to perform model prediction to calculate the number of target cells at each location, and can be matched with the prediction algorithm of the algorithm module 31 (eg, The target cell position predicted by the convolutional network) and the nucleus position generated by the FRST algorithm are used to calculate the target cell value in each sub-region.

Step S63 : The platform end 1 stores the analysis results of each sub-image in the corresponding memory location of the result storage array 110 . After all the sub-images are analyzed, the result storage array 110 of the electronic image can be obtained, and the result storage array 110 records the relevant information of the target cells at each image position.

In one embodiment, the completed result storage array 110 may be stored in the result repository 41 .

Step S64 : The platform 1 obtains the positions of the target cells in all the electronic images based on the result storage array 110 and stores them, for example, in the result library 41 .

In this way, the present invention can generate detailed analysis information of the electronic image, which can be used as a reference for subsequent selection of the scraping area.

Please also refer to FIG. 9 , which is a flowchart of target identification and analysis according to an embodiment of the present invention. Compared with the aforementioned scraping guide method, the scraping guide method of this embodiment further provides the analysis function of the target cells realized by steps S70-S72.

Step S70: The platform 1 obtains the positions of all cells through the identification and analysis module 3, and obtains the positions of all target cells, such as automatic identification through AI.

Step S71: The platform end 1 calculates the number of cells at the location of each target cell through the identification and analysis module 3, so as to calculate the purity of each image location.

Step S72: The platform side 1 calculates the number of target cells and non-target cells in each sub-region.

In this way, the present invention can calculate the purity and quantity of target cells at each image location.

Please also refer to FIG. 10 , which is a flowchart of cell identification and analysis according to an embodiment of the present invention. Compared with the aforementioned scraping guide method, the scraping guide method of this embodiment further provides the target cell localization function achieved by steps S80-S83.

Step S80: the platform side 1 reads the color conversion matrix. The aforementioned color conversion matrix is determined based on the type of target cells to be identified and the staining method used, for example, hematoxylin and eosin (H&E) staining conversion matrix can be used for tumor cells.

Step S81 : the platform side 1 uses the color conversion matrix to decompose each sub-image into a plurality of dyed image channels of different colors.

In one embodiment, when the target cells are tumor cells and the electronic image is a hematoxylin-eosin-stained pathological image, a color conversion matrix can be used to decompose the RGB image (electronic image) into a hematoxylin-stained channel image and an eosin-stained image. Red-stained channel image.

Step S82 : the platform end 1 performs identification processing on the images of each stained image channel, so as to identify the cell position images of the cell types corresponding to the images of each stained image channel.

In one embodiment, the platform end 1 can detect the cell position image using the FRST algorithm in the hematoxylin staining channel image.

It is worth mentioning that the dye hematoxylin can stain the basophilic structure into blue-violet, which usually includes the nucleic acid-containing part, and the staining mainly stains the nucleus into blue-violet; and eosin can stain the eosinophilic structure. The pinkish, eosinophilic structure is usually composed of intracellular and intercellular proteins.

Step S83 : The platform side 1 converts each cell position image into cell coordinates (eg, image coordinates), and stores the cell coordinates and stores them in the aforementioned result storage array 110 .

Thereby, the present invention can effectively identify the specific location of each cell.

The above description is only a preferred specific example of the present invention, and therefore does not limit the scope of the present invention. Therefore, all equivalent changes made by using the content of the present invention are all included in the scope of the present invention. Chen Ming.

S10-S15: Scraping guide generation steps

Claims (20)

A scraping guide method for specimen slides, comprising: a display step including displaying an electronic image of a specimen on an interactive interface, wherein a first analysis result of a first parameter value is marked on the electronic image ; a changing step, including when accepting a parameter value changing operation, displaying the electronic image marked with a second analysis result of a changed second parameter value, wherein the first analysis result and the second analysis result are The distribution of target cells is represented with different smoothness; a segmentation step includes superimposing the electronic image with a region segmentation map to segment the electronic image into a plurality of sub-regions, wherein an output size of the region segmentation map is a size of a solid slide corresponding to the specimen; a selecting step including selecting at least a portion of the sub-region based on the analysis result; and a guiding step including marking the selected sub-region in the region segmentation to generate a scraping guide, and outputting the scraping guide based on the output size, wherein after the output scraping guide is superimposed with the solid glass slide, the mark in the scraping guide marks the solid glass slide Effect. The scraping guide method for specimen slides according to claim 1, wherein the region segmentation map includes a plurality of grid lines, and the plurality of grid lines overlapping the electronic image divide the specimen image of the electronic image into a plurality of grid areas as the plurality of sub-areas; wherein, outputting the scratching guide includes controlling a printing device to print the scratching guide as paper; wherein, the scratching guide includes lines or color blocks Mark the position where the specimen is to be scraped; wherein, the interactive interface is a graphical user interface. The scraping guide method for specimen slides as described in claim 1, before the displaying step, further comprising: A platform end provides web services to a client through the network; and after the client is authenticated, the interactive interface is displayed through a web page. The scraping guide method for specimen slides according to claim 1, wherein the changing step comprises: when the parameter value changing operation is accepted through the interactive interface, identifying the second parameter selected by the parameter value changing operation obtaining the second analysis result of the second parameter value from a database; and presenting the second analysis result on the electronic image to mark the distribution of the target cells in the electronic image. The method for scraping a specimen slide according to claim 4, wherein before acquiring the first analysis result or the second analysis result, the method comprises: acquiring a plurality of different parameter values; A corresponding filter mask, wherein each of the filter masks is used to filter out signals in different frequency ranges; a filter process is performed on the frequency domain of the electronic image based on each of the filter masks to obtain the parameters of the parameter values. an analysis result, wherein each analysis result is used to mark the image position of the target cell with different smoothness; and all the analysis results are stored in the database; wherein, the target cell is a diseased cell. The scraping guide method for specimen slides as described in claim 1, wherein the selecting step comprises: when a sub-area selection operation is accepted through the interactive interface, changing the selected sub-area based on the sub-area selection operation The state of the area is selected, unselected or not scraped; and the changed state of the sub-area is presented in the interactive interface. The scraping guide method for specimen slides according to claim 1, wherein the selecting step comprises: obtaining the purity of each of the sub-regions of the electronic image; selecting the sub-region whose purity is greater than a predetermined purity, The selected adjacent clusters of the plurality of sub-regions are set as a candidate region; and when any of the candidate regions meets a predetermined condition, the candidate region is set as a recommended region. The scraping guide method for specimen slides as claimed in claim 7, wherein the selecting step further comprises: when any of the candidate regions does not meet the predetermined condition, reducing the candidate region to meet the default condition or set as non-recommended area. The scraping guide method for specimen slides as described in claim 1, wherein before outputting the scraping guide, the method comprises: accepting a scale input operation through the interactive interface to obtain an input scale; adjusting the scale of the electronic image the output size, so that the current scale of the electronic image conforms to the input scale; wherein, the guiding step is to output the scratching guide based on the adjusted output size, so that the output scratching guide conforms to the physical glass slide the size of the specimen. The scraping guide method for specimen slides as claimed in claim 1, further comprising: establishing a result storage array of the electronic image; reading each sub-image of the electronic image, The sub-images perform a target identification and analysis to identify the number of target cells in each of the sub-images as an analysis result of the sub-images; store the analysis results of each of the sub-images in the result storage array; and based on the sub-images The result storage array stores the positions of all the target cells. The scraping guide method for specimen slides of claim 10, wherein the target identification and analysis includes using a machine learning-based classifier to identify each of the target cells in the sub-image. The scraping guide method for specimen slides as claimed in claim 11, wherein the target identification and analysis comprises: calculating the number of cells at the positions of the target cells; and calculating the target cells and non-target cells in each of the sub-regions number of target cells. The scraping guide method for specimen slides according to claim 11, wherein identifying the target cells comprises: reading a color conversion matrix; disassembling each of the sub-images into multiple images of different colors using the color conversion matrix performing an identification process on each of the stained image channels to identify the cell location images of the cell types corresponding to the stained image channels; and converting each of the cell location images into cell coordinates, and storing the cell coordinates. A scraping guide system for specimen slides, comprising: an image library for storing an electronic image of a specimen; and a platform end connected to the image library, and the platform end is used for communicating with an image library via a network The client establishes a network connection and provides an interactive interface to the client; wherein the platform is configured to display the electronic image marked with a first analysis result of a first parameter value on the interactive interface, and when accepting a parameter value changing operation via the interactive interface, displaying the electronic image marked with a second analysis result of the changed second parameter value on the interactive interface; Wherein, the platform is configured to superimpose the electronic image and a region segmentation map on the interactive interface to divide the electronic image into a plurality of sub-regions, and an output size of the region segmentation map is an entity corresponding to the specimen the size of the slide; wherein the platform end is configured to select at least a portion of the sub-region based on the analysis results, mark the selected sub-region in the region segmentation map to generate a scraping guide, and based on the output The size of the scraping guide is output; wherein, after the output scraping guide is superimposed with the physical glass slide, the mark in the scraping guide has a marking effect on the physical glass slide; wherein, the first The analysis result and the second analysis result represent the distribution of target cells with different degrees of smoothness. The scraping guide system for specimen slides as described in claim 14, wherein the platform includes a web module configured to provide web services, to connect to the client through a web protocol, and to communicate with the client verifying; an interactive control module for generating the interactive interface displayed on the client, and receiving operation and display information through the interactive interface, wherein the interactive interface is a graphical user interface; and an output control module for using The scraping guide is sent to the client, so that a printing device of the client prints the scraping guide as the output size paper. The scraping guide system for specimen slides as claimed in claim 14, wherein the interactive interface includes a parameter value changing interface; the system includes a database for storing a plurality of analysis results of a plurality of parameter values; Wherein, the platform end is connected to the database, and includes: a parameter value transformation module, which is set to identify the parameter value change operation selected by the parameter value change operation when the parameter value change operation is detected through the parameter value change interface. The second parameter value, which is obtained from the database the second analysis result of the second parameter value and presenting the second analysis result on the electronic image to mark the distribution of the target cell in the electronic image; a segmentation module configured to overlap the A plurality of grid lines of the region division map of the electronic image divide the specimen image of the electronic image into a plurality of grid regions to serve as the plurality of sub-regions; and a guide generation module configured to generate the scraping guide , wherein the scraping guide includes marking the position where the specimen is to be scraped with a line or a color block. The scraping guide system for specimen slides as described in claim 14, wherein the platform includes: a manual selection module configured to, when a sub-region selection operation is detected through the interactive interface, based on The sub-region selection operation changes the state of the selected sub-region to selected, unselected or not scraped; a purity calculation module configured to obtain the purity of each of the sub-regions of the electronic image; an external calculation module a group, which is set to select the sub-region whose purity is greater than a preset purity, and set the selected cluster of adjacent sub-regions as a candidate region; The candidate region removes the low-purity region to improve the purity of the reduced candidate region; a recommendation region decision module is configured to set the candidate region as a recommended region when any of the candidate regions meets a preset condition, and when any of the candidate areas does not meet the preset condition, execute the area reduction module on the candidate area so as to meet the preset condition or set as a non-recommended area; and a rendering module is set from the The interactive interface presents the status of each of the sub-regions in real time. The scraping guide system for specimen slides according to claim 14, wherein the interactive interface includes a scale input interface; wherein the platform includes: an image adjustment module configured to obtain an input scale of a scale input operation through the scale input interface, and adjust the output size of the electronic image so that the current scale of the electronic image conforms to the input scale; and a The guideline generating module is configured to output the scraping guideline based on the adjusted output size, so that the outputted scraping guideline conforms to the size of the specimen of the solid slide. The scraping guide system for specimen slides as described in claim 14, further comprising: a storage module, information connected to the platform end, for storing a result storage array of the electronic image; and an identification and analysis The module, which is informationally connected to the platform, is configured to read each sub-image of the electronic image, and perform a target recognition and analysis on each of the sub-images to identify the number of target cells in each of the sub-images as the sub-image storing the analysis result of each of the sub-images to the result storage array, and storing the positions of all the target cells based on the result storage array. The scraping guide system for specimen slides of claim 19, wherein the identification and analysis module comprises: a learning model configured to perform cell classification based on machine learning to identify each sub-image in the sub-image the target cell; and an algorithm module configured to count the number of cells at each location of the target cell, and to calculate the number of the target cell and non-target cell in each of the sub-regions; wherein the identification and analysis module It is also set to read a color conversion matrix, use the color conversion matrix to disassemble a plurality of dyed image channels in which each of the sub-images is a different color, and perform identification processing on each of the dyed image channels through the learning module to identify the The cell position images of the cell types corresponding to the stained image channel are converted, and the cell position images are converted into cell coordinates, and the cell coordinates are stored.

Images