CN116230214A

CN116230214A - HCC and VETC auxiliary diagnosis device and equipment

Info

Publication number: CN116230214A
Application number: CN202310504685.2A
Authority: CN
Inventors: 许迎科; 于佳辉; 马天宇
Original assignee: Binjiang Research Institute Of Zhejiang University
Current assignee: Binjiang Research Institute Of Zhejiang University
Priority date: 2023-05-08
Filing date: 2023-05-08
Publication date: 2023-06-06
Anticipated expiration: 2043-05-08
Also published as: CN116230214B

Abstract

The invention discloses an HCC and VETC auxiliary diagnosis device and equipment, wherein the equipment comprises an intelligent microscope device, a processor, a memory and a computer program; the processor when executing the computer program implements the steps of: (1) Placing liver tissue glass slides on a movable object carrying platform of an intelligent microscope in batches; (2) automatically focusing the microscope; (3) Collecting a digital image of a low-power mirror view field of a liver tissue glass slide; detecting and labeling the contour of the HCC region; (4) if no HCC region is detected, HCC is negative; if the HCC area is detected, acquiring a middle-power mirror view field image of the HCC area, detecting the contour of the VETC+ area and marking the contour; (5) Generating a diagnostic report based on the results of steps (3) and (4); positioning the movable stage to the next slide; and (6) repeating the steps (2) - (5) to perform automatic diagnosis. The invention realizes full-automatic diagnosis on HCC and VETC+ areas and provides accurate qualitative and quantitative results.

Description

HCC and VETC auxiliary diagnosis device and equipment

Technical Field

The invention relates to the field of medical treatment and artificial intelligence, in particular to an HCC and VETC auxiliary diagnosis device and equipment.

Background

Hepatocellular carcinoma (HCC) is the most common type of primary liver cancer in adults, accounting for 75% -85% and is the most common cause of death in current cirrhosis patients. HCC is the third leading cause of cancer-related death worldwide. Studies have shown that recurrence and metastasis rates within 5 years after HCC patients are as high as 50% or more, severely affecting patient survival.

Tumor-wrapped blood vessels (Vessels that Encapsulate Tumor Clusters Pattern, VETC) are a novel vascular model that promotes HCC metastasis. Studies have shown that the proportion of VETC positive (vetc+) patients account for 39% of HCC. Compared to other HCC types, vetc+ patients had a higher postoperative recurrence rate and worse prognosis.

The gold standard currently clinically diagnosed with vetc+ is the post-operative pathological outcome. The pathologist first observes the pathological section under the microscope with a low power mirror, looking for the approximate location of HCC, then manually switches the transducer to a medium power mirror or a high power mirror, and observes whether the VETC mode exists in the HCC area. For prognosis index prediction, immunohistochemical quantitative analysis is generally used.

With the development of Artificial Intelligence (AI) in the medical field, a large number of AI models are used for medical examination and auxiliary diagnosis. Such as cancer typing, cell detection, three-dimensional reconstruction. Some studies combine AI image processing techniques with microscopes to provide convenience for the diagnosis of the physician.

The pathological section is various and complex, a doctor reads a large number of slices every day, and the pathological section is time-consuming and labor-consuming and has certain subjectivity. VETC is a new special disease species found in HCC in recent years, and has not been incorporated into the gold standard of medical testing, resulting in a certain blind area for doctor diagnosis. But the postoperative recurrence rate of the VETC+ patients is higher and the prognosis is worse. The pathologist has complicated manual slide observation steps, only qualitative analysis can be performed, and the required quantitative results (such as areas, area ratios and the like of HCC and VETC < + >) cannot be obtained by observation only.

Many existing studies based on pathological images first require that the slide be made into digital sections by a scanner device and then be subjected to an auxiliary analysis by a pre-trained AI model, which is an "post-processing" auxiliary diagnostic mode. And the scanner equipment is expensive and the scanning time is long. Therefore, this auxiliary diagnostic mode affects the efficiency of clinical diagnosis.

Existing auxiliary diagnostic modes based on microscope scanning are also developed successively, but they are acquisition and analysis of slides relative to the fixed field of view of the microscope or the trajectory of the manually set stage. The conventional auxiliary diagnosis method based on microscope scanning is not suitable for detecting HCC and VETC because the microscope cannot be triggered by the detected characteristics, and has poor mobility due to the large difference between microscopic images and training data of an AI algorithm, and no related report of the batch HCC and VETC analysis technology exists at present.

Disclosure of Invention

The invention provides an HCC and VETC auxiliary diagnosis device and a characteristic triggering type intelligent microscope device for HCC and VETC auxiliary diagnosis.

The technical scheme of the invention is as follows:

an HCC and VETC aided diagnosis apparatus comprising a smart microscope device, a processor, a memory and a computer program stored in the memory and executable on the processor;

the intelligent microscope device comprises a movable carrying platform, an electric objective lens converter and an image acquisition device;

the movable carrying platform is used for placing and fixing HE-dyed liver tissue glass slides in batches, the movable carrying platform is in an XY plane, the axis of the objective lens is on a Y axis, and the movable carrying platform can move along the X axis, the Y axis and the Z axis; the hole site of the electric objective lens converter is provided with a low power lens and a medium power lens; the image acquisition device is used for acquiring images of the sample to be observed in the field of view of the intelligent microscope;

the processor is electrically connected with the movable object carrying platform of the intelligent microscope device, the electric objective lens converter and the image acquisition device respectively;

the processor, when executing the computer program, performs the steps of:

(1) Placing and fixing HE-stained liver tissue glass slides on a movable object carrying platform of an intelligent microscope in batches, and starting and initializing the intelligent microscope;

(2) Calling a focusing unit to automatically focus the microscope;

(3) In a liver tissue glass slide area, completing traversal acquisition of a low-power mirror view field image through a movable object carrying platform and an image acquisition device to obtain a digital image of the liver tissue glass slide; detecting the contour of the HCC region through an HCC analysis unit and an edge detection unit, and drawing the contour into the digital image to obtain an HCC label of the digital image;

(4) If the HCC analysis unit does not detect the HCC region, the current liver tissue slide is HCC negative;

if the HCC analysis unit detects an HCC region, the VETC analysis unit is triggered: in an HCC area, traversing and collecting the image of the field of view of the intermediate-magnification lens are completed through a movable carrying platform and an image collecting device, the outline of the VETC+ area is detected through a VETC analysis unit and an edge detection unit, and the outline is drawn into the digital image to obtain the VETC+ mark of the digital image;

(5) Invoking an analysis unit, and generating a diagnosis report of the current liver tissue glass slide according to the detection results of the steps (3) and (4); moving the movable object stage to position to the next liver tissue glass slide;

(6) Repeating the steps (2) - (5) for automatic diagnosis until manual stopping or task completion.

In the step (2), the automatic focusing of the microscope comprises coarse focusing and fine focusing; comprising the following steps:

(2-1) the image acquisition means acquiring an image of a current position of a frame, calculating sharpness of the imageS(t)；

(2-2) the movable carrying platform moves upwards in the Z-axis direction by a first coarse focusing step length, an image of the current position of a frame is acquired, and the definition of the image is calculatedS(t+1)；

(2-3) ifS(t+1)＞S(t) Repeating step (2-2) untilS(t+1)≤S(t)；

(2-4) moving the movable carrying platform in the Z-axis direction to the opposite direction for a second coarse focusing step length, collecting an image of the current position of a frame, and calculating the definition of the image;

(2-5) repeating the step (2-4) until the definition of the current image is not more than the definition of the previous image, and completing coarse focusing;

(2-6) moving the movable carrying platform in the Z-axis direction to the reverse direction for fine focusing step length, collecting an image of the current position of a frame and calculating definition every time the movable carrying platform moves, until the definition of the current image is not more than that of the previous image;

(2-7) moving the movable carrying platform to a reverse direction in the Z-axis direction by a fine focusing step length to finish fine focusing and finishing focusing;

the first coarse focusing step length, the second coarse focusing step length and the fine focusing step length are sequentially reduced.

Most preferably, the first coarse focus step is 100 μm, the second coarse focus step is 10 μm, and the fine focus step is 1 μm. Focusing error is less than or equal to 1 mu m.

The definition calculation formula of the image is as follows:

；

wherein, the liquid crystal display device comprises a liquid crystal display device,Sthe sharpness of the image is indicated,p _v is a gray value ofvIs used for the probability of occurrence of a pixel of (c).SThe larger the load floor, the closer to the focal plane.

The step (3) comprises:

(3-1) under the low power lens, carrying out fine focusing on the microscope at the current view field position of the liver tissue glass slide, and after the fine focusing is finished, automatically acquiring and storing an image of the current view field position by the image acquisition unit and recording the coordinates of the current view field position;

when the file real-time monitoring unit monitors a new image, the HCC analysis unit is called to process the image, and a segmentation mask diagram of the image is obtained;

(3-2) moving the movable carrying platform to the next view field position, repeating the step (3-1) until the traversal acquisition of the low-power mirror view field image of the liver tissue glass slide is completed, splicing the images of all view fields to obtain a digital image of the liver tissue glass slide low-power mirror view field, and splicing the segmentation mask images of all view field images to obtain a segmentation mask image of the liver tissue glass slide;

and (3-3) calling an edge detection unit, detecting the outline of the HCC region in the segmentation mask diagram of the liver tissue slide, and drawing the outline into the digital image to obtain the HCC label of the digital image.

Preferably, the file real-time monitoring unit monitors whether new images are successfully written in the designated folder or not by a watch dog library function at a certain frequency.

Preferably, the HCC analysis unit processes the image, including:

(3-1 i) correcting image background and staining based on a low-order and sparsely decomposed image correction method;

(3-1 ii) inputting the corrected image into a pre-trained HCC segmentation module for segmentation to obtain a segmentation mask map.

The HCC segmentation module is constructed based on a U-Net structure, and a transducer is integrated in the U-Net structure; and training the HCC segmentation module by taking the liver tissue full-slice image marked with the HCC region and the normal region as a training sample to obtain the pre-trained HCC segmentation module.

The step (4) comprises:

(4-1) if the HCC analysis unit does not detect the HCC region, the current liver tissue slide is HCC negative;

if the HCC analysis unit detects the HCC area, moving the movable carrying platform to the HCC area, and converting the objective lens of the microscope into a middle-power lens;

(4-2) carrying out fine focusing on the microscope at the current view field position under the intermediate lens, automatically acquiring and storing an image of the current view field position by the image acquisition unit after the fine focusing is finished, and recording the coordinates of the current view field position;

when the file real-time monitoring unit monitors a new image, a VETC analysis unit is called to process the image, and a segmentation mask diagram of the image is obtained;

(4-3) moving the movable carrying platform to the position of the next view field, repeating the step (4-2) until the traversal acquisition of the middle-power mirror view field image of the HCC area is completed, splicing the images of all view fields to obtain the digital image of the middle-power mirror view field in the HCC area, and splicing the segmentation mask images of all view field images to obtain the segmentation mask images of the HCC area;

and (4-4) calling an edge detection unit, detecting the outline of the VETC+ region in the segmentation mask diagram of the HCC region, and drawing the outline into the digital image to obtain the VETC+ mark of the digital image.

Preferably, the VETC analysis unit processes the image, comprising:

(4-2 i) correcting image background and staining based on a low-order and sparsely decomposed image correction method;

(4-2 ii) inputting the corrected image into a pre-trained VETC segmentation module for segmentation to obtain a segmentation mask map.

The VETC analysis unit has the same structure as the HCC analysis unit, and adopts liver tissue whole-slice images marked with an HCC region and a VETC+ region as training samples to train the VETC segmentation module to obtain the pre-trained VETC segmentation module.

Preferably, in step (5), the diagnosis report content includes: a annotated digital image, HCC edge image block, typical VETC + image block, quantitative index.

The ratio of HCC area to normal area was 4-7 according to the split mask map of liver tissue slide: the image block of 1 is referred to as HCC edge image block.

From the segmentation mask map of the HCC region, an image block with 70% of pixels larger than the threshold value is taken as a typical vetc+ image block.

The quantitative index comprises the area of the liver tissue region, the area of the HCC region, the area of the VETC+ region, the area ratio of the HCC region to the liver tissue region, the area ratio of the VETC+ region to the liver tissue region and the area ratio of the VETC+ region to the HCC region.

The area index is obtained by multiplying the resolution of the image acquisition device by the total number of pixels of the liver tissue region, HCC region or vetc+ region; the area ratio index is obtained by the ratio of the number of pixels of the corresponding region.

Based on the same inventive concept, the invention also provides an HCC and VETC auxiliary diagnosis device which is used in a processor, wherein the processor is electrically connected with a movable object carrying platform, an electric objective lens converter and an image acquisition device of the intelligent microscope device respectively;

comprising the following steps:

a focusing unit for automatically focusing the microscope;

the image acquisition unit of the field of view of the low-power mirror is used for carrying out traversal acquisition on the image of the field of view of the low-power mirror through the movable object carrying platform and the image acquisition device in a liver tissue glass slide area to obtain a digital image of the liver tissue glass slide;

the medium-power mirror view field image acquisition unit is triggered when the HCC analysis unit detects an HCC area, and traverses and acquires the medium-power mirror view field image in the HCC area through the movable carrying platform and the image acquisition device;

the HCC analysis unit is used for processing the low-power mirror view field image acquired by the low-power mirror view field image acquisition unit, detecting the contour of the HCC area through the HCC analysis unit and the edge detection unit, and drawing the contour into the digital image;

the VETC analysis unit is triggered when the HCC analysis unit detects the HCC area, processes the intermediate-power-mirror view field image acquired by the intermediate-power-mirror view field image acquisition unit, detects the contour of the VETC+ area through the VETC analysis unit and the edge detection unit, and draws the contour into the digital image;

the file real-time monitoring unit is used for monitoring whether the low-power mirror view field image acquisition unit or the medium-power mirror view field image acquisition unit acquires a new image in real time, and calling the HCC analysis unit or the VETC analysis unit to process the image when the new image is monitored;

the edge detection unit is matched with the HCC analysis unit or the VETC analysis unit to obtain the outline of the HCC area or the VETC+ area, and the outline is drawn into the digital image;

and an analysis unit for generating a diagnosis report of the liver tissue slide according to the results of the HCC analysis unit, the VETC analysis unit and the edge detection unit.

Compared with the prior art, the invention has the beneficial effects that:

(1) The invention combines the AI analysis model with the characteristic triggering type intelligent microscope system, realizes full-automatic diagnosis on HCC and VETC+ areas, provides accurate qualitative and quantitative results, and assists doctors in diagnosis.

(2) The device can automatically detect a plurality of glass slides in batches, realize full-automatic batch diagnosis and improve diagnosis efficiency.

(3) The characteristic triggering type system can simulate the diagnosis step of doctors, namely, the HCC area is searched under the low-power mirror, and then the image is switched to the medium-power mirror to observe the VETC, and the image is not processed after scanning, so that the characteristic triggering type system is directly used for preprocessing a glass slide, is more suitable for clinical auxiliary diagnosis, and has higher speed and higher accuracy compared with other microscopes for scanning the whole glass.

Drawings

FIG. 1 is a schematic diagram of the structure of an HCC and VETC auxiliary diagnostic device;

fig. 2 is a schematic diagram of the workflow of HCC and VETC auxiliary diagnostic devices.

Detailed Description

The invention will be described in further detail below with reference to the drawings and examples, it being noted that the examples described below are intended to facilitate an understanding of the invention and are not intended to limit the invention in any way.

In order to solve the difficulties in the prior art, the invention provides a characteristic triggering type intelligent microscope device and a characteristic triggering type intelligent microscope device for auxiliary diagnosis of hepatocellular carcinoma (HCC) and tumor wrapping type blood Vessel (VETC). The invention realizes the collection and analysis of hematoxylin-eosin (HE) staining glass slide through the cooperation of an Artificial Intelligence (AI) model and an intelligent microscope system, provides diagnostic reports for pathologists in a short time, and comprises typical image blocks of HCC and (liver cancer) tumor wrapping type vascular positive (VETC+), qualitative marks of HCC and VETC+ areas and quantitative results. The invention can effectively relieve the diagnosis pressure and subjectivity of pathologists and provide faster and more accurate diagnosis. The device comprises a characteristic triggering type intelligent microscope system, wherein the characteristic triggering type intelligent microscope system is fed back to a microscope according to an analysis result of an AI unit, and the microscope is triggered to further act; the device comprises an AI analysis unit, a microscope system and a computer device.

The invention collects the glass slide by a microscope, carries out full-automatic qualitative and quantitative analysis on HCC and VETC, and generates a pathology report. The invention provides HCC and VETC qualitative and quantitative results through a pretreatment mode, assists pathologists in clinical diagnosis, and improves diagnosis efficiency and accuracy of pathologists. The pretreatment refers to directly analyzing the glass slide without a scanner, and the obtained spliced microscope image can replace a scanner image to be used for training a model and the like.

By AI analysis of the resulting features, the microscope (simulated doctor) is adaptively triggered for further analysis without manual manipulation. The invention also overcomes the problems of data difference and migration through the corresponding algorithm of the AI unit.

The HCC and VETC auxiliary diagnostic apparatus (AI analysis unit) and device are shown in fig. 1, and the HCC and VETC auxiliary diagnostic flow based on the feature triggered intelligent microscope is shown in fig. 2, comprising the steps of:

step S101, placing and fixing HE-stained liver tissue glass slides on a movable carrying platform in batches, starting a microscope system, and automatically initializing the system.

The initialization includes: serial communication, movable carrying platform positioning, illumination system brightness setting, objective hole position conversion and camera parameter setting.

Serial communication is to connect the microscope system to a computer bus through a USB serial port for communication and control.

In one embodiment, the initial position of the movable stage in the XY plane is the midpoint coordinate position of the first slide, and the initial Z-axis position is a fixed position about 1mm from the focal point; the illumination system brightness is set to 20% maximum brightness; the position of the objective lens is converted into a low power lens; the camera parameters were set to auto exposure, auto white balance, resolution 15361024, pixel type RGB32.

Step S102, calling a focusing unit to perform automatic focusing, wherein the automatic focusing comprises coarse focusing and fine focusing, and the focusing error is smaller than 1um.

The method comprises the following steps: introducing an image sharpness evaluation function

2 coarse focus and 1 fine focus are performed on the Z axis, whereinp _v Is a gray value ofvIs used to determine the probability of the occurrence of a pixel,Sindicating the sharpness of the image.SThe larger the load floor, the closer to the focal plane. In one embodiment, the first coarse focus step size is 100um, the second coarse focus step size is 10um, and the fine focus step size is 1um. The method comprises the following steps: a. the image acquisition device acquires an image of the current position of a frame, calls an image definition evaluation function, and calculates the definition of the image asS(t) The method comprises the steps of carrying out a first treatment on the surface of the b. The movable carrying platform moves upwards in the Z-axis direction by a first coarse focusing step length, the image acquisition device acquires an image of the current position of a frame, an image definition evaluation function is called, and the image definition of the plane is calculated asS(t+1); c. if it isS(t+1)＞S(t) Repeating step b untilS(t+1)≤S(t) The method comprises the steps of carrying out a first treatment on the surface of the d. The movable carrying platform moves a second coarse focusing step length in the opposite direction, the image acquisition device acquires an image of the current position of a frame, an image definition evaluation function is called, and the image definition of the plane is calculated; e. repeating the step d until the image definition of the current plane is not more than the previous image; f. the movable carrying platform moves a fine focusing step length in the opposite direction, and the definition is calculated once every time of movement until the definition of the image of the current plane is not more than the previous image; g. the movable carrying platform moves a fine focusing step length in the opposite direction, focusing is finished, and focusing error is less than or equal to 1 mu m.

Step S103, starting a file real-time monitoring unit, and positioning the movable object carrying platform to the initial position of the first glass slide.

In one embodiment, the file real-time monitoring unit monitors whether new image files are successfully written in a specified folder by a watch dog library function at a frequency of 2 times/second for real-time processing of images. The designated folder is a folder storing the field-of-view image acquired by the image acquisition device.

Step S104, fine focusing is executed on each view field position, and after the fine focusing is finished, the image acquisition unit automatically acquires and stores the image of the view field position and records the current coordinates. The real-time monitoring unit monitors the new image and calls the HCC analysis unit to process in real time.

The HCC analysis unit includes: style correction module, HCC segmentation module. The method comprises the following steps:

a. carrying out style correction on the monitored new image;

b. inputting the image to a pre-trained HCC segmentation module for segmentation;

c. outputting the segmented mask images, and splicing the mask images in sequence.

The style correction module is an image correction method based on low-order and sparse decomposition for correcting image background and staining so that the microscope image adapts to the style of the training data.

And the HCC segmentation module is used for carrying out HCC region segmentation on the collected microscopic images. In one embodiment, the steps are:

a. an image dataset was made using local hospital collected liver tissue whole-section images containing HCC and vetc+ regions, precisely noted by the pathologist, approximately 100.

b. According to 70%:10%:20% is divided into a training set, a verification set and a test set;

c. all data were slice preprocessed to train the U-Net based HCC segmentation module. The HCC segmentation module based on U-Net integrates a transducer module in the U-Net structure and performs fine tuning, and the HCC segmentation module comprises: a transducer module is embedded between each pair of encoder-decoders of the U-Net, setting the initial number of channels of the U-Net to 32. The training data of the HCC segmentation module uses the normal tissue region and HCC region of the image dataset described above. The mIoU of the trained HCC segmentation module is 0.92;

d. embedding the trained HCC segmentation model into an AI analysis unit of the system for segmenting the microscope image in real time;

e. the file real-time monitoring unit monitors a new microscope image, inputs the image into the trained HCC segmentation model, outputs a segmentation mask map, and stores the segmentation mask map in a computer memory. According to the mask diagram, the area ratio of the HCC area to the normal area is 0.4-0.7: the image blocks between 1 are displayed on a computer output device as HCC edge images for assisting a doctor in quick judgment.

Step S105, the movable carrying platform moves a first step length to the next position along the Y-axis direction, and the step S104 is repeated until the acquisition in the Y-axis direction is completed. The first step is the actual distance of the high slide of the low power mirror field image.

Step S106, the movable carrying platform moves to the initial position of the second row along the X, Y axis, and the steps S104 and S105 are repeated. Until the whole glass slide is collected, a digital image of the glass slide and a segmentation mask image are obtained, and the digital image and the segmentation mask image are stored in a computer memory.

Step S107, an OpenCV edge detection unit is called, the outline of the HCC area is detected in the mask diagram, and the outline is drawn in the digital image, so that the HCC label of the digital image is obtained. And acquiring the rectangle where the outline is located by using the same edge detection unit, and recording coordinates.

Step S108, triggering the VETC analysis unit according to the HCC detection result. The method comprises the following steps:

a. calculating corresponding coordinates of the rectangular position in the image on the glass slide;

b. the movable carrying platform automatically moves to the coordinate position in the a;

c. the objective lens converter automatically converts to a middle power lens for collecting and processing microscope images in a small view field;

d. and calling a fine focusing algorithm to focus.

Step S109, fine focusing is executed on each view field position, and after the fine focusing is finished, an image acquisition unit automatically acquires and stores an image of the view field position and records the current coordinates. And the real-time monitoring unit monitors the new image and calls the VETC analysis unit to perform real-time processing.

The VETC analysis unit includes: a style correction module and a VETC segmentation module. The method comprises the following steps:

a. carrying out style correction on the monitored new image;

b. inputting the image to a pretrained VETC segmentation module for segmentation;

The VETC segmentation module is used for carrying out VETC+ region segmentation on the collected microscopic images. In one embodiment, the module is similar to the HCC segmentation module described above. The training data of this module, in contrast, uses the HCC region and the vetc+ region of the image dataset described above. The mIoU of the trained VETC segmentation module is 0.86. And in the pixel fraction output by the VETC segmentation module, an image block with 70% of pixels larger than a threshold value is taken as a typical VETC+ image block and is displayed on computer output equipment for assisting a doctor in quick judgment. The threshold is to evaluate the confidence of the VETC + region, set to 0.95.

Step S110, the movable carrying platform moves a second step length to the next position along the Y-axis direction, and the step S109 is repeated until the acquisition in the Y-axis direction is completed. The second step is the actual distance of the high slide of the intermediate-power view field image.

Step S111, the movable loading platform moves to the initial position of the second row along the X, Y axis, and steps S109 and S110 are repeated. Until the acquisition of the HCC region is completed, a segmentation mask map of the HCC region is obtained and saved to a computer memory.

Step S112, an OpenCV edge detection unit is called, the contour of the VETC+ region is detected in the segmentation mask diagram of the HCC region, the corresponding position of the contour in the digital image of step S106 is calculated, and the contour is drawn, so that the VETC+ mark of the digital image is obtained.

Step S113, a call analysis unit, configured to generate a diagnostic report and contents in the report, including: a annotated digital image, an HCC edge image block, a typical VETC + image block, a quantitative indicator, a diagnostic report.

Specifically, the digital image of the label is obtained by step S107 and step S112; the HCC edge image block is obtained by step S104; a typical vetc+ image block is obtained by step S109; the quantitative index comprises the area of a tissue region, the area of an HCC, the area of a VETC+, the area ratio of the HCC to the tissue region, the area ratio of the VETC+ to the tissue region and the area ratio of the VETC+ to the HCC, and is used for assisting a pathologist in making a treatment scheme and performing prognosis analysis, wherein the area index is obtained by multiplying the resolution of an image acquisition device by the total pixel number of the HCC or the VETC+, and the area ratio index is obtained by the quotient of the pixel number of the corresponding region; the diagnostic report consists of all of the foregoing for reference and confirmation by the pathologist.

Step S114, the movable carrying platform moves to the initial position of the second slide, and the steps S101-S113 are repeated until the system is stopped manually or all slides are analyzed automatically.

A feature-triggered intelligent microscopy apparatus for HCC and VETC-aided diagnosis, comprising:

the image acquisition device, the movable carrying platform, the electric objective lens converter and the illumination system form a main controllable device of the microscope system, and the controllable device is connected with a computer bus.

The image acquisition device is connected to the computer through a USB serial port and used for rapidly acquiring liver tissue slice images in a visual field. In one embodiment, the maximum resolution of the image capture device is 0.5 um/pixel, supporting automatic exposure.

The movable carrying platform is connected to the microscope control bus and used for placing liver tissue slice glass slides in batches, and is a movable carrier for realizing automatic focusing and automatic acquisition functions. In one embodiment, the movable object carrying platform is an electric platform which can move along the X, Y, Z three-axis direction, wherein the precision of the Z-axis direction is 0.1um.

And the electric objective lens converter is connected to the microscope control bus and is used for automatically switching the low, medium or high power objective lens. In one embodiment, the transducer aperture is equipped with a low power mirror (4 x), a medium power mirror (10 x, 20 x) and a high power mirror (40 x).

And the illumination system is connected to the microscope control bus and is used for adjusting the brightness of the field of view.

Computer means comprising a processor, a memory and a computer program stored in the memory and executable on the processor, the processor implementing the above steps when executing the computer program.

The foregoing embodiments have described the technical solutions and advantages of the present invention in detail, and it should be understood that the foregoing embodiments are merely illustrative of the present invention and are not intended to limit the invention, and any modifications, additions, substitutions and the like that fall within the principles of the present invention should be included in the scope of the invention.

Claims

1. An HCC and VETC aided diagnosis apparatus comprising a smart microscope device, a processor, a memory and a computer program stored in the memory and executable on the processor;

the processor, when executing the computer program, performs the steps of:

(2) Calling a focusing unit to automatically focus the microscope;

2. The HCC and VETC auxiliary diagnostic device according to claim 1, wherein in step (2), the auto-focusing of the microscope includes coarse focusing and fine focusing; comprising the following steps:

(2-3) ifS(t+1)＞S(t) Repeating step (2-2) untilS(t+1)≤S(t)；

3. The HCC and VETC auxiliary diagnostic device according to claim 1, wherein the step (3) includes:

4. The HCC and VETC auxiliary diagnostic device according to claim 3, wherein the HCC analysis unit processes the image, comprising:

5. The HCC and VETC aided diagnosis apparatus of claim 4, wherein the HCC segmentation module is constructed based on a U-Net structure, and integrates a transducer in the U-Net structure; and training the HCC segmentation module by taking the liver tissue full-slice image marked with the HCC region and the normal region as a training sample to obtain the pre-trained HCC segmentation module.

6. The HCC and VETC auxiliary diagnostic device according to claim 1, wherein the step (4) includes:

(4-1) if the HCC analysis unit detects the HCC region, moving the movable stage to the HCC region, and converting the objective lens of the microscope into a intermediate power lens;

7. The HCC and VETC auxiliary diagnostic device according to claim 6, wherein the VETC analysis unit processes the image, comprising:

(4-2 ii) inputting the corrected image into a pre-trained VETC segmentation module for segmentation to obtain a segmentation mask map;

the VETC segmentation module is constructed based on a U-Net structure, and a transducer is integrated in the U-Net structure; and training the VETC segmentation module by taking the liver tissue whole-slice image marked with the HCC region and the VETC+ region as a training sample to obtain a pre-trained VETC segmentation module.

8. The HCC and VETC auxiliary diagnostic device according to claim 1, wherein in the step (5), the diagnosis report contents include: labeling digital images, HCC edge image blocks, typical VETC+ image blocks and quantitative indexes;

9. The HCC and VETC auxiliary diagnosis device is characterized by being used in a processor, wherein the processor is electrically connected with a movable carrying platform of an intelligent microscope device, an electric objective lens converter and an image acquisition device respectively;

comprising the following steps:

a focusing unit for automatically focusing the microscope;