WO2021017272A1

WO2021017272A1 - Pathology image annotation method and device, computer apparatus, and storage medium

Info

Publication number: WO2021017272A1
Application number: PCT/CN2019/116925
Authority: WO
Inventors: 杨光
Original assignee: 平安科技（深圳）有限公司
Priority date: 2019-08-01
Filing date: 2019-11-10
Publication date: 2021-02-04
Also published as: CN110675940A

Abstract

A pathology image annotation method and device, a computer apparatus, and a storage medium, relating to the technical field of artificial intelligence. The pathology image annotation method comprises: identifying an image format of an acquired pathology image, and determining whether the pathology image is in a target format or in a conventional format (S2); if the image format is the conventional format, determining that the pathology image corresponding to the conventional format is a region-of-interest image (S3); if the image format is the target format, performing downsampling processing on the pathology image corresponding to the target format, and acquiring a region-of-interest image (S4); receiving an annotation request of an operating user, and acquiring, from a preconfigured type library, an application type corresponding to the annotation request (S5); and matching the application type with description information in a preconfigured annotation library, and selecting an annotation policy corresponding to the successfully matched description information so as to annotate the region-of-interest image (S6). A pathology image is annotated accurately, thereby improving the efficiency of a user when annotating a pathology image.

Description

Pathological image labeling method, device, computer equipment and storage medium

This application is based on the Chinese invention patent application filed on August 1, 2019 with the application number 201910708215.1 and titled "Pathological Image Marking Method, Device, Computer Equipment and Storage Medium", and claims its priority.

Technical field

This application relates to the field of artificial intelligence technology, and in particular to a pathological image labeling method, device, computer equipment and storage medium.

Background technique

At present, the labeling tools on the market, such as LabelMe, etc., only read and label images in common formats, and cannot support the analysis and reading of pathological images, such as ndpi, tif, oct and other image formats; the inventor realized that because The data processed by many AI models is pathological data, which cannot be supported by traditional labeling tools and cannot adapt to diversified labeling requirements. As a result, users cannot label pathological images normally, which affects the work efficiency of users.

Summary of the invention

The embodiments of the present application provide a pathological image labeling method, device, computer equipment, and storage medium to solve the problem that pathological images cannot be accurately labelled, which affects the work efficiency of users.

A method for marking pathological images includes:

Obtain the pathological image to be labeled from the preset image library;

Perform image format recognition on the pathological image, and determine whether it is a pathological image in a target format or a pathological image in a conventional format;

If the image format is the conventional format, determining the pathological image corresponding to the conventional format as the region of interest image;

If the image format is the target format, perform down-sampling processing on the pathological image corresponding to the target format, and use the down-sampled pathological image as the region of interest image;

Receiving a labeling request from an operating user, and obtaining the application type corresponding to the labeling request from a preset type library;

The application type is matched with the description information in the preset annotation library, and the annotation strategy corresponding to the description information that is successfully matched is selected to annotate the region of interest image, wherein the preset annotation library contains the description Information and the labeling strategy corresponding to the description information.

A pathological image marking device, including:

The first acquisition module is used to acquire the pathological image to be labeled from the preset image library;

The recognition module is used to recognize the image format of the pathological image, and determine whether it is a pathological image in a target format or a pathological image in a conventional format;

The conventional format module is configured to determine the pathological image corresponding to the conventional format as the region of interest image if the image format is the conventional format;

The target format module is configured to, if the image format is the target format, perform down-sampling processing on the pathological image corresponding to the target format, and use the down-sampled pathological image as the region of interest image;

The second obtaining module is configured to receive the labeling request of the operating user, and obtain the application type corresponding to the labeling request from the preset type library;

An annotation module, configured to match the application type with the description information in a preset annotation library, and select an annotation strategy corresponding to the description information that is successfully matched to annotate the region of interest image, wherein the preset annotation library Contains the description information and the labeling strategy corresponding to the description information.

A computer device comprising a memory, a processor, and computer readable instructions stored in the memory and capable of running on the processor, and the processor implements the above pathological image labeling method when the processor executes the computer readable instructions A step of.

A non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the above pathological image labeling method is implemented step.

The details of one or more embodiments of the present application are presented in the following drawings and description, and other features and advantages of the present application will become apparent from the description, drawings and claims.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

FIG. 1 is a flowchart of a pathological image labeling method provided by an embodiment of the present application;

FIG. 2 is a flowchart of step S2 in FIG. 1;

FIG. 3 is a flowchart of step S4 in FIG. 1;

Figure 4 is a flowchart of step S42 in Figure 3;

FIG. 5 is a flowchart of step S422 in FIG. 4;

FIG. 6 is a flowchart of graying a pathological image in a pathological image labeling method provided by an embodiment of the present application;

FIG. 7 is a flowchart of storing the annotation information of the pathological image in the pathological image annotation method provided by the embodiment of the present application;

Fig. 8 is a schematic diagram of a pathological image labeling device provided by an embodiment of the present application;

Fig. 9 is a basic structural block diagram of a computer device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by a person of ordinary skill in the art without creative work shall fall within the protection scope of this application.

The pathological image labeling method provided in this application is applied to the server, and the server can be implemented by an independent server or a server cluster composed of multiple servers. In an embodiment, as shown in FIG. 1, a method for marking pathological images is provided, which includes the following steps:

S1: Obtain the pathological image to be labeled from the preset image library.

In the embodiment of the present application, by detecting the preset image library, when it is detected that there is a pathological image in the preset image library, the pathological image is directly acquired. Among them, the preset image library refers to a database dedicated to storing pathological images.

S2: Perform image format recognition on the pathological image, and determine whether it is the pathological image in the target format or the pathological image in the conventional format.

Specifically, the file extension corresponding to the pathological image is obtained, and the image format is identified by matching the file extension with the preset extension, and then it is determined whether the image format of the pathological image is the target format or the regular format.

The file extension is also called the file extension, which is a mechanism used by the operating system to mark the file type.

The default extension refers to the file extension set according to the user's needs, which can be the file extension of ndpi.

S3: If the image format is a conventional format, the pathological image corresponding to the conventional format is determined as the region of interest image.

In the embodiment of the present application, the image format of the pathological image is recognized according to step S2, and when the image format is recognized as the conventional format, the pathological image corresponding to the conventional format is determined as the region of interest image.

It should be noted that the conventional format can specifically refer to the image format of oct and tif, and can also be set according to the actual needs of the user, and there is no limitation here.

S4: If the image format is the target format, perform down-sampling processing on the pathological image corresponding to the target format, and use the down-sampling processed pathological image as the region of interest image.

In the embodiment of the present application, the image format of the pathological image is recognized according to step S2. When the image format is recognized as the target format, the pathological image corresponding to the target format is down-sampled, and the down-sampling processed The pathological image is determined as the region of interest image.

Preferably, in this embodiment, the target format mainly refers to an image format of ndpi.

S5: Receive the labeling request from the operating user, and obtain the application type corresponding to the labeling request from the preset type library.

Specifically, when it is detected that the user clicks on the annotation request corresponding to the pathological image annotation from the client, the annotation request corresponding to the user is obtained, and the application type corresponding to the annotation request is obtained from the preset type library, where the preset type The library refers to a database specially used to store a labeling request and the application type corresponding to the labeling request.

It should be noted that the application types corresponding to the labeling request are mainly divided into three types, namely, classification application, detection application, and segmentation application. The corresponding labeling requests are "category", "detection", and "segmentation".

Classification: refers to the classification of the input image category in a certain sense.

Detection: refers to drawing a rectangular frame, circular frame, etc. on a certain area of an input image.

Segmentation: refers to drawing a free pen closed curve to a certain area of an input image.

S6: Match the application type with the description information in the preset annotation library, and select the annotation strategy corresponding to the successfully matched description information to annotate the image of the region of interest. The preset annotation library contains the description information and the corresponding description information. Labeling strategy.

Specifically, the labeling request corresponding to the application type obtained in step S5 is matched with the description information in the preset labeling library. When the labeling request is the same as the description information, it indicates that the matching is successful, and the labeling strategy pair corresponding to the description information is selected Annotate the image of the region of interest.

It should be noted that the description information contained in the preset annotation library is "classification", "detection" and "segmentation" respectively.

Among them, the labeling strategy corresponding to the description information of "category" is to label the image of the region of interest based on the image format. If the image format is a regular format, the entire region of interest image is directly labeled with preset label information; if the image format is the target Format, it is detected that the user clicks on the relevant area on the image of the area of interest, and the area selected by the user is marked with preset label information.

The preset label information can be a noun or a symbol, and its specific settings can be set according to the actual needs of the user.

It should be noted that when labeling the region of interest image of the classification application type, the region of interest image can be labeled with multiple same preset label information, or the region of interest image can be labeled with multiple different preset label information .

The labeling strategy corresponding to the description information of "detection" is to obtain a preselection box, and mark the preselection box for the area selected by the user based on the detection that the user clicks on the relevant area on the image of the region of interest on the client. Among them, the preselection box may be a rectangular box or a circular box, and there is no limitation here.

The labeling strategy corresponding to "segmentation" in the description information is to obtain the preset free pen tool. According to the detection that the user clicks on the relevant area on the image of the region of interest on the client, the coordinates of the area clicked by the user are marked freely curve. Among them, the preset free pen tool mainly refers to a tool used to mark an image with a closed curve.

Further, in the process of labeling the image, the scale, setWorldMatrix, and translate functions in the interface of the object QPainter in the Qt framework can also be used to implement the translation function, rotation function, and zoom function of the image, respectively.

In this embodiment, by recognizing the image format of the pathological image, determining the image of the region of interest according to the image format, receiving the labeling request of the operating user, and obtaining the application type corresponding to the labeling request, and selecting the corresponding application type from the preset labeling library Annotation strategy of the image of the region of interest is annotated. In this way, accurate labeling of pathological images can be realized. By recognizing the image format, it can effectively perform down-sampling processing for pathological images of different formats, avoiding the subsequent inability to directly accurately label pathological images in some image formats, and improving the applicability of labeling pathological images , Selecting the corresponding labeling strategy according to the application type can ensure the accuracy of labeling pathological images, thereby improving the working efficiency of operating users in labeling pathological images.

In one embodiment, as shown in Fig. 2, in step S2, the image format identification of the pathological image, and determining whether it is the pathological image in the target format or the pathological image in the conventional format includes the following steps:

S21: Obtain the file extension corresponding to the pathological image from the preset type table.

In the embodiment of the present application, the file extension corresponding to the pathological image is directly obtained from the preset type table. Among them, the preset type table refers to a data table specially used for storing file extensions corresponding to pathological images.

S22: Compare the file extension with the preset extension.

Specifically, the file extension acquired in step S21 is compared with the preset extension.

S23: If the file extension is different from the preset extension, the image format corresponding to the pathological image is determined to be the conventional format.

Specifically, according to the method of comparing the file extension with the preset extension according to step S22, if the comparison result is that the file extension is different from the preset extension, the image format of the pathological image corresponding to the file extension is determined to be the conventional format.

For example, the file extension corresponding to pathology image A is oct. If the default extension is ndpi, compare the file extension oct with the preset extension ndpi. Since oct and ndpi are different, the image format of pathology image A is determined It is a regular format.

S24: If the file extension is the same as the preset extension, the image format corresponding to the pathological image is determined as the target format.

Specifically, according to the method of comparing the file extension with the preset extension in step S22, if the comparison result is that the file extension is the same as the preset extension, the image format of the pathological image corresponding to the file extension is determined as the target format.

For example, the file extension corresponding to pathology image B is ndpi. If the preset extension name is ndpi, compare the file extension ndpi with the preset extension ndpi. Since both are ndpi, the image format of pathology image B is determined as Target format.

In this embodiment, by acquiring the file extension corresponding to the pathological image, and comparing the file extension with the preset extension, the image format of the conventional format or the target format is determined. In this way, accurate recognition of the image format is realized, and the accuracy of subsequent determination of the region of interest image for the image format is improved.

In one embodiment, as shown in FIG. 3, in step S4, that is, if the image format is the target format, the pathological image corresponding to the target format is down-sampled, and the down-sampled pathological image is used as the region of interest image including The following steps:

S41: If the image format is the target format, perform down-sampling processing on the pathological image corresponding to the target format to obtain a thumbnail image.

In the embodiment of this application, if the image format is the target format, the pathological image corresponding to the target format is imported into the preset down-sampling library, and the pathological image is down-sampled according to the preset down-sampling coefficient to obtain the processing After the thumbnail image. Among them, the preset down-sampling library refers to a database specially used for down-sampling processing of pathological images. The preset downsampling factor is a constant set according to the actual needs of the user, and the value range of the constant is 0-9.

It should be noted that since the storage space of the pathological image in the target format is relatively large, usually a few G, the entire pathological image cannot be loaded at one time, so it is necessary to down-sample the pathological image in the target format so that the subsequent normal Extraction of regions of interest in pathological images.

Down-sampling processing refers to an N*M image, setting the down-sampling coefficient k, and taking one pixel every k pixels in each row and every column of the N*M image to form a new image.

If 10 down-sampling levels are preset for pathological images, the down-sampling coefficient is set to 0-9. When the down-sampling coefficient is 0, it means that the pathological image will not be down-sampled. When the down-sampling coefficient is 9, it means that The pathological image undergoes maximum down-sampling processing.

It should be noted that the larger the down-sampling coefficient k, the larger the corresponding down-sampling rate, and the smaller the size of the image.

For example, if the down-sampling coefficient is set to 1, when the pathological image is down-sampled, for each row and each column of the pathological image, one pixel is taken out as the target pixel at every interval of 1 pixel, and finally according to Each target pixel constitutes a thumbnail image corresponding to the original pathological image.

S42: Extract the region of interest from the thumbnail image to obtain an image of the region of interest.

Specifically, the thumbnail image is imported into the preset processing library to extract the region of interest, and the region of interest image after the region of interest extraction is obtained. Among them, the preset processing library refers to a processing library specifically used to extract regions of interest from thumbnail images.

In this embodiment, the thumbnail image is obtained by down-sampling the pathological image in the target format, and the area of interest is extracted from the thumbnail image to obtain the area of interest image. In this way, accurate acquisition of the region of interest image corresponding to the target format is realized, and the accuracy of subsequent labeling of the region of interest image is ensured.

In one embodiment, as shown in FIG. 4, in step S42, the extraction of the region of interest on the thumbnail image to obtain the region of interest image includes the following steps:

S421: Obtain preset parameters of the region of interest.

In the embodiment of the present application, the preset region of interest parameter refers to the parameter corresponding to the partial region image selected by the user from the thumbnail image. By directly acquiring the preset region of interest parameters from the parameter database. Among them, the parameter database refers to a database specially used for storing preset parameters of the region of interest.

The preset region of interest parameters include the coordinate points of the region of interest and the corresponding length and width of the region of interest.

For example, you can read a partial area image selected by the user by calling the reading interface in the OpenSlide library, and specify the X, Y coordinates and width and height of the starting point in the thumbnail image to obtain the area of interest parameters.

S422: Generate an image mask according to preset parameters of the region of interest.

In the embodiments of the present application, the image mask refers to the area where the selected image, figure or object, or the image to be processed is shielded to control the image processing. Import the preset region of interest parameters into the preset processing library for image mask generation processing, and obtain the image mask corresponding to the preset region of interest parameters.

Among them, the preset processing library refers to a database specifically used for image mask generation processing.

S423: Perform an AND operation on the image mask and the thumbnail image to obtain an image of the region of interest.

Specifically, the image mask obtained in the root step S422 is multiplied by the pixel value of each pixel in the image mask by the pixel value of the pixel corresponding to the same position in the thumbnail image, and each pixel is obtained according to the multiplication result. The pixel value of each pixel is obtained, and the new image is taken as the image of the region of interest.

For example, there are 9 pixels in a thumbnail image, and the coordinates of the corresponding pixels are A(0,0), A1(0,1), A2(0,2), B(1,0), B1 (1,1), B2(1,2), C(2,0), C1(2,1) and C2(2,2), the corresponding pixel values are 90, 0, 23, 90, 50, 22, 23, 255 and 89, where the pixels of the region of interest are A, A1, B, and B1 respectively; there are 9 pixels in the same position as the thumbnail image in the image mask, and the coordinates of the corresponding pixels are respectively For Q(0,0), Q1(0,1), Q2(0,2), W(1,0), W1(1,1), W2(1,2), E(2,0), E1(2,1) and E2(2,2), since the pixels of the region of interest in the thumbnail image are A, A1, B, and B1 respectively, the corresponding pixels in the same position in the image mask are respectively Q , Q1, W, and W1, so the pixel values corresponding to Q, Q1, W, and W1 are all 1, and the pixel values corresponding to Q2, W2, E, E1, and E2 are all 0. Change the value of each pixel in the image mask The pixel value is multiplied by the pixel value of the corresponding pixel at the same position in the thumbnail image to obtain the coordinates (0, 0), (0, 1), (0, 2), (1, 0), (1, 1) ), (1, 2), (2, 0), (2, 1) and (2, 2) correspond to pixel values of 90, 0, 0, 90, 50, 0, 0, 0 and 0, respectively. The image of the pixel value corresponding to this pixel is the region of interest image.

In this embodiment, the image mask is generated according to preset parameters of the region of interest, and the image mask and the thumbnail image are used for calculation to obtain the region of interest image. In this way, accurate acquisition of the image of the region of interest is realized, and the accuracy of subsequent labeling of the image of the region of interest is ensured.

In an embodiment, as shown in FIG. 5, in step S422, generating an image mask according to preset region of interest parameters includes the following steps:

S4221: Determine the coordinate parameters of the pixel points of the region of interest and the region of non-interest in the thumbnail image according to the preset region of interest parameters, where the thumbnail image includes the region of interest and the region of non-interest.

In the embodiment of the present application, the thumbnail image is composed of pixels, and each pixel point corresponds to a coordinate point. Since the preset region of interest parameters include the coordinate points of the region of interest, the thumbnail image is compared with the preset interest The area composed of coordinate points with the same area parameters is determined as the area of interest of the thumbnail image, and the coordinate points in the area of interest of the thumbnail image are determined, that is, the coordinate parameters of the pixels in the area of interest are determined; The area composed of coordinate points different from the preset area of interest parameters is determined as the non-interest area of the thumbnail image, and the coordinate points in the non-interest area in the thumbnail image are determined, that is, the pixel points of the non-interest area are determined The coordinate parameters.

For example, a thumbnail image is composed of 6 pixels A, B, C, D, E, and F, and the coordinate points corresponding to each pixel are (0, 0), (0, 1), (0, 2). ), (1, 0), (1, 1), (1, 2), the coordinate points of the region of interest included in the preset region of interest parameter are (0, 0), (0, 1), (0 , 2), the area composed of the same coordinate points as the preset area of interest parameters in the thumbnail image is determined as the area of interest of the thumbnail image, that is, the area of interest in the thumbnail image is composed of pixel points A, B, C The corresponding coordinate parameters are (0, 0), (0, 1), (0, 2); the area composed of coordinate points different from the preset area of interest parameters in the thumbnail image is determined as the thumbnail image The non-interest area, that is, the area of interest in the thumbnail image is composed of pixels D, E, and F, and the corresponding coordinate parameters are (1, 0), (1, 1), (1,2).

S4222: Generate an image template with the same coordinate parameters through a preset port, where the image template includes target pixels in the region of interest and ordinary pixels in the non-interest region.

In the embodiment of the present application, based on the coordinate parameters of the pixel points of the region of interest and the non-interest region of the thumbnail image in step S4221, an image template with the same coordinate parameters is generated through a preset port, and the image template is Pixels with the same coordinate parameters as the region of interest are determined as target pixels, and pixels in the image template with the same coordinate parameters as the non-interest region are determined as ordinary pixels.

Among them, the preset port refers to a processing port dedicated to generating an image template.

S4223: Set the pixel values of the target pixel and the ordinary pixel in the image template to the preset target value and the preset ordinary value, respectively, and use the set image template as the image mask.

Specifically, according to step S4222, the target pixel and the ordinary pixel in the image template are obtained, the pixel value of the target pixel is set to the preset target value, the pixel value of the ordinary pixel is set to the preset ordinary value, and the image After the pixel values of all target pixels and common pixels in the template are set, the image template is determined as an image mask.

Among them, the preset target value refers to a value set according to user requirements for highlighting the color of the target pixel with respect to the color of the common pixel, for example, it may be specifically 0.

The preset common value refers to a numerical value that is set to highlight the color of the common pixel relative to the color of the target pixel according to user requirements, and may be specifically 255.

It should be noted that since the image mask is generated based on the coordinate parameters of the thumbnail image, the pixels in the image mask correspond to the pixels in the thumbnail image.

Further, since the pixel value of the pixel in the image mask is mainly the preset target value or the preset normal value, when the subsequent AND operation is performed using the image mask and the thumbnail image, the pixel value of the pixel in the image mask is mainly used. The value is multiplied by the pixel value of the pixel in the thumbnail image.

In this embodiment, the corresponding image template is generated according to the coordinate parameters of the thumbnail, and the pixel values of the pixels in the image template are set to obtain the image mask. In this way, accurate acquisition of the image mask is realized, the accuracy of subsequent operations using the image mask is ensured, and the accuracy of the pathological image standard is further guaranteed.

In an embodiment, as shown in FIG. 6, after step S4 and before step S5, the pathological image labeling method further includes the following steps:

S71: traverse the pixels in the pathological image, and obtain the RGB component value of each pixel.

Specifically, the pixel points in the pathological image are traversed according to a preset traversal mode to obtain the RGB component value of each pixel point, where R, G, and B represent the colors of the three channels of red, green, and blue, respectively.

Among them, the preset traversal method can be based on the pixel point of the upper left corner of the pathological image as the starting point, and traverse line by line from top to bottom from left to right, or traverse from the midline position of the pathological image to both sides at the same time. It can also be other traversal methods, and there is no restriction here.

S72: According to the RGB component value of the pixel, the pathological image is grayed out according to the following formula:

g(x,y)=k ₁ *R(x,y)+k ₂ *G(x,y)+k ₃ *B(x,y) Formula (1)

Among them, x and y are the abscissa and ordinate of each pixel in the pathological image, g (x, y) is the gray value of the pixel (x, y) after graying, R (x, y) Is the color component of the R channel of the pixel (x, y),

G(x,y) is the color component of the G channel of the pixel (x,y), B(x,y) is the color component of the B channel of the pixel (x,y), k ₁ , k ₂ and k ₃ Both are constants.

In the embodiment of this application, in order to achieve accurate extraction of the information content in the pathological image, the pathological image needs to be grayed out first. Among them, the parameter values of k ₁ , k ₂ and k ₃ can be performed according to actual application needs. Setting is not limited here. By adjusting the value range of k ₁ , k ₂ , and k ₃ , the proportions of R channel, G channel and B channel can be adjusted respectively.

The RGB model is a commonly used way of expressing color information. It uses the brightness of the three primary colors of red, green and blue to quantitatively represent colors. This model is also called the additive color mixing model, which is a method in which RGB three-color light is superimposed on each other to achieve color mixing, so it is suitable for display of luminous bodies such as displays.

Gray-scale means that in the RGB model, if R=G=B, the color represents only one gray-scale color. The value of R=G=B is called the gray-scale value. Therefore, each pixel of the gray-scale image Only one byte is needed to store the gray value, and the gray range is 0-255.

It should be noted that in the embodiments of the present application, the pathological image is weighted to calculate the gray value by formula (1). In other embodiments, the component method, the maximum value method, or the average value method may also be used to gray the pathological image. There are no restrictions here.

In this embodiment, by traversing the pixels in the pathological image and obtaining the RGB component value of the corresponding pixel, according to the obtained RGB component value of each pixel, the pathological image is grayed out using formula (1), In this way, the pixel value range of the pixels in the pathological image is set between 0-255, which further reduces the amount of original data of the pathological image and improves the calculation efficiency in the subsequent processing and calculation.

In an embodiment, as shown in FIG. 7, after step S6, the pathological image labeling method further includes the following steps:

S81: Obtain the user type of the operating user from the preset user library.

In the embodiment of the present application, the user types are mainly registered users and anonymous users. Before an operating user can annotate pathological images in the client, he or she needs to log in as a registered user or as an anonymous user. Obtain the user type of the operating user directly from the preset user library. Among them, the preset user database refers to a database dedicated to storing user types of operating users.

S82: Perform permission judgment on the data saving request of the user type, and obtain the target user who has the data saving request permission.

In the embodiment of the present application, a registered user has a special user database for storing data for labeling pathological images, and the pathological image data can be saved in the client. The data annotated by anonymous users in the client is only temporarily saved, and pathological image data cannot be saved in the client.

Specifically, according to step S81, the user type corresponding to the operating user is obtained. If the user type is a registered user, it means that the user type has the permission to save the data; if the user type is anonymous, it means that the user type does not have the data save request. , And determine the registered user as the target user.

It should be noted that registered users include their corresponding id.

S83: Receive the data saving request sent by the target user, and save the marking information of the pathological image of the target user in the user database.

Specifically, when the data saving request sent by the target user on the client is detected, the id contained in the registered user is used to match the id with the library id in the preset storage library, and the successfully matched storage library is determined as the user database , And save the target user's annotation information of the pathological image in the client to the user database.

Among them, the preset storage library refers to the storage library and the library id corresponding to the storage library.

In this embodiment, the permission of the data saving request is judged according to the user type, the operating user who has the permission of the data saving request is determined as the target user, and the marking information of the pathological image of the target user is saved in the user database. In this way, the storage and processing of the target user data is realized, the security of the target user's storage of the pathological image annotation data is ensured, and the target user is convenient for the target user to call the pathological image annotation data, thereby improving the work efficiency of the target user.

It should be understood that the size of the sequence number of each step in the foregoing embodiment does not mean the order of execution. The execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiment of the present application.

In one embodiment, a pathological image labeling device is provided, and the pathological image labeling device corresponds to the pathological image labeling method in the above-mentioned embodiment one to one. As shown in FIG. 8, the pathological image labeling device includes a first acquisition module 81, a recognition module 82, a conventional format module 83, a target format module 84, a second acquisition module 85 and a labeling module 86. The detailed description of each functional module is as follows:

The first acquiring module 81 is configured to acquire pathological images to be labeled from a preset image library;

The recognition module 82 is used to recognize the image format of the pathological image, and determine whether it is a pathological image in a target format or a pathological image in a conventional format;

The conventional format module 83 is used to determine the pathological image corresponding to the conventional format as the region of interest image if the image format is a conventional format;

The target format module 84 is configured to, if the image format is the target format, perform down-sampling processing on the pathological image corresponding to the target format, and use the down-sampling processed pathological image as the region of interest image;

The second obtaining module 85 is configured to receive the labeling request of the operating user, and obtain the application type corresponding to the labeling request from the preset type library;

The annotation module 86 is used to match the application type with the description information in the preset annotation library, and select the annotation strategy corresponding to the successfully matched description information to annotate the image of the region of interest. The preset annotation library contains the description information and The labeling strategy corresponding to the description information.

Further, the identification module 82 includes:

The third acquisition sub-module is used to acquire the file extension corresponding to the pathological image from the preset type table;

The comparison sub-module is used to compare the file extension with the preset extension;

Compare different sub-modules, used to determine the image format corresponding to the pathological image as the conventional format if the file extension is different from the preset extension;

Compare the same sub-module, used to determine the image format corresponding to the pathological image as the target format if the file extension is the same as the preset extension.

Further, the target format module 84 includes:

The down-sampling sub-module is used to perform down-sampling processing on the pathological image corresponding to the target format if the image format is the target format to obtain a thumbnail image;

The extraction sub-module is used to extract the region of interest from the thumbnail image to obtain the region of interest image.

Further, the extraction sub-module includes:

The fourth acquiring unit is used to acquire preset parameters of the region of interest;

An image mask generating unit, configured to generate an image mask according to preset parameters of the region of interest;

The arithmetic unit is used to perform an AND operation between the image mask and the thumbnail image to obtain an image of the region of interest.

Further, the generating unit includes:

The coordinate determination subunit is used to determine the coordinate parameters of the pixel points of the region of interest and the non-interest region in the thumbnail image according to the preset region of interest parameters, where the thumbnail image includes the region of interest and the non-interest region;

The image template generating subunit is used to generate an image template with the same coordinate parameters through a preset port, where the image template contains target pixels of the region of interest and ordinary pixels of the non-interest region;

The image mask determining subunit is used to set the pixel values of the target pixel and the ordinary pixel in the image template to the preset target value and the preset ordinary value, respectively, and use the set image template as the image mask.

Further, the pathological image labeling device further includes:

The traversal module is used to traverse the pixels in the pathological image and obtain the RGB component value of each pixel;

The gray-scale calculation module is used to perform gray-scale processing on the pathological image according to the RGB component value of the pixel according to the following formula:

g(x,y)=k ₁ *R(x,y)+k ₂ *G(x,y)+k ₃ *B(x,y) Formula (1)

Among them, x and y are the abscissa and ordinate of each pixel in the pathological image, g (x, y) is the gray value of the pixel (x, y) after graying, R (x, y) Is the color component of the R channel of the pixel (x,y), G(x,y) is the color component of the G channel of the pixel (x,y), and B(x,y) is the pixel (x,y) The color components of the B channel, k ₁ , k ₂ and k ₃ are all constants.

Further, the pathological image labeling device further includes:

The fifth obtaining module is used to obtain the user type of the operating user from the preset user library;

The authorization judgment module is used to judge the authorization of the data saving request of the user type, and obtain the target user who has the authorization of the data saving request;

The saving module is used to receive the data saving request sent by the target user, and save the marking information of the pathological image of the target user in the user database.

Some embodiments of the application disclose computer equipment. For details, please refer to Fig. 9, which is a block diagram of the basic structure of the computer device 90 in an embodiment of this application.

As shown in FIG. 9, the computer device 90 includes a memory 91, a processor 92, and a network interface 93 that are communicatively connected to each other through a system bus. It should be pointed out that FIG. 9 only shows a computer device 90 with components 91-93, but it should be understood that it is not required to implement all the shown components, and more or fewer components may be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculation and/or information processing in accordance with pre-set or stored instructions. Its hardware includes but is not limited to microprocessors, dedicated Integrated Circuit (Application Specific Integrated Circuit, ASIC), Programmable Gate Array (Field-Programmable Gate Array, FPGA), Digital Processor (Digital Signal Processor, DSP), embedded devices, etc.

The computer device may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server. The computer device can interact with the user through a keyboard, a mouse, a remote control, a touch panel, or a voice control device.

The memory 91 includes at least one type of readable storage medium, the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static memory Random access memory (SRAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), programmable read only memory (PROM), magnetic memory, magnetic disks, optical disks, etc. In some embodiments, the memory 91 may be an internal storage unit of the computer device 90, such as a hard disk or memory of the computer device 90. In other embodiments, the memory 91 may also be an external storage device of the computer device 90, such as a plug-in hard disk equipped on the computer device 90, a smart memory card (Smart Media Card, SMC), and a secure digital (Secure Digital, SD) card, Flash Card, etc. Of course, the memory 91 may also include both the internal storage unit of the computer device 90 and its external storage device. In this embodiment, the memory 91 is generally used to store an operating system and various application software installed in the computer device 90, such as computer readable instructions of the pathological image labeling method. In addition, the memory 91 can also be used to temporarily store various types of data that have been output or will be output.

The processor 92 may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 92 is generally used to control the overall operation of the computer device 90. In this embodiment, the processor 92 is configured to run computer-readable instructions or processed data stored in the memory 91, for example, computer-readable instructions for running the pathological image labeling method.

The network interface 93 may include a wireless network interface or a wired network interface, and the network interface 93 is generally used to establish a communication connection between the computer device 90 and other electronic devices.

This application also provides another implementation manner, that is, a non-volatile computer-readable storage medium storing pathological image data information entry process, and the pathological The image data information entry process can be executed by at least one processor, so that the at least one processor executes the steps of any one of the pathological image labeling methods described above.

Through the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiments can be implemented by means of software plus the necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is better.的实施方式。 Based on this understanding, the technical solution of this application essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, The optical disc) includes several instructions to enable a computer device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in the various embodiments of the present application.

Finally, it should be noted that, obviously, the embodiments described above are only a part of the embodiments of this application, not all of them. The drawings show the preferred embodiments of this application, but do not limit the patents of this application. range. This application can be implemented in many different forms. On the contrary, the purpose of providing these examples is to make the understanding of the disclosure of this application more thorough and comprehensive. Although this application has been described in detail with reference to the foregoing embodiments, for those skilled in the art, it is still possible for those skilled in the art to modify the technical solutions described in each of the foregoing specific implementations, or equivalently replace some of the technical features. . All equivalent structures made using the contents of the description and drawings of this application, directly or indirectly used in other related technical fields, are similarly within the scope of patent protection of this application.

Claims

A method for labeling pathological images, characterized in that the method for labeling pathological images includes:

Obtain the pathological image to be labeled from the preset image library;

Perform image format recognition on the pathological image, and determine whether it is a pathological image in a target format or a pathological image in a conventional format;

If the image format is the conventional format, determining the pathological image corresponding to the conventional format as the region of interest image;

If the image format is the target format, perform down-sampling processing on the pathological image corresponding to the target format, and use the down-sampled pathological image as the region of interest image;

Receiving a labeling request from an operating user, and obtaining the application type corresponding to the labeling request from a preset type library;

The application type is matched with the description information in the preset annotation library, and the annotation strategy corresponding to the description information that is successfully matched is selected to annotate the region of interest image, wherein the preset annotation library contains the description Information and the labeling strategy corresponding to the description information.
The pathological image labeling method according to claim 1, wherein the step of recognizing the image format of the pathological image and determining whether it is a pathological image in a target format or a pathological image in a conventional format comprises:

Acquiring the file extension corresponding to the pathological image from the preset type table;

Comparing the file extension with the preset extension;

If the file extension is different from the preset extension, determining the image format corresponding to the pathological image as a regular format;

If the file extension is the same as the preset extension, the image format corresponding to the pathological image is determined as the target format.
The pathological image labeling method according to claim 1, wherein if the image format is the target format, the pathological image corresponding to the target format is down-sampling processing to down-sample the processed pathological image The step of using an image as the image of the region of interest includes:

If the image format is the target format, performing down-sampling processing on the pathological image corresponding to the target format to obtain a thumbnail image;

Extracting the region of interest on the thumbnail image to obtain the region of interest image.
The pathological image labeling method according to claim 3, wherein the step of extracting the region of interest from the thumbnail image to obtain the image of the region of interest comprises:

Obtain preset parameters of the region of interest;

Generating an image mask according to the preset region of interest parameters;

Performing an AND operation on the image mask and the thumbnail image to obtain the region of interest image.
The pathological image labeling method according to claim 4, wherein the step of generating an image mask according to the preset region of interest parameters comprises:

The coordinate parameters of the pixel points of the region of interest and the non-interest region in the thumbnail image are determined according to the preset region of interest parameters, wherein the thumbnail image includes the region of interest and the non-interest region area;

Generating an image template identical to the coordinate parameters through a preset port, wherein the image template includes target pixels of the region of interest and ordinary pixels of the non-interest region;

The pixel values of the target pixel point and the ordinary pixel point in the image template are respectively set as a preset target value and a preset ordinary value, and the set image template is used as the image mask.
The pathological image labeling method according to any one of claims 1 to 5, wherein if the image format is the target format, the pathological image corresponding to the target format is down-sampled to reduce After the step of sampling the processed pathological image as the image of the region of interest, before the step of receiving the labeling request from the operating user and obtaining the application type corresponding to the labeling request from the preset type library, the pathological image Labeling methods also include:

Traverse the pixels in the pathological image to obtain the RGB component value of each pixel;

According to the RGB component values of the pixels, the pathological image is grayed out according to the following formula:

g(x,y)=k 1 *R(x,y)+k 2 *G(x,y)+k 3 *B(x,y)

Wherein, x and y are the abscissa and ordinate of each pixel in the pathological image, g(x,y) is the gray value of the pixel (x,y) after grayscale processing, R(x,y) is the color component of the R channel of the pixel (x,y), G(x,y) is the color component of the G channel of the pixel (x,y), B(x, y) is the color component of the B channel of the pixel (x, y), and k 1 , k 2 and k 3 are all constants.
The pathological image labeling method according to any one of claims 1 to 5, wherein the application type is matched with the description information in a preset labeling library, and the description information corresponding to the successfully matched description information is selected After the step of labeling the region of interest image by the labeling strategy, the pathological image labeling method further includes:

Obtaining the user type of the operating user from a preset user library;

Perform permission judgment on the data saving request of the user type, and obtain the target user who has the data saving request permission;

Receiving the data saving request sent by the target user, and saving the annotation information of the pathological image of the target user in the user database.
A pathological image labeling device, characterized in that the pathological image labeling device comprises:

The first acquisition module is used to acquire the pathological image to be labeled from the preset image library;

The recognition module is used to recognize the image format of the pathological image, and determine whether it is a pathological image in a target format or a pathological image in a conventional format;

The conventional format module is configured to determine the pathological image corresponding to the conventional format as the region of interest image if the image format is the conventional format;

The target format module is configured to, if the image format is the target format, perform down-sampling processing on the pathological image corresponding to the target format, and use the down-sampled pathological image as the region of interest image;

The second obtaining module is configured to receive the labeling request of the operating user, and obtain the application type corresponding to the labeling request from the preset type library;

An annotation module, configured to match the application type with the description information in a preset annotation library, and select an annotation strategy corresponding to the description information that is successfully matched to annotate the region of interest image, wherein the preset annotation library Contains the description information and the labeling strategy corresponding to the description information.
The pathological image labeling device according to claim 8, wherein the identification module comprises:

The third acquiring submodule is used to acquire the file extension corresponding to the pathological image from the preset type table;

The comparison sub-module is used to compare the file extension with the preset extension;

Comparing different sub-modules, configured to determine the image format corresponding to the pathological image as a regular format if the file extension is different from the preset extension;

The compare same sub-module is configured to determine the image format corresponding to the pathological image as the target format if the file extension is the same as the preset extension.
The pathological image labeling device according to claim 8, wherein the target format module comprises:

The down-sampling sub-module is configured to perform down-sampling processing on the pathological image corresponding to the target format if the image format is the target format to obtain a thumbnail image;

The extraction submodule is used to extract the region of interest from the thumbnail image to obtain the region of interest image.
A computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and capable of running on the processor, wherein the processor executes the computer-readable instructions as follows step:

Obtain the pathological image to be labeled from the preset image library;

Perform image format recognition on the pathological image, and determine whether it is a pathological image in a target format or a pathological image in a conventional format;

If the image format is the conventional format, determining the pathological image corresponding to the conventional format as the region of interest image;

If the image format is the target format, perform down-sampling processing on the pathological image corresponding to the target format, and use the down-sampled pathological image as the region of interest image;

Receiving a labeling request from an operating user, and obtaining the application type corresponding to the labeling request from a preset type library;

The application type is matched with the description information in the preset annotation library, and the annotation strategy corresponding to the description information that is successfully matched is selected to annotate the region of interest image, wherein the preset annotation library contains the description Information and the labeling strategy corresponding to the description information.
The computer device according to claim 11, wherein the step of recognizing the image format of the pathological image and determining whether it is a pathological image in a target format or a pathological image in a conventional format comprises:

Acquiring the file extension corresponding to the pathological image from the preset type table;

Comparing the file extension with the preset extension;

If the file extension is different from the preset extension, determining the image format corresponding to the pathological image as a regular format;

If the file extension is the same as the preset extension, the image format corresponding to the pathological image is determined as the target format.
The computer device according to claim 11, wherein if the image format is the target format, the pathological image corresponding to the target format is down-sampled, and the down-sampled pathological image is used as The steps of the region of interest image include:

If the image format is the target format, performing down-sampling processing on the pathological image corresponding to the target format to obtain a thumbnail image;

Extracting the region of interest on the thumbnail image to obtain the region of interest image.
The computer device according to claim 13, wherein the step of extracting the region of interest from the thumbnail image to obtain the image of the region of interest comprises:

Obtain preset parameters of the region of interest;

Generating an image mask according to the preset region of interest parameters;

Performing an AND operation on the image mask and the thumbnail image to obtain the region of interest image.
15. The computer device of claim 14, wherein the step of generating an image mask according to the preset region of interest parameters comprises:

The coordinate parameters of the pixel points of the region of interest and the non-interest region in the thumbnail image are determined according to the preset region of interest parameters, wherein the thumbnail image includes the region of interest and the non-interest region area;

Generating an image template identical to the coordinate parameters through a preset port, wherein the image template includes target pixels of the region of interest and ordinary pixels of the non-interest region;

The pixel values of the target pixel point and the ordinary pixel point in the image template are respectively set to a preset target value and a preset ordinary value, and the set image template is used as the image mask.
A non-volatile computer-readable storage medium storing computer-readable instructions, wherein the computer-readable instruction is executed by a processor, Make the processor execute the following steps:

Obtain the pathological image to be labeled from the preset image library;

Perform image format recognition on the pathological image, and determine whether it is a pathological image in a target format or a pathological image in a conventional format;

If the image format is the conventional format, determining the pathological image corresponding to the conventional format as the region of interest image;

If the image format is the target format, perform down-sampling processing on the pathological image corresponding to the target format, and use the down-sampled pathological image as the region of interest image;

Receiving a labeling request from an operating user, and obtaining the application type corresponding to the labeling request from a preset type library;

The application type is matched with the description information in the preset annotation library, and the annotation strategy corresponding to the description information that is successfully matched is selected to annotate the region of interest image, wherein the preset annotation library contains the description Information and the labeling strategy corresponding to the description information.
The non-volatile computer-readable storage medium of claim 16, wherein the step of recognizing the image format of the pathological image and determining whether it is a pathological image in a target format or a pathological image in a conventional format comprises :

Acquiring the file extension corresponding to the pathological image from the preset type table;

Comparing the file extension with the preset extension;

If the file extension is different from the preset extension, determining the image format corresponding to the pathological image as a regular format;

If the file extension is the same as the preset extension, the image format corresponding to the pathological image is determined as the target format.
16. The non-volatile computer-readable storage medium of claim 16, wherein if the image format is the target format, down-sampling the pathological image corresponding to the target format is performed, The step of using the down-sampled pathological image as the image of the region of interest includes:

If the image format is the target format, performing down-sampling processing on the pathological image corresponding to the target format to obtain a thumbnail image;
The non-volatile computer-readable storage medium according to claim 18, wherein the step of extracting the region of interest from the thumbnail image to obtain the image of the region of interest comprises:

Obtain preset parameters of the region of interest;

Generating an image mask according to the preset region of interest parameters;

Performing an AND operation on the image mask and the thumbnail image to obtain the region of interest image.
The non-volatile computer-readable storage medium of claim 19, wherein the step of generating an image mask according to the preset region of interest parameters comprises:

The coordinate parameters of the pixel points of the region of interest and the non-interest region in the thumbnail image are determined according to the preset region of interest parameters, wherein the thumbnail image includes the region of interest and the non-interest region area;

Generating an image template identical to the coordinate parameters through a preset port, wherein the image template includes target pixels of the region of interest and ordinary pixels of the non-interest region;

The pixel values of the target pixel point and the ordinary pixel point in the image template are respectively set as a preset target value and a preset ordinary value, and the set image template is used as the image mask.