JP7115508B2 - PATHOLOGICAL IMAGE DISPLAY SYSTEM, PATHOLOGICAL IMAGE DISPLAY METHOD AND PROGRAM - Google Patents

Description

The present technology relates to a pathological image display system, pathological image display method, and program for displaying image data obtained by a microscope in the fields of medicine, pathology, biology, materials, and the like.

In the fields of medicine, pathology, etc., images of slices of living cells, tissues, organs, etc. obtained by an optical microscope are digitized, and based on the digital images, doctors, pathologists, etc. examine the slices, Systems for diagnosing patients have been proposed.

For example, in the method described in Patent Document 1, an image optically obtained by a microscope is digitized by a video camera equipped with a CCD (Charge Coupled Device), the digital signal is input to a control computer system, and a monitor is visualized in A pathologist looks at the image displayed on the monitor and performs an examination (see, for example, paragraphs [0027] and [0028] of Patent Document 1 and FIG. 5).

JP-A-2009-37250

When imaging a section using an optical microscope, the section is placed on a glass slide to prepare a slide. For some reason, multiple sections may be placed on a single glass slide. When the entire glass slide on which a plurality of sections are placed is imaged in this manner, only an integrated image of the entire glass slide is obtained, and an image of each individual section is not obtained. Therefore, it is difficult to handle images of individual slices separately, which may impair user convenience.

On the other hand, if each of a plurality of sections placed on a glass slide is individually imaged, an image of each individual section can be obtained, so that the images of individual sections can be handled separately. However, for example, when a plurality of sections obtained by slicing one specimen multiple times are mounted on a single glass slide, the images of the individual sections captured individually are associated with each other. It is difficult to handle, and there is a risk of impairing convenience for the user.

In view of the circumstances as described above, an object of the present technology is to provide a pathological image display system, a pathological image display method, and a program that are more convenient for users.

In order to solve the above problems, an information processing method, which is a first aspect of the present technology, includes:
Acquiring at least one slide image data obtained by photographing a slide on which a plurality of sections obtained by cutting one specimen in the same direction are placed,
calculating position information indicating relative positions of the first subject area and the second subject area in the acquired slide image data;
determining a first display area of the first subject area;
determining a second display area of the second subject area corresponding to the first display area based on the position information;
The image data of the first display area and the image data of the second display area are output.

In the above information processing method,
extracting feature points of the image data of the first subject area and the image data of the second subject area, and based on the feature points, the image data of the first display area and the second display area; may be generated to calculate the offset between corresponding points with the image data of .

In the above information processing method,
dividing the first subject area and the second subject area into a plurality of triangles with the feature points as vertices, and based on the triangles, the image data of the first display area and the image data of the second display area; Information may be generated for calculating offsets between corresponding points with respect to the image data.

In the above information processing method,
A point at which a change in luminance value exceeds a predetermined threshold may be set as the feature point.

In the above information processing method,
Corresponding feature points in the image data of the first display area and the image data of the second display area may be determined using image correlation.

In the above information processing method,
A plurality of subject areas may be extracted from the slide image data, and the plurality of subject areas having a high degree of similarity may be grouped.

In the above information processing method,
Consecutive numbers may be set for the plurality of grouped subject regions based on the grouping result.

In the above information processing method,
The consecutive numbers may be changeable by the user.

In the above information processing method,
Control may be performed so that the calculated position information and the information for calculating the offset are stored in a storage unit.

In the above information processing method,
When a specification of a range of the first subject area to be displayed in the acquired slide image data is received from the user, the specified range is calculated as the first display area, and the second subject area is calculated. The second display area in the second subject area may be calculated based on the position information when an instruction to move the display to is received from the user.

In the above information processing method,
When a specification of a range of the first subject area to be displayed in the acquired slide image data is received from the user, the specified range is calculated as the first display area, and the second subject area is calculated. calculating the second display area in the second subject area based on the information for calculating the position information and the offset when receiving an instruction from the user to move the display to may be

In the above information processing method,
The position information and the offset so that a point in the image data of the first display area and a corresponding point in the image data of the second display area match each other in the respective display areas. may be calculated based on the information for calculating the second display area in the second subject area.

In the above information processing method,
A certain point of the image data of the first display area may be the center point of the image data, and a corresponding point of the image data of the second display area may be the center point of the image data.

In the above information processing method,
The image data of the first display area and the image data of the second display area may be assigned to a plurality of display spaces and displayed.

In the above information processing method,
The sizes of the plurality of display spaces may be equal to each other.

In the above information processing method,
The image data of the first display area and the image data of the second display area may be allocated to the plurality of display spaces and displayed in synchronization.

An information processing system, which is a second aspect of the present technology,
an imaging unit that captures a slide on which a plurality of sections obtained by cutting one specimen in the same direction are mounted;
an acquisition unit that acquires at least one slide image data obtained by imaging;
a position information calculation unit that calculates position information indicating relative positions of the first subject area and the second subject area in the acquired slide image data;
and a display processing unit that outputs image data of the first display area and image data of the second display area corresponding to the first display area.

An information processing device, which is a third aspect of the present technology,
an acquisition unit that acquires at least one slide image data obtained by photographing a slide on which a plurality of sections obtained by cutting one specimen in the same direction are mounted;
a position information calculation unit that calculates position information indicating relative positions of the first subject area and the second subject area in the acquired slide image data;
determining a second display area of the second subject area corresponding to the first display area based on the position information, and obtaining information about the first display area and information about the second display area; and a display processing unit for outputting.

A program, which is the fourth aspect of the present technology,
Acquiring at least one slide image data obtained by photographing a slide on which a plurality of sections obtained by cutting one specimen in the same direction are placed,
calculating position information indicating relative positions of the first subject area and the second subject area in the acquired slide image data;
determining a second display area of the second subject area corresponding to the first display area based on the position information, and image data of the first display area and an image of the second display area; It is a program that controls data output.

A server device, which is a fifth aspect of the present technology,
an acquisition unit that acquires at least one slide image data obtained by photographing a slide on which a plurality of sections obtained by cutting one specimen in the same direction are mounted;
a position information calculation unit that calculates position information indicating relative positions of the first subject area and the second subject area in the acquired slide image data.

In the above server device,
The position information calculation unit may be configured to extract a plurality of subject areas from the slide image data and group the plurality of subject areas having a high degree of similarity.

In the above server device,
The position information calculation unit may be configured to set consecutive numbers for the grouped plurality of subject areas based on the grouping result.

In the above server device,
The consecutive numbers may be changeable by the user.

The above server device
The apparatus may further include a storage unit that stores information for calculating the calculated position information and the offset.

A display control device, which is a sixth aspect of the present technology,
Slide image data obtained by photographing a slide on which a plurality of sections obtained by cutting one specimen in the same direction are placed, and the first subject area and the second subject area in the slide image data a communication unit that receives image data of each of the first object region and the second object region and the position information of each from a server device that stores the position information of each;
determining a first display area of the received image data of the first subject area, and determining a second display area of the second subject area corresponding to the first display area based on the position information; a determining coordinate calculation unit;
and a display control section for controlling display of the image data of the first display area and the image data of the second display area.

In the above display control device,
The communication unit may be configured to receive offset information between corresponding feature points of the image data of the first display area and the image data of the second display area.

In the above display control device,
The coordinate calculation unit calculates the specified range as the first display area when receiving from the user a specification of the range of the first subject area to be displayed in the slide image data, and calculates the specified range as the first display area. may be configured to determine the second display area in the second subject area based on the position information when receiving a command from the user to move the display to the subject area of .

In the above display control device,
The coordinate calculation unit calculates the specified range as the first display area when receiving from the user a specification of the range of the first subject area to be displayed in the slide image data, and calculates the specified range as the first display area. is configured to determine the second display area in the second subject area based on the position information and the offset information when receiving a command from the user to move the display to the subject area of may be
may be configured.

In the above display control device,
The coordinate calculation unit is configured to make the positions of a certain point of the image data of the first display area and the corresponding point of the image data of the second display area match each other in each of the display areas, The second display area in the second subject area may be determined based on the position information and the offset information.

In the above display control device,
A certain point of the image data of the first display area may be the center point of the image data, and a corresponding point of the image data of the second display area may be the center point of the image data.

In the above display control device,
The display processing unit may be configured to allocate the image data of the first display area and the image data of the second display area to a plurality of display spaces and display them.

In the display control device described above, the plurality of display spaces may have the same size.

In the above display control device,
The display processing unit may be configured to allocate the image data of the first display area and the image data of the second display area to the plurality of display spaces and display them in synchronization.

As described above, according to the present technology, convenience for the user is improved.

It is a figure showing composition of an information processing system concerning an embodiment concerning this art. 1 is a diagram showing a functional configuration of an image server implemented using a typical computer system; FIG. It is a figure for demonstrating a slide image. FIG. 4 is a diagram for explaining feature matching; FIG. FIG. 4 is a diagram for explaining feature matching; FIG. FIG. 4 is a diagram for explaining feature matching; FIG. FIG. 4 is a diagram for explaining feature matching; FIG. FIG. 4 is a diagram for explaining feature matching; FIG. FIG. 2 is a diagram showing the functional configuration of a viewer implemented using a typical computer system; FIG. FIG. 10 is a diagram for explaining setting of numbers of specimen regions; FIG. 10 is a diagram for explaining setting of numbers of specimen regions; FIG. 10 is a diagram for explaining setting of numbers of specimen regions; FIG. 4 is a diagram for explaining discrete display; FIG. 4 is a flowchart showing discrete display processing by a viewer; FIG. 10 is a diagram showing a sample region selection screen; FIG. 10 is a diagram for explaining jump display; FIG. 10 is a flowchart showing jump display processing by a viewer; FIG. 4 is a diagram for explaining synchronous display; 1 is a diagram showing the hardware configuration of a typical computer; FIG. It is a figure which shows the structure of the information processing apparatus which concerns on a modification.

Hereinafter, embodiments according to the present technology will be described with reference to the drawings.

[Information processing system]
FIG. 1 is a diagram showing the configuration of an information processing system according to an embodiment of the present technology.
This information processing system 1 has an image server 100 and one or more viewers 200A and 200B. These can be connected to each other through network 300 . The network 300 may be a WAN (Wide Area Network) such as the Internet, or a LAN (Local Area Network). Also, the network 300 may be wired or wireless. For simplicity of explanation, the case where two viewers 200A and 30B are connected is shown here, but the number of viewers may be three or more. In this specification, when the viewer is not specified individually, it is referred to as "viewer 200". The image server 100 and the one or more viewers 200 may be implemented by, for example, a typical computer.

[Image server 100]
A functional configuration of the image server 100 will be described.
FIG. 2 is a diagram showing the functional configuration of the image server 100 implemented using a typical computer system.

The image server 100 includes an image acquisition unit 109 (acquisition unit), a slide image storage unit 101, a thumbnail image generation unit 110 (generation unit), a thumbnail image storage unit 102, and a specimen region detection unit 103 (detection unit). , an offset coordinate storage unit 104 (first storage unit), a feature matching unit 105, a triangle/matrix storage unit 106, a server control unit 107, and a server communication unit . Each functional unit is realized in computer resources based on a program loaded in RAM (Random Access Memory).

The image acquisition unit 109 acquires an image through a network or acquires slide image data recorded in a removable recording medium, for example, and stores the slide image storage unit 101 by associating the slide image data with the slide ID. accumulate in As shown in FIG. 3, this "slide image" is a glass slide on which a plurality of sections 601 to 607 obtained by cutting one specimen 600 in the same direction are placed in a row and discretely. 610 is an image obtained by photographing. In some cases, such as when a large number of sections are obtained from one specimen 600, a plurality of glass slides on which a plurality of sections are placed are prepared. This "slide ID" is an ID for individually identifying data of a plurality of slide images. Note that "in a line...arranged" means not only a state of being strictly linearly arranged, but also a state of being arranged in a direction from one end side to the other end side of the glass slide.

The thumbnail image generation unit 110 generates a thumbnail image of the slide images accumulated in the slide image accumulation unit 101, and records the thumbnail image data and the slide ID in the thumbnail image storage unit 102 in association with each other. This "thumbnail image" is reduced data obtained by lowering the resolution of the slide image data. The slide image storage unit 101 and thumbnail image storage unit 102 are set as rewritable nonvolatile memories.

The specimen area detection unit 103 reads thumbnail image data from the thumbnail image storage unit 102, and from the read thumbnail image data, a plurality of areas (specimen areas) each including a plurality of sections placed on the glass slide. ) is detected. Detection of this specimen region is performed, for example, as follows. That is, the specimen region detection unit 103 recognizes a location in the image where the brightness sharply changes as a boundary of an object on the glass slide (edge extraction), and recognizes a closed curve having a size equal to or larger than a threshold as an edge (contour) of the segment. do. Subsequently, the specimen area detection unit 103 extracts specimen areas each including the recognized segment from the thumbnail image data. For example, the sample region detection unit 103 selects the range of the sample region so that the slice is positioned substantially in the center of the sample region and the slice is within the range of the sample region. Each specimen area has the same arbitrary shape and size, and in this embodiment each specimen area is rectangular.

The specimen region detection unit 103 also calculates offset coordinates (position information) of each specimen region. The "offset coordinates" are information representing the position of each individual specimen region in the coordinate space of the slide image containing the images of the multiple slices. Specifically, the "offset coordinates" refer to, for example, one vertex of a rectangular glass slide (for example, the lower left end) as the origin, and an arbitrary point within each specimen region (for example, one vertex of a rectangular specimen region). For example, it is a value representing the position of the lower left end) by the difference (distance) in two axial directions from this origin. The specimen area detection unit 103 associates each metadata of the calculated offset coordinates, the specimen area number of the specimen area indicated by the offset coordinates, and the slide number of the slide image containing the specimen area, respectively, to obtain the offset coordinates. Record in the storage unit 104 . This “number” reflects the order of section cutting, that is, the order in which the sections are arranged in a direction substantially perpendicular to the cut surface, and is represented by an integer of 1 or more, for example, and is set by the specimen region detection unit 103. . The offset coordinate storage unit 104 is set as a rewritable nonvolatile memory.

Here, a method for setting slide numbers and sample area numbers will be described. First, the specimen region detection unit 103 performs feature matching between images of a plurality of specimen regions detected from thumbnail image data as described above. This feature matching may be performed by the same technique as the feature matching by the feature matching unit 150, which will be described later. Based on the results of feature matching, the specimen region detection unit 103 determines the degree of similarity between sections A to F within a plurality of specimen regions included in one slide image. Then, as shown in FIG. 10, the specimen region detection unit 103 detects that the specimen regions containing sections B and C with high similarity are located close to each other and the specimen regions containing sections D and E with high similarity are close to each other. Grouping is done so that it is located in Then, as shown in FIG. 11, the specimen region detection unit 103 detects that the section A with a large area is one side (for example, the base) of the specimen 600 and continues from there to the section F with a small area (for example, the tip). Then, as shown in FIG. 12, consecutive specimen region numbers are set so that the positional intervals of the feature points become narrower.

When the specimen area numbers of a plurality of specimen areas included in all slide images are set, the specimen area detection unit 103 calculates the degree of similarity between specimen areas at both ends of a certain slide image and specimen areas at both ends of another slide image. judge. The sample region detection unit 103 determines that slide images including sample regions with high similarity are continuous slide images, and sets continuous slide numbers. The specimen area detection unit 103 rewrites the slide ID recorded in the slide image storage unit 101 and the thumbnail image storage unit 102 with the set slide number.

Note that the specimen region number and slide number may be modified by the user using the viewer 200 . Alternatively, when the user arranges the sections in order from left to right on the glass slide, or when the user wants to position the sample region suspected of having a disease at the top, the user uses the viewer 200 to display the sample. A region number and a slide number may be set. Alternatively, the viewer 200 may take in information input to the LIS (Laboratory Information System) to set the sample area number and the slide number.

The feature matching unit 105 performs feature matching (details will be described later) between images of specimen regions of the slide images accumulated in the slide image accumulation unit 101 . Specifically, the feature matching unit 105 reads slide image data from the slide image storage unit 101 . Then, the feature matching unit 105 reads, from the offset coordinate storage unit 104, a plurality of sample region numbers and offset coordinates associated with the slide numbers associated with the read slide image data. The feature matching unit 105 detects a plurality of specimen regions from the slide image based on the read offset coordinates and the size given in common to each specimen region. The feature matching unit 105 performs feature matching between the images of the plurality of detected specimen regions, and calculates triangles and affine transformation matrices. The feature matching unit 105 associates the data of the calculated triangles and affine transformation matrices with the metadata of the sample region number and slide number specifying each sample region, and records them in the triangle/matrix storage unit 106 . The triangle/matrix storage unit 106 is set to a rewritable nonvolatile memory.

In response to a request from the viewer 200 , the server control unit 107 reads the corresponding image data from the slide image storage unit 101 and supplies the image data to the viewer 200 using the server communication unit 108 .

[Feature matching]
Here, a specific method of feature matching between images of specimen regions by the feature matching unit 105 will be described.
First, the feature matching unit 105 uses an extraction algorithm such as the Harris algorithm to extract luminance values from an image of an arbitrary reference detection area (referred to as detection area A), as shown in FIG. points (called feature points; black points in the figure) where the change in is greater than the threshold are extracted. This Harris algorithm has the advantage of being robust against image rotation and taking no feature points on the gradient.

Subsequently, the feature matching unit 105 uses the normalized correlation of the image to determine where each feature point in the image of the detection area A corresponds to in the image of another detection area (referred to as the detection area B). decide. Specifically, as shown in FIG. 5, the feature matching unit 105 sets a search area SA in the image of the detection area B with reference to the feature point AP (see FIG. 4) in the image of the detection area A serving as a reference. do. The feature matching unit 105 searches the search area SA of the detection area B, calculates the normalized correlation between the texture pattern of the detection area B and the texture pattern of the detection area A, and finds the point with the highest value ( ) is determined as a corresponding point (white point in the figure) BP to the feature point (black point in the figure) AP. In the example shown in the figure, the feature matching unit 105 selects the point with the highest value 0.98 among the values 0.04, 0.87, and 0.98 obtained by calculating the normalized correlation as the corresponding point. BP is determined. Note that if this highest point is less than a predetermined threshold value (for example, 0.90), the feature matching unit 105 determines that this feature point AP cannot be used for matching. Further, when the number of feature points determined to be adopted does not reach the predetermined threshold, the feature matching unit 105 performs matching between the detection areas A and B because the similarity between the detection areas A and B is low. is judged to be impossible.

After calculating the normalized correlation for all the feature points as described above, as shown in FIG. 6, the feature matching unit 105 divides the image of the detection region A into It is divided into a plurality of triangles with the adopted feature points (black dots in the figure) as vertices. As shown in FIG. 7, the feature matching unit 105, while maintaining the division topology of the detection region A, also adds corresponding points (white point) into multiple triangles with vertices.

Subsequently, the feature matching unit 105 identifies a triangle AT including the display center coordinates AC of the image of the detection area A by the inside/outside determination. The feature matching unit 105 calculates an affine transformation matrix for transforming the triangle BT of the detection area B corresponding to the triangle AT of the detection area A into this triangle AT. The calculated affine transformation matrix is used to calculate the coordinates (in this case, center coordinates) of a corresponding point in another detection region based on the coordinates (for example, center coordinates) of an arbitrary point in one detection region. can be done. More specifically, as shown in FIG. 8, the affine transformation matrix is, for example, the display center coordinates AC (see FIG. 8) of the image of the specimen region A (see FIG. 6) on the image of the specimen region B. The position BC can be determined and used for the process of moving the display center of the screen of the specimen area B to the position BC.

[Viewer 200]
A functional configuration of the viewer 200 will be described.
FIG. 9 is a diagram showing the functional configuration of viewer 200 implemented using a typical computer system.

The viewer 200 includes a viewer control unit 207, a viewer communication unit 204, an alignment data creation unit 208, an alignment data storage unit 209, a first coordinate calculation unit 203, a second coordinate calculation unit 206, a third A coordinate calculation unit 210 , a display processing unit 205 , a display unit 201 and an input unit 202 . Each functional unit is implemented in computer resources based on a program loaded in RAM.

The viewer control unit 207 transmits a request to the image server 100 using the viewer communication unit 204, supplies the received data to each functional unit of the viewer 200, and performs processing such as controlling each functional unit.

The alignment data creation unit 208 calculates biaxial offset amounts between a plurality of specimen regions in the coordinate space of the slide image based on the triangle and affine transformation matrix data obtained by feature matching. More specifically, the alignment data creation unit 208 creates a triangle containing a point located at, for example, the central coordinates of a certain specimen region as a reference point, and a triangle containing a point corresponding to this reference point in another specimen region. , offset amounts in the two-axis directions of coordinates in the coordinate space of the specimen region are calculated as alignment data. The alignment data creation unit 208 records the calculated alignment data in the alignment data storage unit 209 . The alignment data storage unit 209 is set as a rewritable nonvolatile memory.

The first coordinate calculation unit 203 performs processing such as calculating the position information of the display area for performing an operation (discrete display) for sequentially displaying only the sample area while skipping areas other than the sample area.

The second coordinate calculation unit 206 performs processing such as calculating position information of a display area for performing an operation (jump display) for sequentially displaying corresponding areas of the sample area. The “corresponding regions” refer to regions within a plurality of sections that match in a direction substantially perpendicular to the cut surface.

The third coordinate calculation unit 210 divides the display screen and performs processing such as calculating the position information of the display region for performing an operation (synchronized display) for synchronizing and displaying a plurality of sample regions. conduct.

The display processing unit 205 outputs display data for displaying image data to the display unit 201 .

[Operation of viewer 200]
Next, the operation of the viewer 200 will be explained. The operation will be explained in the following order.
1. Operation for skipping areas other than the specimen area and displaying only the specimen area in order (discrete display)
2. Operation for sequentially displaying the corresponding areas of the sample area (jump display)
3. Operation for splitting the display screen and synchronizing and displaying multiple sample areas (synchronous display)

[1. Discrete display]
In the discrete display, processing is performed to skip regions other than the specimen region (region where no section is placed) in the slide image and display only the specimen region in order.

FIG. 13 is a diagram showing a specific example of discrete display. The figure shows a first glass slide 610 and a second glass slide 620 . If the sections from one specimen cannot fit on one glass slide, they are placed across multiple glass slides. In the figure, among a plurality of sections cut in order of first to fourteenth sections 601 to 614, first to seventh sections 601 to 607 are placed on a first glass slide 610 in this order, and An example is shown in which 8th through 14th sections 608-614 are placed on a second glass slide 620 in that order. It is also assumed that first to fourteenth specimen regions 621 to 634 containing first to fourteenth sections 601 to 614 are respectively detected. First, a rectangular area at the lower left corner of the first sample area 621 is displayed on the display unit 201 as the display area 641 . The user uses the input unit 202 such as a mouse to input a command to move the image in the display area 641 vertically and horizontally. When an instruction to move the display area 641 to the right is input, the display area moves horizontally within the first sample area 621 to the display area 641a at the lower right corner of the first sample area 621 (arrow A). Further, when an instruction to move to the right is input, the area between the first and second specimen areas 621 and 622 where no section is placed is skipped, and the lower left corner of the second specimen area 622 is displayed. The display area moves to area 641b (arrow B). Assume that this movement between sample regions is repeated, the display region moves to the display region 641c at the lower right corner of the seventh sample region 621, and an instruction to move to the right is input. In this case, since there is no specimen area to the right of the seventh specimen area 627 on the first glass slide 610, The display area moves (arrow C).

Next, this discrete display processing will be described in more detail.
FIG. 14 is a flowchart showing discrete display processing by the viewer 200. FIG.

The user selects discrete display as the display mode using the input unit 202, for example. Upon receiving the selection instruction, the viewer control unit 207 uses the viewer communication unit 204 to read the thumbnail image of the slide image recorded in the thumbnail image storage unit 102 of the image server 100 in association with the slide number. The viewer control unit 207 causes the display processing unit 205 to generate a specimen region selection screen based on the read slide number and thumbnail image of the slide image, and causes the display unit 201 to display the screen.

FIG. 15 is a diagram showing a specimen area selection screen.
As shown in the figure, the sample region selection screen 220 includes a slide tray 221 , a sample region tray 222 , an observation display area 223 and a display region display area 224 . Thumbnail images 230 , 231 , and 232 of slide images read from the image server 100 by the viewer control unit 207 are displayed on the slide tray 221 as a list. The thumbnail images 230, 231, and 232 of the slide images are arranged in order of slide number from top to bottom. The user selects one thumbnail image 231 from the plurality of thumbnail images 230 , 231 , 232 displayed on the slide tray 221 using the input unit 202 such as a mouse. The viewer control unit 207 causes the display processing unit 205 to display the selected thumbnail image 231 in a form that can be identified by a frame line (231A) or the like, and displays a thumbnail image 231C corresponding to the selected thumbnail image 231 in the display area. Displayed in the display area 224 .

The viewer control unit 207 transmits the specimen region number and the offset coordinates as metadata recorded in the offset coordinate storage unit 104 of the image server 100 in association with the slide number associated with the selected thumbnail image through viewer communication. Read out using the unit 204 . In addition, the viewer control unit 207 uses the viewer communication unit 204 to transmit the specimen region number, the triangle, and the affine matrix as metadata recorded in the triangle/matrix storage unit 106 of the image server 100 in association with the slide number. read out. The viewer control unit 207 causes the display processing unit 205 to display the thumbnail images 231a to 231g of the plurality of sample regions in the selected thumbnail image 231 based on the offset coordinates and the slide number as the read metadata. 222 as a list. The thumbnail images 231a to 231g of specimen regions are arranged in order of specimen region number from left to right.

The user selects one thumbnail image 231d from the plurality of thumbnail images 231a to 231g displayed on the sample region tray 222 using the input unit 202 such as a mouse. The selected thumbnail image 231d is displayed so as to be identifiable by a frame line (231B) or the like. Also, the image of the specimen region of the selected thumbnail image 231d is displayed in the observation display area 223. FIG. In this observation display area 223, in addition to displaying an image of the entire specimen area, a part of the specimen area (display area) is displayed at an arbitrary resolution. Further, in the thumbnail image 231C displayed in the display area display area 224, the position of the specimen area currently displayed in the observation display area 223 is displayed so as to be identifiable by a frame line (224a) or the like. The operation of selecting the slide to be displayed and the sample area is common to discrete display, jump display, and synchronous display.

The viewer control unit 207 detects user input to the input unit 202 and determines the slide number and sample area number of the slide and sample area to be displayed first. The viewer control unit 207 then supplies the offset coordinates associated with the slide number and specimen region number as metadata to the first coordinate calculation unit 203 that performs processing for discrete display. The first coordinate calculation unit 203 calculates the display to be displayed first based on the offset coordinates of the specimen region as metadata, and the size and initial position (for example, the lower left corner of the specimen region) commonly given to each display region. Calculate the location information of the region. The size and initial position commonly given to each of these display areas are recorded in a rewritable nonvolatile memory. The first coordinate calculation unit 203 supplies the calculated position information to the viewer control unit 207, and the viewer control unit 207 sends a request for obtaining an image corresponding to this display area using the viewer communication unit 204. Send to server 100 .

The server control unit 107 of the image server 100 receives this request using the server communication unit 108 and reads the image data of the display area included in the request from the slide image storage unit 101 . The server control unit 107 transmits the read image data to the viewer 200 using the server communication unit 108 .

The viewer control unit 207 of the viewer 200 receives the image data of the display area using the viewer communication unit 204 and supplies the received image data of the display area to the display processing unit 205 . The display processing unit 205 outputs display data for displaying this image data to the display unit 201, and displays it in the observation display area 223 of the sample region selection screen 220 (step S101).

The user uses the input unit 202 such as a mouse to input an instruction to move the image of the display area displayed in the observation display area 223 vertically and horizontally. The viewer control unit 207 detects input of a movement command to the input unit 202 by the user, and supplies the detected movement command to the first coordinate calculation unit 203 . The first coordinate calculation unit 203 calculates the position information of the destination display area based on this movement command. The first coordinate calculation unit 203 supplies the calculated position information to the viewer control unit 207, and the viewer control unit 207 sends a request for obtaining an image corresponding to this display area using the viewer communication unit 204. Send to server 100 .

After that, as in step S101, the server control unit 107 of the image server 100 reads the corresponding image data from the slide image storage unit 101 in response to a request from the viewer 200, and transmits it to the viewer 200 using the server communication unit 108. do. As a result, the viewer 200 acquires the image data of the destination display area and updates the display content of the display unit 201 (step S102).

When the first coordinate calculation unit 203 determines that the destination display area has reached the boundary of the specimen area based on the calculated position information (Yes in step S103), the boundary is located at either end of the specimen area. (Right end, bottom end, left end, top end) is determined (steps S104, S105, S106). When the display area is located at the right end of the specimen area, the first coordinate calculation unit 203 increments the specimen area number of the specimen area to be displayed when detecting a rightward movement instruction from the user (Yes in step S104). do. The specimen area number thus obtained is the specimen area number of the specimen area to be displayed next. The first coordinate calculation unit 203 sets the display area within the specimen area corresponding to the specimen area number obtained by incrementing as follows. That is, the first coordinate calculation unit 203 maintains the Y offset between the upper end of the sample region and the upper end of the display region, and adjusts the display in the target sample region so that the left end of the sample region matches the left end of the display region. A region is set (step S107).

On the other hand, when the display area is located at the lower end of the specimen area and the first coordinate calculation unit 203 detects a downward movement instruction from the user (Yes in step S105), the first coordinate calculation unit 203 calculates the specimen area number of the specimen area to be displayed. is incremented. The first coordinate calculation unit 203 sets the display area within the specimen area corresponding to the specimen area number obtained by incrementing as follows. That is, the first coordinate calculation unit 203 maintains the X offset between the left end of the sample region and the left end of the display region, and adjusts the display in the target sample region so that the upper end of the sample region and the upper end of the display region match. A region is set (step S108).

On the other hand, when the display area is located at the left end of the specimen area and the first coordinate calculation unit 203 detects an instruction to move leftward from the user (Yes in step S106), the first coordinate calculation unit 203 calculates the specimen area number of the specimen area to be displayed. is decremented. The first coordinate calculation unit 203 sets the display area within the specimen area corresponding to the specimen area number obtained by decrementing as follows. That is, the first coordinate calculation unit 203 maintains the Y offset between the upper end of the specimen region and the upper end of the display region, and adjusts the display area in the destination specimen region so that the right end of the specimen region coincides with the right end of the display region. A region is set (step S109).

On the other hand, when the display area is located at the upper end of the specimen area and the first coordinate calculation unit 203 detects an upward movement instruction from the user (No in step S106), the first coordinate calculation unit 203 calculates the specimen area number of the specimen area to be displayed. is decremented. The first coordinate calculation unit 203 sets the display area within the specimen area corresponding to the specimen area number obtained by decrementing as follows. That is, the first coordinate calculation unit 203 maintains the X offset between the left end of the sample region and the left end of the display region, and adjusts the display in the target sample region so that the lower end of the sample region and the lower end of the display region match. A region is set (step S110).

In this way, when the display area within the target specimen area is set based on the user's movement command supplied from the viewer control unit 207 (steps S107 to S110), the first coordinate calculation unit 203 calculates the set The position information of the display area is supplied to the viewer control unit 207 . The viewer control unit 207 then uses the viewer communication unit 204 to transmit a request for obtaining an image corresponding to this display area to the image server 100 .

After that, as in step S101, the server control unit 107 of the image server 100 reads the corresponding image data from the slide image storage unit 101 in response to a request from the viewer 200, and transmits it to the viewer 200 using the server communication unit 108. do. As a result, the viewer 200 acquires the image data of the destination display area and updates the display content of the display unit 201 .

If the specimen region number obtained by incrementing in steps S107 and S108 is not recorded in the offset coordinate storage unit 104, the first coordinate calculation unit 203 increments the slide number of the currently displayed slide. Then, the first coordinate calculation unit 203 sets the inside of the sample region corresponding to the sample region number=1 associated with the slide number obtained by the increment as an observation target. If the sample region number obtained by decrementing in steps S109 and S110 is not recorded in the offset coordinate storage unit 104 (sample region number=0), the first coordinate calculation unit 203 calculates the currently displayed slide. Decrement the slide number of Then, the first coordinate calculation unit 203 sets the specimen area corresponding to the maximum specimen area number associated with the slide number obtained by decrementing as an observation target.

When the first coordinate calculation unit 203 determines that there is no specimen region to be displayed next as a result of the above calculation (Yes in step S111), the discrete display processing ends.

As described above, the information processing system 1 of the present embodiment acquires image data obtained by photographing a slide on which a plurality of sections obtained by cutting one specimen in the same direction are placed discretely. The acquiring unit 109 detects a plurality of sample regions having the same shape and including individual sections in the acquired image data, and calculates position information relatively indicating the position of each individual sample region in the coordinate space of the image data. an offset coordinate storage unit 104 for storing calculated position information; and a viewer control unit 207 for switching display between sample regions based on the stored position information. .

That is, the viewer control unit 207 switches the display between sample regions based on the position information, which is metadata. As a result, the viewer control unit 207 skips regions other than the specimen region, even though the image data acquired by the acquisition unit is image data obtained by photographing the slide itself on which the plurality of sections are placed. , the display can be switched between sample areas. In addition, the first storage unit records the specimen regions as position information, which is metadata, so that display switching between specimen regions can be performed efficiently and at high speed. Furthermore, by using the position information, which is the metadata, even when a plurality of sample regions spans image data of a plurality of slides, similar to the case of sequentially displaying a plurality of sample regions detected from one image data, Display processing can be performed efficiently.

The information processing system 1 of the present embodiment further includes a thumbnail image thumbnail image generation unit 110110 that generates reduced data obtained by lowering the resolution of image data, and the specimen area detection unit 103 detects a plurality of specimen areas from the reduced data. By doing so, a plurality of specimen regions in the image data are detected.

In other words, the thumbnail image generation unit 110 detects the specimen area from the reduced data whose resolution is lowered, so that the specimen area can be detected efficiently and at high speed.

In the information processing system 1 of the present embodiment, a plurality of image data of a plurality of sample regions are managed in cutting order, and the viewer control unit 207 designates the sample region to be displayed in the acquired image data and the range therein. When receiving an instruction from the user, the range within the specified sample area is calculated as the display area, and when the display area is at the end of the sample area, the instruction is received from the user to move the display area outside the end. , the display is switched to the next sample area in cutting order based on the stored position information.

That is, even when a display area that is part of the sample area is displayed, the display can be switched between sample areas by skipping areas other than the sample area.

In the information processing system 1 of the present embodiment, the viewer control unit 207 changes the display area when the calculated display area is at the end of the sample area in one axial direction of the plurality of sample areas in the coordinate space of the image data. When an instruction is received from the user to move in one axial direction to the outside of the distal end, the display is switched to the next specimen region in cutting order while fixing the position in the other axial direction.

As a result, the position of the movement destination display area within the specimen area corresponds to the position of the movement destination display area of the movement source within the specimen area, so that convenience for the user can be expected.

In the information processing system 1 of the present embodiment, the image acquisition unit 109 photographs a slide on which a plurality of sections obtained by cutting one sample in the same direction are arranged discretely in a line. The viewer control unit 207 acquires the obtained image data, and when the calculated display area is at the end of the sample area in the direction in which the plurality of sample areas are arranged in the coordinate space of the image data, the viewer control unit 207 moves the display area from the end. When an instruction to move in the alignment direction is received from the user first, the display area of the target sample area is calculated based on the stored position information.

As a result, even when displaying a plurality of sample regions arranged in a line, the display can be switched between the sample regions by skipping the regions other than the sample regions.

In the information processing method of this embodiment, the image acquisition unit 109 acquires image data obtained by photographing a slide on which a plurality of sections obtained by cutting one sample in the same direction are placed discretely. and the specimen region detection unit 103 detects a plurality of specimen regions having the same shape including individual sections in the acquired image data, and relatively compares the positions of the individual specimen regions in the coordinate space of the image data. The position information to be displayed is calculated, the offset coordinate storage unit 104 stores the calculated position information, and the viewer control unit 207 switches display between sample regions based on the stored position information.

[2. Jump display]
Next, the jump display will be explained. In the jump display, processing is performed to sequentially display areas corresponding to the specimen area.

FIG. 16 is a diagram showing a specific example of jump display. 13, the first to fourteenth sections 601 to 614 are placed on first and second glass slides 610 and 620 in this order, and the first to fourteenth sections 601 to 614 are An example in which first to fourteenth specimen regions 621 to 634 are respectively detected is shown. With the display area 642 at an arbitrary position within the first specimen area 621 being displayed, the currently displayed first specimen area 621 is selected from the specimen area tray 222 ( FIG. 15 ) on the specimen area selection screen 220 . The thumbnail image of the sample region 622 to the right of the thumbnail image is selected as the destination sample region. When the command for moving the display to the image of another specimen area is input in this way, the display area is moved to the display area 642a of the second specimen area 622 (arrow D). Here, the display areas 642 and 642a are located at the same position in the coordinate space of the specimen areas 621 and 622 having the same shape. In other words, regions that are continuous in a direction substantially orthogonal to the cut surface of the specimen are displayed as the display areas 642 and 642a. This movement between sample regions is repeated, and the display region is moved to the display region 642b of the seventh sample region 621 (located at the same position as the display regions 642 and 642a in the coordinate space of the sample regions of the same shape). do. In addition, the thumbnail image of the second glass slide 620 located below the thumbnail image of the first glass slide 610 in the sample area tray 222 (FIG. 15) is manipulated to move to the second glass slide 620. Assume that an instruction for In this case, the display area (located at the same position as the display area 642b in the coordinate space of the same-shaped specimen area) moves to the display area 642c of the eighth specimen area 628 of the second glass slide 620 (arrow E ).

Next, this jump display processing will be described in more detail.
FIG. 17 is a flow chart showing jump display processing by the viewer 200 .

The slide and sample area to be displayed first are selected in the same manner as the slide and sample area to be displayed first are selected in the discrete display. Also, the user selects the jump display as the display mode using the input unit 202, for example.

Upon judging the selection of jump display, the viewer control unit 207 supplies the slide number, specimen region number, triangle and affine transformation matrix data as metadata to the alignment data creation unit 208 . After obtaining these metadata, the alignment data creation unit 208 calculates the offsets in the two-axis directions on the coordinate space of the slide image of the specimen regions for all combinations of the plurality of specimen regions. More specifically, the alignment data creation unit 208 uses an affine transformation matrix to create a triangle that includes a point located at, for example, the center coordinates of a given specimen region as a reference point, and a corresponding reference point in another specimen region. Calculate the offset in the two-axis direction on the absolute coordinate with the triangle containing the point to be. This offset serves as alignment data for adjusting the display area within one specimen area with respect to the display area within the other specimen area. The alignment data creation unit 208 records the calculated alignment data in the alignment data storage unit 209 (step S201). This alignment data generation operation is common to the display modes of jump display and synchronous display.

The viewer control unit 207 converts the offset coordinates associated with the slide number and sample region number of the display target slide and sample region selected by the user into a second coordinate calculation unit 206 that performs processing for jump display. supply to The second coordinate calculation unit 206 calculates the position information of the display area to be displayed first based on the offset coordinates and the size and initial position commonly given to each display area (for example, the lower left corner of the sample area). do. The size and initial position commonly given to each of these display areas are recorded in a rewritable nonvolatile memory. The second coordinate calculation unit 206 supplies the calculated position information to the viewer control unit 207, and the viewer control unit 207 uses the viewer communication unit 204 to send a request for acquiring an image corresponding to this display area. Send to server 100 .

The server control unit 107 of the image server 100 receives this request using the server communication unit 108 and reads the image data of the display area included in the request from the slide image storage unit 101 . The server control unit 107 transmits the read image data to the viewer 200 using the server communication unit 108 .

The viewer control unit 207 of the viewer 200 receives the image data of the display area using the viewer communication unit 204 and supplies the received image data of the display area to the display processing unit 205 . The display processing unit 205 outputs display data for displaying this image data to the display unit 201 and displays it in the observation display area 223 of the specimen region selection screen 220 (step S202).

The user uses the input unit 202 such as a mouse to input an instruction to move the image of the display area displayed in the observation display area 223 vertically and horizontally. The viewer control unit 207 detects input of a movement command to the input unit 202 by the user, and supplies the detected movement command to the second coordinate calculation unit 206 . The second coordinate calculation unit 206 calculates the position information of the destination display area based on this movement command. The second coordinate calculation unit 206 supplies the calculated position information to the viewer control unit 207, and the viewer control unit 207 uses the viewer communication unit 204 to send a request for acquiring an image corresponding to this display area. Send to server 100 .

After that, as in step S202, the server control unit 107 of the image server 100 reads the corresponding image data from the slide image storage unit 101 in response to a request from the viewer 200, and transmits it to the viewer 200 using the server communication unit 108. do. As a result, the viewer 200 acquires the image data of the destination display area and updates the display content of the display unit 201 (step S203).

The user uses the input unit 202 to input an instruction to move the display from the image of the specimen region displayed in the observation display area 223 to the image of another specimen region. This input is, for example, selecting a specimen area to the right or left of the currently displayed specimen area from a group of thumbnail images displayed in the specimen area tray 222 (FIG. 15) of the specimen area selection screen 220. It is done by Alternatively, this input is performed by selecting a slide above or below the currently displayed slide from a group of thumbnail images displayed in the slide tray 221 (FIG. 15) of the sample region selection screen 220. When the viewer control unit 207 detects that the user has input a movement command to the input unit 202 (Yes in step S204), the viewer control unit 207 detects whether the input specimen region of the movement destination is adjacent to the right or the left, or whether the slide of the movement destination is on the top. It is determined whether it is below (steps S205 and S206).

When the viewer control unit 207 determines that the destination specimen area is the adjacent right side or the destination slide is the bottom (Yes in step S205), the viewer control unit 207 increments the specimen area number of the specimen area to be displayed. On the other hand, when the viewer control unit 207 determines that the destination specimen area is to the left or the destination slide is on the top (Yes in step S206), the viewer control unit 207 decrements the specimen area number of the specimen area to be displayed. The specimen area number thus obtained becomes the specimen area number of the destination specimen area.

The viewer control unit 207 notifies the second coordinate calculation unit 206 of the specimen area number of the movement destination specimen area and the position information of the currently displayed display area (movement source display area). The second coordinate calculation unit 206 reads out the offset coordinates associated with the destination specimen region number from the offset coordinate storage unit 104 . Further, the second coordinate calculation unit 206 reads from the alignment data storage unit 209 the alignment data for matching the movement destination sample region with the movement source sample region on the coordinate space. Then, based on the offset coordinates and alignment data as metadata, the second coordinate calculation unit 206 determines whether corresponding points in the display areas of the movement source and the movement destination match each other in the respective display areas. A display area within the specimen area of the movement destination corresponding to the display area of the movement source is set so as to be .

In other words, a certain point (for example, the center point of the display area) in the display area of the movement source and the corresponding point in the display area of the movement destination may not match each other in the respective display areas. For example, even if there is a disease in the center of the display area of the movement source, the disease may be located at a position shifted from the center in the display area of the movement destination. Therefore, based on the alignment data, the second coordinate calculation unit 206 determines that a certain point in the display area of the movement source and a corresponding point in the display area of the movement destination match each other in the respective display areas. (for example, the center of the display area), the display area of the destination is adjusted (steps S207 and S208).

The second coordinate calculation unit 206 calculates the position information of the display area adjusted in steps S207 and S208. The second coordinate calculation unit 206 supplies the calculated position information to the viewer control unit 207, and the viewer control unit 207 uses the viewer communication unit 204 to send a request for acquiring an image corresponding to this display area. Send it to the server 100 .

After that, as in step S202, the server control unit 107 of the image server 100 reads the corresponding image data from the slide image storage unit 101 in response to a request from the viewer 200, and transmits it to the viewer 200 using the server communication unit 108. do. As a result, the viewer 200 acquires the image data of the destination display area and updates the display content of the display unit 201 .

When the second coordinate calculation unit 206 determines that there is no specimen region to be displayed next as a result of the above calculation (Yes in step S209), the discrete display processing ends.

As described above, the information processing system 1 of the present embodiment includes the alignment data creation unit 208 that detects the feature points of each of a plurality of sample regions and calculates the offset amount of the coordinates of each feature point in the coordinate space of the sample region, and the calculation and an alignment data storage unit 209 that stores the offset amount that has been set. When the viewer control unit 207 receives an instruction from the user to jump the display area to another specimen area, the viewer control unit 207 stores the stored position information and the stored position information. Based on the offset amount obtained, the jump destination display area is calculated so that the positions of the feature points in the display area before and after the jump match.

That is, since the alignment data creation unit 208 sets the display area based on the offset amount between the feature points included in the plurality of sample areas as metadata, the viewer control unit 207 controls the display area before and after the jump. Feature point positions can be matched. In addition, by using metadata for setting the display area, the corresponding display area can be accurately calculated, and the display processing can be performed efficiently and at high speed.

In the information processing system 1 of the present embodiment, the alignment data creation unit 208 calculates an affine transformation matrix for matching feature points in a plurality of detected specimen regions in the display space, and converts the calculated affine transformation matrix to Calculate the offset amount.

By using the affine transformation matrix, it is possible to calculate the offset amounts in the two axial directions. This makes it possible to more accurately calculate the corresponding display areas before and after the jump.

[3. Synchronous display]
Next, synchronous display will be described. In the synchronous display, processing is performed to divide the display screen and display corresponding regions of a plurality of sample regions in synchronization with each other.

FIG. 18 is a diagram showing a specific example of synchronous display. In this synchronous display, the user selects a plurality of (for example, four) sample regions 621 to 624 as display targets. The plurality of specimen areas 621 to 624 are individually assigned to display spaces 651 to 654 obtained by dividing the observation display area 223 (FIG. 15) of the specimen area selection screen 220 into a plurality of areas of approximately equal size. Is displayed. More specifically, the points 621a to 624a located at the center of the plurality of sample regions 621 to 624 are arranged so that their positions in the display spaces 651 to 654 match each other (located at the center). sample areas 621 to 624 are displayed.

Next, this synchronous display processing will be described in more detail.
Using the input unit 202, the user selects a slide to be displayed from the slide tray 221 (FIG. 15) of the sample region selection screen 220 displayed on the display unit 201, and selects a plurality of slides to be displayed from the sample region tray 222 (for example, 4) specimen regions. The viewer control unit 207 detects a user's input to the input unit 202, and determines slide numbers and multiple sample area numbers of slides to be displayed and multiple sample areas. The viewer control unit 207 then supplies the slide number and the plurality of specimen region numbers to the third coordinate calculation unit 210 that performs processing for synchronous display. Further, the viewer control unit 207 divides the observation display area 223 (FIG. 15) of the sample region selection screen 220 as a display space into display spaces equal in number to the designated multiple (for example, four) sample regions. The viewer control unit 207 then supplies the coordinate information of each divided display space to the third coordinate calculation unit 210 .

The third coordinate calculation unit 210 calculates the offset coordinates associated with the sample region numbers of the plurality of display target sample regions and the alignment data for adjusting the offset between the plurality of display target sample regions. , the display area is set so that the center points of a plurality of specimen areas coincide. That is, based on the offset coordinates and alignment data as metadata, the third coordinate calculation unit 210 performs the calculation so that the points positioned at the centers of the plurality of sample regions are aligned with each other in the display space. Set the display area to (so that it is centered). The third coordinate calculation unit 210 calculates position information of each set display area. The third coordinate calculation unit 210 supplies the obtained position information to the viewer control unit 207, and the viewer control unit 207 uses the viewer communication unit 204 to send a request for acquiring an image corresponding to this display area. Send to the image server 100 .

After that, as in step S202, the server control unit 107 of the image server 100 reads out the corresponding image data from the slide image storage unit 101 in response to a request from the viewer 200, and sends it to the viewer 200 using the server communication unit 108. Send. Thereby, the viewer 200 acquires image data of a plurality of display areas and displays them on the display unit 201 . Specifically, the viewer control unit 207 divides the observation display area 223 (FIG. 15) of the sample region selection screen 220 displayed on the display unit 201 into areas of approximately the same size as the number of sample regions to be displayed. To divide. Then, the display processing unit 205 divides the observation display area 223 ( FIG. 15 ) of the specimen region selection screen 220 displayed on the display unit 201 into a plurality of areas of approximately equal size, and displays a plurality of The image data in the display area are individually allocated and displayed.

As described above, in the information processing system 1 of the present embodiment, when the viewer control unit 207 receives from the user designation of a plurality of specimen regions to be displayed in acquired image data, the display space is designated for the plurality of specimen regions. The number of display spaces is divided into the same number as the number of display spaces, and based on the stored position information and the stored offset amount, the position of the feature point in the specimen area displayed in the plurality of display spaces is matched. , to calculate display areas to be displayed in a plurality of display spaces.

That is, the viewer control unit 207 matches the positions of the feature points in the multiple display spaces based on the offset amount between the feature points included in the multiple sample areas as metadata. can be done. In addition, by using metadata for setting the display area, the corresponding display area can be calculated accurately, and the display processing can be performed efficiently and at high speed.

[typical computer]
Next, the configuration of a typical computer that can be used for the image server 100 and viewer 200 will be described.

FIG. 19 is a diagram showing the hardware configuration of a typical computer 400. As shown in FIG.
As shown in the figure, the computer 400 includes a CPU (Central Processing Unit) 401 , a ROM (Read Only Memory) 402 and a RAM 403 . The computer 400 also includes an input device 404, an output device 405, a storage device 406, a media interface device 407, a network connection device 408, and a bus 409 connecting these devices.

The CPU 401 functions as an arithmetic processing device and a control device, and controls general operations of the computer 400 according to various programs. The ROM 402 stores programs, calculation parameters, and the like used by the CPU 401 . The RAM 403 temporarily stores programs executed by the CPU 401, parameters that change as appropriate during execution, and the like.

The synchronization processing unit 25 of the synchronization server 40, the viewer control units 37A and 37B of the viewers 30A and 30B, and the like are, for example, in the hardware configuration of the computer 400, the CPU 401, the programs stored in the ROM 402, the working area of the RAM 403, and the like. It is realized by

The input device 404 includes input means for the user to input information, such as a mouse, keyboard, touch panel, button, microphone, switch, and lever, and an input control circuit that generates an input signal based on the user's input and outputs it to the CPU 401 . etc. A user of the computer 400 can input various data to the CPU 401 and instruct processing operations by operating the input device 404 .

The output device 405 includes, for example, a display device such as a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD) device, an OLED (Organic Light Emitting Diode) device. In addition, output device 405 includes audio output devices such as speakers and headphones.

A storage device 406 is a device for storing programs and user data. The storage device 406 includes a storage medium and a read/write device that writes data to and reads data from the storage medium. The storage device 106 is configured by, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like.

The media interface device 407 is a reader/writer for storage media. The media interface device 407 reads and writes data from and to the attached removable recording medium 2A such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory.

A network connection device 408 is, for example, an interface for connecting to the network 300 . The network connection device 408 may be a device compatible with wireless LAN (Local Area Network), a device compatible with wireless USB, or a wire communication device that performs wired communication.

[Modification]
FIG. 20 is a diagram showing a configuration of an information processing device according to a modification.
In the above embodiment, the information processing system 1 including the image server 100 and one or more viewers 200A and 200B that can be connected to each other via the network 300 has been described. On the other hand, one information processing device 500 can also be employed as an information processing system. The information processing apparatus 500 has the same functional configuration as the image server 100 and the viewer 200 of the above embodiment, but differs in that it does not have the server control unit 107, the server communication unit 108, and the viewer communication unit 204. . Furthermore, the control unit 207A has the same function as the viewer control unit 207 of the above-described embodiment, and directly receives data from the slide image storage unit 101, thumbnail image storage unit, offset coordinate storage unit 104, and triangle/matrix storage unit 106. read out. The information processing device 500 may be configured by the typical computer 400 described above, for example. This information processing apparatus 500 can also perform processing similar to that of the above-described embodiment.

Note that the present technology can also adopt the following configuration.
(1) an acquisition unit that acquires image data obtained by photographing a slide on which a plurality of sections obtained by cutting one specimen in the same direction are discretely mounted;
Detecting a plurality of specimen regions having the same shape and including each of the sections in the acquired image data, and calculating position information relatively indicating the position of each of the specimen regions in the coordinate space of the image data. a detection unit;
a first storage unit that stores the calculated position information;
an information processing system comprising: a control unit that switches display between the specimen regions based on the stored position information.

(2) The information processing system according to (1) above,
further comprising a generation unit that generates reduced data obtained by lowering the resolution of the image data;
The information processing system, wherein the detection unit detects the plurality of specimen regions in the image data by detecting the plurality of specimen regions from the reduced data.

(3) The information processing system according to any one of (1) to (2) above,
a plurality of image data of the plurality of specimen regions are managed in cutting order;
When the control unit receives from a user a designation of the specimen region to be displayed in the acquired image data and a range therein, the control unit calculates the range in the designated specimen region as a display region, When the display area is located at the end of the specimen area and an instruction to move the display area outside the end is received from the user, the display area is displayed next to the display area in cutting order based on the stored position information. An information processing system that switches to the display of the sample area.

(4) The information processing system according to any one of (1) to (3) above,
When the calculated display area is at the end of the specimen area in one axial direction of the plurality of specimen areas in the coordinate space of the image data, the control unit moves the display area outside the end. An information processing system that, upon receiving an instruction from a user to move in one axial direction, switches to display of an adjacent specimen region in the cutting order while fixing the position in the other axial direction.

(5) The information processing system according to any one of (1) to (4) above,
The acquisition unit acquires image data obtained by photographing a slide on which a plurality of sections obtained by cutting one specimen in the same direction are arranged discretely in a row, and
When the calculated display area is at the end of the sample area in the direction in which the plurality of sample areas are aligned in the coordinate space of the image data, the control unit moves the display area ahead of the end. an information processing system for calculating a display area of the specimen area of the movement destination based on the stored position information when an instruction to move the specimen area in the direction is received from the user.

(6) The information processing system according to any one of (1) to (5) above,
an alignment unit that detects feature points in each of the plurality of specimen regions and calculates an offset amount of the coordinates of each of the feature points in the coordinate space of the specimen region;
a second storage unit that stores the calculated offset amount,
When receiving an instruction from the user to jump the display area to another sample area, the control unit determines the display area before and after the jump based on the stored position information and the stored offset amount. an information processing system that calculates the display area of a jump destination such that the positions of the feature points of the two match.

(7) The information processing system according to any one of (1) to (6) above,
The alignment unit calculates an affine transformation matrix for matching the detected feature points in the plurality of specimen regions in a display space, and calculates the offset amount based on the calculated affine transformation matrix. processing system.

(8) The information processing system according to any one of (1) to (7) above,
The control unit divides the display space into a number of display spaces equal to the number of the designated plurality of specimen regions when receiving from the user designation of a plurality of specimen regions to be displayed in the acquired image data. , based on the stored position information and the stored offset amount, the plurality of display spaces so that the positions of the feature points within the specimen region displayed in the plurality of display spaces match; An information processing system that calculates the display area to be displayed on the display.

(9) The acquisition unit acquires image data obtained by photographing a slide on which a plurality of sections obtained by cutting one specimen in the same direction are discretely placed,
A position in which the detecting unit detects a plurality of specimen regions having the same shape and including each of the sections in the acquired image data, and relatively indicates the position of each of the specimen regions in the coordinate space of the image data. calculate the information,
A first storage unit stores the calculated position information,
An information processing method, wherein a control unit switches display between the specimen regions based on the stored position information.

DESCRIPTION OF SYMBOLS 1... Information processing system 103... Specimen region detection part 104... Offset coordinate storage part 109... Image acquisition part 207... Viewer control part

Claims (41)

an acquisition unit that acquires first pathological image data including the first specimen region and the second specimen region;
an image generator that generates second pathological image data having a resolution lower than that of the first pathological image data based on the first pathological image data;
A first feature point of the first specimen region is extracted from the second pathological image data, and a first feature point corresponding to the first feature point of the second specimen region is extracted from the second pathological image data. an extraction unit that extracts 2 feature points;
a specifying unit that specifies a first region of the first pathological image data based on the first feature points, and specifies a second region of the first pathological image data based on the second feature points; ,
A pathological image display system comprising: a display control unit that performs control to synchronously display the first region of the first pathological image data and the second region of the first pathological image data. The pathological image display system according to claim 1,
dividing the first specimen region and the second specimen region into a plurality of triangles having the feature points as vertices, and generating the first pathological image data of the first region and the second specimen region based on the triangles; a pathological image display system that generates information for calculating an offset between corresponding points of the region of the first pathological image data and the first pathological image data. The pathological image display system according to claim 1 or 2,
A pathological image display system, wherein a point at which a change in luminance value exceeds a predetermined threshold is defined as the feature point. The pathological image display system according to any one of claims 1 to 3,
A pathological image display system that determines feature points of the second specimen region corresponding to feature points of the first specimen region using image correlation. The pathological image display system according to any one of claims 1 to 4,
A pathological image display system that calculates an offset amount of the second specimen region corresponding to the first specimen region based on the feature points. The pathological image display system according to claim 5,
The pathological image display system, wherein the offset amount is positional information of the first specimen region and the second specimen region in the coordinate space of the first pathological image data. The pathological image display system according to claim 5 or 6,
A pathological image display system that controls to store the calculated offset amount in a storage unit. The pathological image display system according to any one of claims 1 to 7,
A pathological image display system for extracting a plurality of sample regions from the second pathological image data and grouping the plurality of sample regions having a high degree of similarity. The pathological image display system according to any one of claims 1 to 8,
A pathological image display system for extracting a plurality of sample regions from the second pathological image data and setting consecutive numbers to the plurality of sample regions having a high degree of similarity. The pathological image display system according to claim 9,
The pathological image display system, wherein the consecutive numbers are changeable by a user. The pathological image display system according to any one of claims 1 to 10,
The display control unit displays a first display space for displaying the first region of the first pathological image data and the second region of the first pathological image data on a display screen. setting a second display space for displaying the center position of the first region of the first pathological image data in the first display space and the second display space of the second region of the first pathological image data; A pathological image display system configured to match and display the center positions in the display space of the . The pathological image display system according to claim 11,
The pathological image display system, wherein the first display space and the second display space are equal in size. The pathological image display system according to any one of claims 1 to 11,
A pathological image display system, further comprising an imaging unit that captures an image of a specimen obtained from a patient to obtain the first pathological image data. The pathological image display system according to any one of claims 1 to 13,
The pathological image display system, wherein the first pathological image data are images of a plurality of slices obtained by cutting one sample in the same direction. obtaining first pathological image data including a first specimen region and a second specimen region;
generating second pathological image data having a resolution lower than that of the first pathological image data based on the first pathological image data;
A first feature point of a first specimen region is extracted from the second pathological image data, and a second feature point corresponding to the first feature point of a second specimen region is extracted from the second pathological image data. extract feature points,
identifying a first region of the first pathological image data based on the first feature points, identifying a second region of the first pathological image data based on the second feature points;
A pathological image display method, comprising: synchronously displaying the first region of the first pathological image data and the second region of the first pathological image data. The pathological image display method according to claim 15,
dividing the first specimen region and the second specimen region into a plurality of triangles having the feature points as vertices, and generating the first pathological image data of the first region and the second specimen region based on the triangles; A pathological image display method for generating information for calculating an offset between points corresponding to the first pathological image data in a region of . The pathological image display method according to claim 15 or 16,
A pathological image display method, wherein a point at which a change in luminance value exceeds a predetermined threshold is defined as the feature point. The pathological image display method according to any one of claims 15 to 17,
A pathological image display method, wherein feature points of the second specimen region corresponding to feature points of the first specimen region are determined using image correlation. The pathological image display method according to any one of claims 15 to 18,
A pathological image display method comprising: calculating an offset amount of the second specimen region corresponding to the first specimen region based on the feature points. The pathological image display method according to claim 19,
The pathological image display method, wherein the offset amount is positional information of the first specimen region and the second specimen region in a coordinate space of the first pathological image data. The pathological image display method according to claim 19 or 20,
A pathological image display method for controlling to store the calculated offset amount in a storage unit. The pathological image display method according to any one of claims 15 to 21,
A pathological image display method, comprising: extracting a plurality of sample regions from the second pathological image data; and grouping the plurality of sample regions having a high degree of similarity. The pathological image display method according to any one of claims 15 to 22,
A pathological image display method comprising: extracting a plurality of sample regions from the second pathological image data; and setting consecutive numbers to the plurality of sample regions having a high degree of similarity. The pathological image display method according to claim 23,
The pathological image display method, wherein the consecutive numbers can be changed by a user. The pathological image display system according to any one of claims 15 to 24,
a first display space for displaying the first region of the first pathological image data on a display screen; and a second display for displaying the second region of the first pathological image data. A space is set, and a center position of the first region of the first pathological image data in the first display space and a center position of the second region of the first pathological image data in the second display space are determined. A pathological image display method for matching and displaying . The pathological image display method according to claim 25 ,
The pathological image display method, wherein the first display space and the second display space are equal in size. The pathological image display method according to any one of claims 15 to 26,
A pathological image display method, comprising capturing an image of a specimen obtained from a patient and obtaining the first pathological image data. The pathological image display method according to any one of claims 15 to 27,
The pathological image display method, wherein the first pathological image data are images of a plurality of slices obtained by cutting one sample in the same direction. obtaining first pathological image data including a first specimen region and a second specimen region;
generating second pathological image data having a resolution lower than that of the first pathological image data based on the first pathological image data;
A first feature point of the first specimen region is extracted from the second pathological image data, and a first feature point corresponding to the first feature point of the second specimen region is extracted from the second pathological image data. 2 feature points are extracted, a first region of the first pathological image data is specified based on the first feature points, and a second region of the first pathological image data is specified based on the second feature points. identify the region of
A program for operating a computer to synchronously display the first region of the first pathological image data and the second region of the first pathological image data. 30. A program according to claim 29,
dividing the first specimen region and the second specimen region into a plurality of triangles having the feature points as vertices, and generating the first pathological image data of the first region and the second specimen region based on the triangles; a program for operating a computer to generate information for calculating offsets between corresponding points of the region of the first pathological image data and the first pathological image data. 31. The program according to claim 29 or 30,
A program that causes a computer to operate such that a point where a change in luminance value exceeds a predetermined threshold is defined as the feature point. The program according to any one of claims 29 to 31,
A program that causes a computer to operate to determine feature points of the second specimen region that correspond to feature points of the first specimen region using image correlation. The program according to any one of claims 29 to 32,
A program that causes a computer to operate to calculate an offset amount of the second specimen region corresponding to the first specimen region based on the feature points. 34. The program of claim 33,
The offset amount is positional information of the first specimen region and the second specimen region in the coordinate space of the first pathological image data. 35. The program according to claim 33 or 34,
A program for operating a computer to control and store the calculated offset amount in a storage unit. The program according to any one of claims 29-35,
A program that causes a computer to operate so as to extract a plurality of sample regions from the second pathological image data and group the plurality of sample regions having a high degree of similarity. The program according to any one of claims 29-35,
A program that causes a computer to operate so as to extract a plurality of sample regions from the second pathological image data and set consecutive numbers to the plurality of sample regions having a high degree of similarity. 38. The program of claim 37,
The sequential number is user-changeable. The program. The program according to any one of claims 29-38,
a first display space for displaying the first region of the first pathological image data on a display screen; and a second display for displaying the second region of the first pathological image data. A space is set, and a center position of the first region of the first pathological image data in the first display space and a center position of the second region of the first pathological image data in the second display space are determined. A program that causes a computer to operate in such a way that it matches and displays to each other. 40. A program according to claim 39 ,
The program, wherein the first display space and the second display space are equal in size to each other. A program according to any one of claims 29-40,
The program, wherein the first pathological image data are images of a plurality of sections obtained by cutting one sample in the same direction.

Images