JP6721046B2 - Image processing apparatus, image processing method and program - Google Patents

Description

The present invention relates to an image processing device, an image processing method, and a program.

BACKGROUND ART So-called pathological diagnosis has been actively performed by observing a tissue section collected from an organism such as a person or an animal with a microscope to diagnose the presence or absence of a lesion and the type of the lesion. In the pathological diagnosis, first, the collected tissue is dehydrated to fix it, and after being subjected to a treatment such as blocking with paraffin, it is cut into slices with a thickness of 2 to 8 μm, the paraffin is removed, stained, and observed under a microscope to examine cells. Image data of an image is generated (photographed), and morphological information such as changes in size and shape of cell nuclei, changes in tissue pattern, and staining information are analyzed to make a diagnosis.

Further, in recent years, many techniques for improving the efficiency of image processing on image data have been proposed. For example, in Patent Document 1, when a position on the display screen is designated according to an operation of a mouse or the like, a free curve or a polygon is drawn from a start point on the display screen according to the designation, and a final drawing point is drawn. A technique for drawing a free-form curve or a polygon of a closed curve by returning to the starting point and matching is described.

JP, 9-326038, A

By the way, in a pathological image in which various cells are intricate, it is difficult to generate image data of only desired cells, and at the time of analysis, a region of interest (ROI: Region of Interest) is set as a region to be analyzed from the displayed image. Interest), that is, setting an area in the image.
Such work is generally performed using the technique for drawing a closed curve described in Patent Document 1 above.
However, for example, as shown in FIG. 10A, when a curved line L41 indicated by a two-dot chain line is drawn in an image in which a region of interest exists over two continuous sides t11 and t12, the curved line L41 is a straight line indicated by a one-dot chain line. It closes at L42.
Further, for example, as shown in FIG. 10B, when a curved line L51 indicated by a chain double-dashed line is drawn in an image in which a region of interest exists across two opposite sides t11, t13, the curved line L51 is a straight line indicated by a dash-dotted line. It closes at L52.
Therefore, for example, in the case of an image in which the region of interest spreads out of the displayed image as shown in FIGS. 10A and 10B, a work of surrounding the image manually occurs to surround the desired region. It was time-consuming.

An object of the present invention is to provide an image processing device, an image processing method, and a program that can reduce the trouble when setting an area in an image.

In order to solve the above problems, according to the image processing device of claim 1 of the present invention,
Designating means for designating a locus from a start point to an end point on the image displayed on the display means,
A closed area setting means for setting a closed area based on the locus from the start point to the end point designated by the designating means,
An image processing apparatus, wherein the closed region setting means sets a region surrounded by the trajectory and an edge portion of the image as the closed region when a start point and an end point are designated outside the image. Will be provided.

Further, according to the image processing apparatus of claim 2 of the present invention,
When the start point and the end point are specified outside the image, the closed region setting means selects a route along the edge of the image so that the area of the closed region is minimized. An image processing apparatus according to claim 1 is provided.

Further, according to the image processing device of claim 3 of the present invention,
The closed area setting means displays a plurality of candidate areas that can be set as the closed area on the image when a start point and an end point are specified outside the image,
The image processing apparatus according to claim 1, wherein a candidate area selected by an operator's selection operation is set as the closed area.

Further, according to the image processing apparatus of claim 4 of the present invention,
When the start point and the end point are designated outside the image, the closed region setting means calculates a predetermined feature amount for each of a plurality of candidate regions that can be set as the closed region, and the candidate region having the highest feature amount is calculated. The image processing apparatus according to claim 1, wherein is set as the closed region.

Further, according to the image processing device of claim 5 of the present invention,
The designating means is capable of designating a second trajectory from a second start point to an end point on the image in which the closed region is set,
When the start point and the end point of the second time are designated outside the image, the closed region setting means sets the region surrounded by the second trajectory and the edge of the image as the second closed region. At this time, the image processing apparatus according to claim 1, wherein an area that does not overlap with the closed area is set as the second closed area.

Further, according to the image processing device of claim 6 of the present invention,
The image processing apparatus according to claim 1, wherein the image is a medical image.

According to the image processing method of claim 7 of the present invention,
A closed area setting step of setting a closed area based on the locus from the start point to the end point designated by the designating means on the image displayed on the display means,
In the closed area setting step, when a start point and an end point are specified outside the image, an area surrounded by the locus and the edge of the image is set as the closed area. Will be provided.

According to the program of claim 8 of the present invention,
Computer
The closed area setting means sets the closed area based on the locus from the start point to the end point designated by the designating means on the image displayed on the display means,
Provided by a program, wherein the closed region setting means sets a region surrounded by the trajectory and an edge portion of the image as the closed region when a start point and an end point are designated outside the image. To be done.

According to the present invention, it is possible to reduce the trouble when setting a region in an image.

It is a figure which shows roughly the structure of a pathological diagnosis support system. It is a block diagram which shows roughly the functional structure of an image processing apparatus. It is a flow chart which shows a flow of image field setting processing roughly. It is a figure for explaining image field setting processing. It is a figure for explaining image field setting processing. It is a figure for explaining image field setting processing. It is a figure for explaining image field setting processing. It is a figure for explaining image field setting processing. It is a figure for explaining image field setting processing. It is a figure for demonstrating the modification of an image area setting process. It is a figure for demonstrating the modification of an image area setting process. It is a figure for demonstrating the modification of an image area setting process. It is a figure for demonstrating the modification of an image area setting process. It is a figure for demonstrating the conventional problem. It is a figure for demonstrating the conventional problem.

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples.

<Configuration of pathological diagnosis support system 10>
FIG. 1 shows an example of the overall configuration of a pathological diagnosis support system 10 including the image processing apparatus of the present invention.
The pathological diagnosis support system 10 acquires a microscopic image of a tissue section of a human body stained with a predetermined staining reagent, and analyzes the acquired microscopic image to detect the expression of a specific biological substance in the tissue section of the observation target. This is a system that outputs a quantitatively expressed feature amount.

As shown in FIG. 1, the pathological diagnosis support system 10 is configured by connecting a microscope image acquisition apparatus 1A and an image processing apparatus 2A so that data can be transmitted and received via an interface such as a cable 3A. The pathological diagnosis support system 10 may be a device in which the microscope image acquisition device 1A and the image processing device 2A are integrally formed.
The connection method between the microscope image acquisition apparatus 1A and the image processing apparatus 2A is not particularly limited. For example, the microscope image acquisition apparatus 1A and the image processing apparatus 2A may be connected by a LAN (Local Area Network) or may be wirelessly connected.

The microscope image acquisition apparatus 1A is a known optical microscope with a camera, and acquires a microscope image of a tissue section on a slide mounted on a slide fixing stage and transmits it to the image processing apparatus 2A.
The microscope image acquisition device 1A includes an irradiation unit, an image forming unit, an image pickup unit, a communication I/F, and the like. The irradiation unit includes a light source, a filter, and the like, and irradiates the tissue section on the slide mounted on the slide fixing stage with light. The image forming means is composed of an eyepiece lens, an objective lens, and the like, and forms an image of transmitted light, reflected light, or fluorescence emitted from the tissue section on the slide by the irradiated light. The imaging means is a microscope-installed camera that includes a CCD (Charge Coupled Device) sensor or the like and that captures an image formed on the imaging surface by the imaging means to generate digital image data of a microscope image. The communication I/F transmits the image data of the generated microscope image to the image processing device 2A.
The microscope image acquisition apparatus 1A is provided with a bright field unit in which an irradiation unit and an imaging unit suitable for bright field observation are combined, and a fluorescence unit in which an irradiation unit and an imaging unit suitable for fluorescence observation are combined. Bright field/fluorescence can be switched by switching.

Note that the microscope image acquisition apparatus 1A is not limited to a microscope with a camera, and for example, a virtual microscope slide creation apparatus (for example, a special microscope that scans a slide on a slide fixing stage of the microscope to acquire a microscope image of the entire tissue section). Table 2002-514319 gazette) etc. may be used. According to the virtual microscope slide creation apparatus, it is possible to acquire image data in which the entire image of the tissue section on the slide can be viewed at once on the display unit.

The image processing apparatus 2A analyzes the microscope image transmitted from the microscope image acquisition apparatus 1A.
As shown in FIG. 2, the image processing apparatus 2A includes a control unit 21, an operation unit 22, a display unit 23, a communication I/F 24, a storage unit 25, and the like, and each unit is connected via a bus 26. There is.

The control unit 21 is configured to include a CPU (Central Processing Unit), a RAM (Random Access Memory), and the like, executes various processes in cooperation with various programs stored in the storage unit 25, and the image processing apparatus 2A. Control the operation of.
For example, the control unit 21 executes the image area setting process (see FIG. 3) in cooperation with the image processing program stored in the storage unit 25, and realizes the function as the closed area setting unit.

The operation unit 22 includes a keyboard having character input keys, number input keys, various function keys, and the like, and a pointing device such as a mouse or a touch pen. And the operation signal according to are output to the control unit 21 as input signals. The operation unit 22 functions as a specifying unit that specifies a locus from a start point to an end point on the image displayed on the display screen.

The display unit 23 includes a monitor such as a CRT (Cathode Ray Tube) or an LCD (Liquid Crystal Display), and displays various display screens according to the instruction of the display signal input from the control unit 21. Functions as a means.

The communication I/F 24 is an interface for transmitting/receiving data to/from an external device such as the microscope image acquisition device 1A.

The storage unit 25 is composed of, for example, an HDD (Hard Disk Drive) or a semiconductor nonvolatile memory. The storage unit 25 stores various programs and various data as described above.

In addition, the image processing apparatus 2A may include a LAN adapter, a router, etc., and may be connected to an external device via a communication network such as a LAN.

In the present embodiment, the image processing device 2A performs analysis using the image transmitted from the microscope image acquisition device 1A.
The image analyzed in the present embodiment is an image obtained by photographing a biological material in an arbitrary tissue sample prepared by a known method such as a tissue section collected from a human body. Specific examples of the biological substance include biological substances such as cell nuclei and cell membranes. The biological substance may or may not be dyed, and when dyeing, any known reagent and dyeing method may be used.

Specifically, the image analyzed in the present embodiment is, for example, a microscope image acquisition apparatus for a tissue sample stained with a hematoxylin-staining reagent (H staining reagent) or a hematoxylin-eosin staining reagent (HE staining reagent). 1A is a microscope image obtained by magnifying and imaging in bright field in 1A. Hematoxylin (H) is a blue-violet pigment that stains biological materials such as cell nuclei, bone tissues, part of cartilage tissues, and serum components (basophil tissues). Eosin (E) is a red to pink pigment that stains biological substances such as cytoplasm, connective tissue of soft tissue, erythrocytes, fibrin, endocrine granules (e.g., eosinophilic tissue).
The image analyzed in the present embodiment may be a fluorescence image obtained by capturing an image of a tissue sample in which a biological substance is stained with a fluorescent substance by using a fluorescence microscope.

<Operation of Pathological Diagnosis Support System 10 (Including Image Processing Method)>
The operation of the pathological diagnosis support system 10 for acquiring the above-described image and performing image processing using the image processing method of the present invention will be described below.
The observation target here includes, for example, a tissue sample stained with an H staining reagent, but is not limited to this.

First, the operator places the tissue section on the slide and stains it with the H staining reagent.
Then, the microscope image acquisition device 1A is used to acquire a bright field image according to the following procedure. (A1) The operator installs the slide on the slide fixing stage of the microscope image acquisition apparatus 1A.
(A2) The bright field unit is set, the imaging magnification and the focus are adjusted, and the observation target region on the tissue section is placed in the field of view.
(A3) Imaging is performed by the imaging means to generate image data of the bright field image, and the image data is transmitted to the image processing device 2A.

Then, in the image processing apparatus 2A, the image area setting process shown in FIG. 3 is executed by the cooperation of the control unit 21 and the image processing program stored in the storage unit 25.
Specifically, in the image processing apparatus 2A, when the image data is received from the microscope image acquisition apparatus 1A by the communication I/F 24, the control unit 21 creates a bright field image in which the tissue-stained cell region is emphasized. Then, it is displayed on the display screen of the display unit 23.
When the operator draws a locus of a curve or a straight line (specifies a locus) on the displayed bright-field image using the region setting tool, the control unit 21 executes the following image region setting process. ..

First, when a locus is drawn on an image by the operation of the operation unit 22 (mouse or the like) by the operator, the control unit 21 acquires the coordinates of the locus designated along with the movement of the operation unit 22 (step S1).

Next, the control unit 21 determines whether or not the acquired number of coordinates is 3 points or more (step S2), and when the number of acquired coordinates is less than 3 points (step S2: NO), this processing ends.

On the other hand, when the acquired number of coordinates is three points or more (step S2: YES), the control unit 21 determines whether the starting point of the locus is outside the screen (step S3), and the starting point is not outside the screen. (Step S3: NO), this processing ends.

On the other hand, when the start point of the trajectory is outside the screen (step S3: YES), the control unit 21 determines whether the end point is outside the screen (step S4), and when the end point is not outside the screen (step S4). : NO), this processing ends.

On the other hand, when the end point of the trajectory is outside the screen (step S4: YES), the control unit 21 determines whether or not points other than the start point and the end point are in the image (step S5), and the points other than the start point and the end point are determined. If the point is not in the image (step S5: NO), this process ends.

On the other hand, when the other points are in the image (step S5: YES), the control unit 21 determines whether the start point and the end point (end point) are outside the same side of the image (step S6). ..

Then, when the start point and the end point are outside the same side of the image (step S6: YES), the control unit 21 ends this processing.
In this case, the normal function of the area setting tool connects the start point and the end point, and the area surrounded by the locus and the edge of the image is set as the closed area in the area surrounded by the start point and the end point. It will be.

On the other hand, when the start point and the end point are not outside the same side of the image (step S6: NO), the control unit 21 determines whether each of the start point and the end point is outside two consecutive sides of the image. Yes (step S7).

Then, when each of the start point and the end point is outside the continuous two sides of the image (step S7: YES), the control unit 21 sets a new coordinate 1 at a position including a vertex at which the continuous two sides of the image intersect. A point is added (step S8: closed region setting step), and this processing ends.

Here, FIG. 4A shows an example in which the locus L11 is designated so that the start point P1 and the end point P2 are outside the two continuous sides t11 and t12 of the image.
In this case, as shown in FIG. 4B, a new point P3 is added slightly outside the vertex X1 at which the two continuous sides t11 and t12 intersect.
As a result, as shown in FIG. 4B, the locus L11 and the edge portion of the image in the area surrounded by the locus L11, the straight line L12 connecting the start point P1 and the point P3, and the straight line L13 connecting the end point P2 and the point P3. The area surrounded by is set as the closed area R1.

Note that in the present embodiment, as shown in FIG. 4B, the start point P1 and the end point P2 are formed at positions separated from the edge portions (sides t11 and t12) of the image, and the start point P1 and the end point P2 are the sides of the image. A straight line L12 and a straight line L13 (parallel to the sides of the image) are formed, so that the straight line L12 and the straight line L13 are separated from the edge of the image and are surrounded by the locus L11, the straight line L12, and the straight line L13. In the closed area, the area surrounded by the locus L11 and the edge of the image is set as the closed area R1.
However, instead of the start point P1 and the end point P2, the intersection of the locus L11 and the edge of the image may be obtained, and a straight line along the side of the image may be created from the intersection. In this case, a new point P3 is added on the vertex X1, and the straight line L12 and the straight line L13 overlap the edge of the image.

Returning to FIG. 3, when the start point and the end point are not outside the two continuous sides of the image (step S7: NO), the control unit 21 has the start point and the end point outside the two opposite sides of the image. It is determined whether or not (step S9).

Then, when the start point and the end point are not outside the two opposite sides of the image (step S9: NO), the control unit 21 ends this processing.

On the other hand, when each of the start point and the end point is outside the two opposite sides of the image (step S9: YES), the control unit 21 sets the start point and the end point in two directions along the side of the image (clockwise and counterclockwise). The distance when traveling in the clockwise direction is calculated (step S10).

Then, the control unit 21 selects a route along the edge of the image so that the area of the closed region is minimized, that is, a new route is added to a position including the apex on the route having the smaller calculated distance. Two coordinates are added (step S11: closed region setting step), and this process ends.

Here, FIG. 5A shows an example in which the locus L21 is designated such that the start point P1 and the end point P2 are outside the two opposite sides t11 and t13 of the image.
In this case, as shown in FIG. 5B, new points P4 and P5 are added slightly outside the vertices X2 and X3 on the path having the smaller calculated distance.
As a result, as shown in FIG. 5B, in the area surrounded by the locus L21, the straight line L22 connecting the start point P1 and the point P4, the straight line L23 connecting the point P4 and the point P5, and the straight line L24 connecting the end point P2 and the point P5. The area surrounded by the locus L21 and the edge of the image is set as the closed area R2.

In the present embodiment, as shown in FIG. 5B, the start point P1 and the end point P2 are formed at positions separated from the edge portions (sides t11 and t13) of the image, and the start point P1 and the end point P2 are the sides of the image. Since the straight line L22, the straight line L23, and the straight line L24 (which are parallel to the sides of the image) are formed along the line L22, the straight line L23, and the straight line L24 are separated from the edge of the image, the locus L21, the straight line A region surrounded by the locus L21 and the edge of the image in the region surrounded by L22, the straight line L23, and the straight line L24 is set as the closed region R2.
However, instead of the start point P1 and the end point P2, the intersection point of the locus L21 and the edge portion of the image may be obtained, and the route along the side of the image may be created from the intersection point. In this case, new points P4 and P5 are added on the vertices X2 and X3, and the straight line L22, the straight line L23, and the straight line L24 overlap the edge portion of the image.

As described above, in the present embodiment, the operation unit 22 that specifies the locus from the start point to the end point on the image displayed on the display unit 23 and the locus from the start point to the end point that is specified by the operation unit 22. And a control unit 21 that sets a closed region based on the closed region. When the start point and the end point are specified outside the image, the control unit 21 sets the region surrounded by the locus and the edge of the image as the closed region. Set.
Therefore, for example, when surrounding a region of interest in an image in which the region of interest spreads out of the displayed image, it is not necessary to manually surround the periphery of the image, and the time and effort to set the region in the image are reduced. can do. In addition, it is possible to prevent errors due to manual work. Therefore, working efficiency can be improved.

Further, in the present embodiment, when the start point and the end point are specified outside the image, the control unit 21 selects a route along the edge of the image so that the area of the closed region is minimized.
Therefore, in particular, it is possible to reduce the time and labor required to surround a small area at the corner portion or one end portion of the image, and it is possible to improve work efficiency.

Further, in the present embodiment, the image displayed on the display screen is a medical image.
Therefore, particularly in the analysis of a pathological image in which various cells are intricate, the region to be analyzed can be efficiently surrounded.

In the above-described embodiment, the process of automatically setting the area of the closed region to the minimum has been described as an example, but the closed region may be set by the operator's selection operation.
In this case, when the start point and the end point are designated outside the image, for example, as shown in FIGS. 6A and 6B, the control unit 21 sets a plurality of candidate regions K1 and K2 that can be set as closed regions on the image. The candidate area is displayed and selected by the operator's selection operation, and is set as a closed area.
With such a configuration, the intention of the operator can be more reflected.

In addition, when the start point and the end point are specified outside the image, the control unit 21 calculates a predetermined feature amount for each of a plurality of candidate regions that can be set as closed regions, and determines the candidate region with the highest feature amount as the closed region. It may be configured to set as.
Examples of the predetermined feature amount include the number of cells, the circularity of cells, color information such as RGB values, luminance values, and the thresholds thereof. Further, the feature amount is preset by the operator and can be changed as necessary.
FIG. 7 is an example in which the candidate region with the higher circularity of cells is set as the closed region R3.
With such a configuration, the intention of the operator can be more reflected.

Further, in the above-described embodiment, the configuration in which one closed region is set for the image has been described as an example, but the second locus is specified on the image in which the closed region is set, and the second closed region is set. A closed area can also be set.
Specifically, when the second locus is designated on the image in which the closed region (first closed region) is set and the start point and the end point of the second locus are outside the image, the control unit 21 Sets a region that does not overlap the first closed region as the second closed region.
For example, as shown in FIG. 8A, when the locus L31 is designated at the first time and the first closed region R5 is set at the right end portion in FIG. 8A, as shown in FIG. When the locus L32 is designated, the second closed region R6 is set for the remaining region of the first time.
It should be noted that in the image, the region remaining by setting the first closed region R5 and the second closed region R6 may be set as the region of interest.

Further, in the above embodiment, a rectangular image has been described as an example, but the shape of the image is not limited to this.
For example, as shown in FIG. 9, the image may have an elliptical shape or the like, or, although not shown, may have a pentagonal or higher polygonal shape.
In the case of an image having such a shape, the control unit 21 will add coordinates at predetermined intervals along the peripheral side of the image.

Further, in the above-described embodiment, a medical image used for pathological diagnosis has been described as an example, but it goes without saying that the type of image is not limited to this.

INDUSTRIAL APPLICABILITY The present invention is suitable for providing an image processing device, an image processing method, and a program that can reduce the trouble when setting a region in an image.

10 Pathological diagnosis support system 1A Microscope image acquisition device 2A Image processing device 3A Cable 21 Control unit (closed region setting means)
22 Operation part (designating means)
23 Display unit (display means)
24 Communication I/F
25 storage unit 26 buses L11, L21, L31, L32 loci L12, L13, L22, L23, L24 straight line L13 straight line P1 start point P2 end point R1, R2, R3, R5, R6 closed areas E3, E4 candidate areas

Claims (8)

Designating means for designating a locus from a start point to an end point on the image displayed on the display means,
A closed area setting means for setting a closed area based on the locus from the start point to the end point designated by the designating means,
An image processing apparatus, wherein the closed region setting means sets a region surrounded by the trajectory and an edge portion of the image as the closed region when a start point and an end point are designated outside the image. .. When the start point and the end point are specified outside the image, the closed region setting means selects a route along the edge of the image so that the area of the closed region is minimized. The image processing apparatus according to claim 1. The closed area setting means displays a plurality of candidate areas that can be set as the closed area on the image when a start point and an end point are specified outside the image,
The image processing apparatus according to claim 1, wherein a candidate area selected by an operator's selection operation is set as the closed area. When the start point and the end point are designated outside the image, the closed region setting means calculates a predetermined feature amount for each of a plurality of candidate regions that can be set as the closed region, and the candidate region having the highest feature amount is calculated. The image processing apparatus according to claim 1, wherein is set as the closed region. The designating means is capable of designating a second trajectory from a second start point to an end point on the image in which the closed region is set,
When the start point and the end point of the second time are designated outside the image, the closed region setting means sets the region surrounded by the second trajectory and the edge of the image as the second closed region. At this time, the image processing apparatus according to claim 1, wherein an area that does not overlap with the closed area is set as the second closed area. The image processing apparatus according to any one of claims 1 to 5, wherein the image is a medical image. A closed area setting step of setting a closed area based on the locus from the start point to the end point designated by the designating means on the image displayed on the display means,
In the closed area setting step, when a start point and an end point are specified outside the image, an area surrounded by the locus and the edge of the image is set as the closed area. .. Computer
The closed area setting means sets the closed area based on the locus from the start point to the end point designated by the designating means on the image displayed on the display means,
The program, wherein the closed region setting means sets a region surrounded by the trajectory and the edge of the image as the closed region when a start point and an end point are designated outside the image.

Images