JP6092336B1 - Image processing system, image processing method, and image processing program - Google Patents

Abstract

An image processing system, an image processing method, and an image processing program for efficiently specifying a target area in a photographed image obtained by photographing the same subject by a plurality of methods. An image processing device is connected to an output unit and a captured image storage unit that stores first and second captured images captured using the first and second methods for an object. Control unit 21. Then, the control unit 21 generates a first image in which the target area included in the first captured image is colored with the first color scheme and the background is blackened, and the target area included in the second captured image is Colored in a second color scheme different from the first color scheme to generate a second image with a blackened background, and in each of the first and second images, alignment is performed using the colored target regions, respectively. Then, a composite image colored with the third color scheme generated by the first color scheme and the second color scheme is generated and output. [Selection] Figure 1

Description

  The present invention relates to an image processing system, an image processing method, and an image processing program for visualizing a target area in captured images obtained by capturing the same subject by a plurality of methods.

  In order to confirm specific constituent cells in the specimen tissue, the state of the cells may be evaluated using an image obtained by photographing a cross section of the specimen tissue. Therefore, an image search system for searching cell images from a morphological viewpoint has been studied (for example, see Patent Document 1). In the image database of the image search system described in this document, cell images and cell morphological feature information are stored in association with each other. Then, the image database is searched based on the morphological characteristics of the cell input as the search condition, and the cell image that matches the search condition is displayed on the display unit.

  Furthermore, an image processing system for recognizing a specific constituent cell contained in a specimen in a slice image obtained by photographing a cross section of the specimen has been studied (for example, see Patent Document 2). In the technique described in this document, the control unit of the image processing apparatus records slice images and performs alignment in consecutive slice images before and after. Next, the control unit performs contour alignment for the other slice images with respect to the tissue contour specified in the first slice image. And a control part performs a two-dimensional evaluation process about the cell contained in each slice image. Next, the control unit identifies the same or separate cells for the cells specified in each image, executes a three-dimensional evaluation process for counting the number of cells, and outputs the number of cells.

JP 2003-30202 A JP 2014-6100 A

  In the medical and biomedical science fields, in order to observe tissue specimens from various angles, the same tissue may be photographed by a plurality of means. For example, if the same tissue is stained a plurality of times by a plurality of cell staining techniques, a plurality of images in which the presence of a plurality of substances is visualized can be obtained. In each of the plurality of images, the region where each target material exists is stained with a specific color, so that the presence of each material can be easily confirmed.

Here, there are cases where a region (target region) where a plurality of substances are present simultaneously is specified, or a region where only one of the plurality of substances is present is specified. In this case, it is necessary to compare and visually check the images, and it takes time and effort to confirm the images. Further, when there are many regions to be confirmed, there is a limit to the number that can be counted while visually comparing a plurality of images.
The present invention has been made in view of the above-described problems, and an object of the present invention is to provide an image processing system, an image processing method, and an image processing for visualizing a target area in captured images obtained by capturing the same subject by a plurality of methods. To provide a program.

An image processing system for solving the above problem includes a captured image storage unit that stores a plurality of captured images captured using a plurality of methods, and a control unit connected to the output unit. And the said picked-up image memory | storage part records the 1st picked-up image which image | photographed the said object with the 1st method, and the 2nd picked-up image which image | photographed the said object with the 2nd method. Control unit assigns a first color with respect to the first method, allocates a second color for different second approach and the first approach, the first being recorded in the captured image storage unit the target area included in one of the captured image colored with the first color, the target area included the background to generate a first image obtained by blackening, the recorded second captured image in the captured image storage unit Are colored in a second color scheme different from the first color scheme to generate a second image having a black background. Then, using said modified as mutual information is maximized, the first, the second image, overlay target regions are colored respectively in register position, the first color and the second A composite image colored with the third color scheme generated by the two color scheme is generated and output. Thereby, the target area | region contained in a 1st image and a 2nd image can be visualized using the synthesized image colored with the 3rd color scheme.

  In the image processing system, it is preferable that the first color scheme and the second color scheme are selected from three primary colors. Thereby, it is a kind of color expression method, and based on the three primary colors (RGB) of red (Green) and blue (Blue), it is possible to easily visually recognize the region where the color scheme is superimposed in the composite image. it can. In other words, each color region can be identified by acting as a color stimulus on three types of pyramidal cells in the human retina.

  In the image processing system, luminance information is acquired in the first and second captured images, and first and second inverted images in which a target area is whitened based on the luminance information are generated, and the first It is preferable to multiply the first and second inverted images by the values of the first color scheme and the second color scheme as a whole. Thereby, a target area | region can be colored efficiently.

-In the said image processing system, the pixel value of the pixel contained in the said 1st image and the pixel value of the pixel contained in the said 2nd image are further used for the said 1st, 2nd image. calculating a mutual information amount when deformed one, the mutual information amount using the modified such that the maximum will rows aligned between the first and second images. As a result, it is possible to align a plurality of images using the characteristics of the same subject.

  In the image processing system, the composite image is represented by an HSV color space using hue, saturation, and lightness, and using values specified in the HSV color space, It is preferable to specify the target area. Here, the hue represents the type of color, the saturation represents the vividness of the color, and the lightness represents the brightness of the color. Thereby, a color can be adjusted in consideration of a specific hue.

  -In the said image processing system, it is preferable to deform | transform a target area | region by performing the closing process by morphology operation in the said synthesized image, and to count the number of said target areas. Thereby, the deformation | transformation of the target area | region by the nonuniformity of a coloring area | region, a chip | tip, etc. can be corrected.

  In the image processing system, the first and second captured images are images obtained by capturing slice samples of the evaluation target tissue. In the first technique, the first and second captured images are included in the evaluation target tissue. The substance expression region is colored, and in the second method, the expression region of the second substance contained in the tissue to be evaluated is colored, and the region where the first substance and the second substance are expressed is the target area. It is preferable to detect as Thereby, a specific area | region (for example, pathological cell) can be specified in cellular tissue.

  According to the present invention, it is possible to efficiently specify a target area in a photographed image obtained by photographing the same subject by a plurality of methods.

It is explanatory drawing of the system of this embodiment, Comprising: (a) is a system outline, (b) is a picked-up image memory | storage part, (c) is explanatory drawing of a synthesized image memory | storage part. Explanatory drawing of the process sequence of this embodiment. Explanatory drawing of the process sequence of this embodiment. Explanatory drawing of the process sequence of this embodiment. It is explanatory drawing of the process sequence of this embodiment, Comprising: (a) is an alignment process, (b) is a cell recognition process, (c) It is explanatory drawing of the closing process by morphological operation. It is explanatory drawing of the image of this embodiment, (a) is the 1st image which dye | stained protein (alpha), (b) is the 2nd image which dye | stained protein (beta), (c) is the 3rd image which dye | stained protein (gamma) , (D) is a fourth image assigned red for the first image, (e) is a fifth image assigned green for the second image, and (f) is a sixth image assigned blue for the third image. Figure. 2A and 2B are explanatory diagrams of an image according to the present embodiment, in which FIG. 3A is a composite image that has not been aligned, and FIG.

  Hereinafter, an embodiment will be described with reference to FIGS. In this embodiment, an image processing system for specifying a target region (for example, a pathological cell) in an image obtained by photographing the same tissue with a plurality of means will be described.

  Here, as in Patent Document 2, it is assumed that the number of specific cells (second solid object) included in the evaluation target tissue (first solid object) is counted. For this purpose, slice specimens are prepared by slicing the evaluation target tissue at equal intervals (slice intervals). The constituent cells included in the tissue to be evaluated are assumed to have an ellipsoidal shape.

  In the present embodiment, specific cells are stained by a predetermined cell staining method in the cross section (slice specimen) of the tissue to be evaluated. In this case, in order to detect each detection region of a plurality of substances, each slice sample is stained a plurality of times by different cell staining methods. Specifically, there is a case where a different color dye is used, or a case where the color is removed after dyeing and the next dyeing is performed. Then, for each staining, an image obtained by magnifying a slice sample is prepared for a tissue colored with a different substance. In these images, since a plurality of types of information can be visualized and combined, they are hereinafter referred to as multimodal images.

As shown in FIG. 1A, an image processing apparatus 20 connected to an input unit 10 and an output unit 15 is used for this image processing.
The input unit 10 includes input means for inputting various instructions such as a keyboard and a pointing device. The output unit 15 includes output means for outputting information processing results, such as a display.

  The image processing apparatus 20 is a computer that calculates the number of cells of a specific tissue included in a plurality of slice images obtained by photographing a cross section of a tissue specimen. The image processing apparatus 20 includes a control unit 21, a captured image storage unit 22, a composite image storage unit 23, and a calculation information storage unit 24.

  The control unit 21 functions as a control unit including a CPU, a RAM, a ROM, and the like, and performs processing described later (processing including a management stage, an image acquisition stage, a color assignment stage, a position adjustment stage, a count stage, and the like). By executing an image processing program for this purpose, the control unit 21 functions as a management unit 210, a color assignment unit 211, a position adjustment unit 212, an image composition unit 213, a count unit 214, and the like.

  The management unit 210 acquires a slice image to be processed, and executes processing for controlling each unit described later. Similar to Patent Document 2, the management unit 210 includes a memory that temporarily stores a slice interval and a magnification when a slice image is created.

The color assigning unit 211 executes a process of assigning a color arrangement to a plurality of images obtained by photographing the same tissue.
The position adjustment unit 212 executes processing for adjusting the positions of a plurality of images obtained by imaging the same tissue. For example, the position of the shifted image is adjusted depending on the shooting position.
The image composition unit 213 performs a process for identifying a target area by compositing a plurality of images obtained by photographing the same tissue.

  The count unit 214 executes a process of counting the extraction area based on the target area in the composite image recorded in the composite image storage unit 23. In the present embodiment, similarly to the two-dimensional evaluation means described in Patent Document 2, processing for counting the number of images of specific cells photographed in each slice image is executed. Furthermore, in the same manner as the three-dimensional evaluation means described in Patent Document 2, a specific cell included in the evaluation target tissue is determined by determining the same or different specific cells in consideration of the three-dimensional structure of the evaluation target tissue. Execute the process of counting the number of cells. For this reason, in order to ensure the continuity of specific cells included in a plurality of slice images, alignment of specific cells included in the slice images and image range (contour) for counting specific cells in the slice images To identify.

  As illustrated in FIG. 1B, the captured image storage unit 22 stores captured image management data 220 for images (slice images) obtained by capturing each cross section obtained by slicing the evaluation target tissue. The captured image management data 220 is recorded when a slice image is designated by the user. The captured image management data 220 includes data relating to an image number, a staining code, and a slice image.

  Data relating to an identifier for specifying each slice image is recorded in the image number data area. As this image number, a sequential number in the order of the cross-sectional positions is set in order from the initial value (here, “0”).

In the staining code data area, data relating to an identifier for specifying the staining technique (first and second techniques) used for staining the cell tissue is recorded.
In the slice image data area, data relating to an image obtained by photographing the tissue structure at the cross-sectional position specified by the image number is recorded. In the present embodiment, a plurality of images (first and second captured images) associated with different staining codes are recorded at the same cross-sectional position.

  As shown in FIG. 1C, the composite image management unit 230 records composite image management data 230 obtained by combining the captured images. This composite image management data 230 is recorded when a composite image is generated. The composite image management data 230 includes data relating to an image number, a composite image, and a use determination flag.

In the image number data area, data relating to an identifier for specifying the cross-sectional position of each composite image is recorded. Here, the same number as the image number recorded in the captured image management data 220 is used.
In the composite image data area, data relating to the composite image of the tissue structure at the cross-sectional position specified by the image number is recorded. As will be described later, this synthesized image is applied to the technique described in Patent Document 2 to count specific cells.

  In the use determination flag data area, data relating to an identifier for determining whether or not the composite image can be used is recorded. Specifically, as in the three-dimensional evaluation process (Patent Document 2), in the three-dimensional cell counting process described later, this composite image is not used when there is a large deviation from other composite images and positioning cannot be performed. A non-use flag indicating that is recorded.

  The calculation information storage unit 24 records calculation information used for counting specific cells. Similar to Patent Document 2, deformation information management data, tissue contour management data, in-slice cell management data, and grouping management data are recorded in the calculation information storage unit 24. In the deformation information management data, information for deforming the slice image is recorded in order to ensure the continuity of the photographed specific cells. In the tissue outline management data, information for specifying a range in which specific cells are counted in the slice image is recorded. In the intra-slice cell management data, information for extracting specific cells included in each slice image is recorded. In the grouping management data, information for specifying a group of specific cells photographed with a plurality of slice images in consideration of the three-dimensional structure is recorded.

Next, in the image processing apparatus 20 configured as described above, a processing procedure in the case of evaluating a specific cell included in the evaluation target tissue will be described with reference to FIGS.
(Overview of the whole)
First, the overall outline will be described with reference to FIG. When evaluating small cells contained in the tissue to be evaluated, the tissue specimen is sliced at a predetermined interval (slice interval). Then, cells are stained in each cross section, and a slice image file photographed at a predetermined magnification is created. As described above, in the present embodiment, a plurality of stainings are performed using different cell staining methods, and imaging is performed for each staining. This slice image file is stored in a storage unit (hard disk or the like) of the image processing apparatus 20 or a storage medium. In this case, a plurality of slice images can be included in one file, or a plurality of slice image files in which file names are set in the sliced order can be used. Then, the image processing program of the image processing apparatus 20 is activated.

  In this case, first, the control unit 21 of the image processing apparatus 20 executes an image acquisition process (step S1-1). Specifically, the management unit 210 of the control unit 21 outputs to the output unit 15 an image designation screen for designating a slice image obtained by imaging the tissue to be evaluated. Here, the slice image to be evaluated recorded in the storage unit (hard disk or the like) of the image processing apparatus 20 or the storage medium is designated using the image designation screen. In this case, the management unit 210 acquires the designated slice image and assigns an image number based on the order included in the file or the order of the file names. Then, the management unit 210 generates captured image management data 220 in which the slice image associated with the image number is recorded and registers it in the captured image storage unit 22.

  FIG. 3 schematically shows a captured image used in the present embodiment. Here, it is assumed that two captured images 500 and 501 are used. The captured image 500 includes stained regions R1, R2, and R3, and the captured image 501 includes stained regions R1 and R3.

  Next, the control unit 21 of the image processing apparatus 20 executes a color assignment process (step S1-2). Specifically, the color assigning unit 211 of the control unit 21 assigns a color necessary for synthesis to the stained region in the image. Here, any one of the three primary colors is assigned for each staining method. In FIG. 3, the first color scheme (for example, RGB “red”) is used for the stained regions R1 to R3 of the captured image 500, and the second color scheme (for example, “green” for RGB) is used for the stained regions R1 and R3 of the captured image 501. ). Details of this processing will be described later with reference to FIG.

Next, the following processing is repeated for each cross-sectional position. In the present embodiment, images of the same cross-sectional position are specified by slice images associated with a common image number.
Here, the control unit 21 of the image processing apparatus 20 executes alignment processing (step S1-3). Specifically, the position adjustment unit 212 of the control unit 21 adjusts the positional deviation for each photographing of the same cell tissue stained by different staining methods. Details of this processing will be described later with reference to FIG.

  Next, the control unit 21 of the image processing apparatus 20 executes a composition process (step S1-4). Specifically, the image composition unit 213 of the control unit 21 synthesizes multimodal images at the same cross-sectional position to generate a composite image. In the image 502 of FIG. 3, the stained regions R1 and R3 where the captured image 500 and the captured image 501 are combined have a third color scheme (the combined color of “red” and “green”). On the other hand, the stained region R2 of the photographed image 500 maintains the original first color scheme (“red”). Details of this processing will be described later with reference to FIG.

Next, the control unit 21 of the image processing apparatus 20 performs a count process of the two-dimensional cells of the designated color (Step S1-5). Specifically, the count unit 214 of the control unit 21 counts the number of specific cells (particles) using the color area specified in the two-dimensional composite image. Specifically, the cell recognition processing described later is performed on the continuous slice images synthesized with the designated color to recognize cells inside the color region. Then, in the image 503 in FIG. 3, the stained regions R1 and R3 in which the third color scheme (“yellow” in which “red” and “green” are combined) are expressed are counted as specific cells. The two-dimensional evaluation process of Patent Document 2 is used for counting specific cells.
Then, the above process is repeated until all the cross-sectional positions are completed.

  Next, the control unit 21 of the image processing apparatus 20 executes a three-dimensional cell count process (step S1-6). Specifically, the count unit 214 of the control unit 21 aligns the synthesized continuous section images and counts the number of cells inside the three-dimensional tissue. For the count of specific cells, the three-dimensional evaluation process of Patent Document 2 is used.

(Color assignment process)
Next, the color assignment process (step S1-2) will be described with reference to FIG. Here, the dyed area in the image is assigned to a color necessary for synthesis.

  First, the control unit 21 of the image processing apparatus 20 executes a graying process (step S2-1). Specifically, the color assignment unit 211 of the control unit 21 deletes the color information of the captured image (color image) and sets only the luminance information. For example, in the image 510, it is assumed that the staining regions R1 to R3 are stained “red” with respect to a background color close to white. As in the image 511, by deleting the color information, the stained regions R1 to R3 are converted into an image having a color scheme close to black with respect to the background region using luminance.

  Next, the control unit 21 of the image processing apparatus 20 executes black and white reversal processing (step S2-2). Specifically, the color assignment unit 211 of the control unit 21 inverts black and white for a grayed image. In this case, the image is converted into an image having a color scheme in which the staining area is close to white and a background area is close to black. As in the image 512, the stained regions R1 to R3 are converted into a color close to white with respect to the background region.

  Next, the control unit 21 of the image processing apparatus 20 executes a designated color assignment process (step S2-3). Specifically, the color assignment unit 211 of the control unit 21 multiplies the entire black-and-white inverted image by the RGB value of the designated color corresponding to the shooting method. In this case, a dyed region having a color scheme close to white is converted into a designated color. On the other hand, the background area of a color close to black maintains black. In this case, as in the image 513, the dyed regions R1 to R3 are colored in a designated color (for example, “green” in the first color scheme) with respect to a background region having a color scheme close to black.

  Next, the control unit 21 of the image processing apparatus 20 performs background cut processing (step S2-4). Specifically, the color assignment unit 211 of the control unit 21 sets the area below the threshold value to black. In this embodiment, the threshold value is calculated by a known technique (for example, Otsu's method). Note that the user may specify the threshold while confirming the image. In this case, as in the image 514, each RGB value of the background region becomes the maximum value (255), and the background color is converted from a color close to black to black.

(Alignment processing)
Next, the alignment process (step S1-3) will be described with reference to FIG. Here, a plurality of images obtained by photographing the same tissue are aligned. Specifically, in the alignment of multimodal images, an affine transformation condition (parallel movement, rotation, resizing) that maximizes the mutual information amount is obtained.

  First, the control unit 21 of the image processing apparatus 20 executes the same position image specifying process (step S3-1). Specifically, the position adjustment unit 212 of the control unit 21 specifies the multimodal image in which the photographing number of the cross-sectional position to be processed is recorded and the color assignment processing is performed.

Then, the following processing is repeated for each affine transformation condition.
Here, the control unit 21 of the image processing apparatus 20 executes a mutual information amount calculation process (step S3-2). Specifically, the position adjustment unit 212 of the control unit 21 calculates the mutual information amount using the following equation.

Here, “ai” is the pixel value (0 to 255) of the image A, and “bj” is the pixel value (0 to 255) of the image B.
The simultaneous distribution function p (ai, bj) can be expressed as follows.

  The peripheral probability distribution functions p (ai) and p (bj) can be expressed as follows.

  Further, the two-dimensional histogram h (ai, bj) can be expressed as follows.

Then, the position adjustment unit 212 stores the calculated mutual information amount in the memory in association with the affine transformation conditions (parallel movement, rotation, resizing).
The above processing is repeated until all the affine transformation conditions are completed.

Next, the control unit 21 of the image processing apparatus 20 executes a process of specifying the position where the mutual information amount is the maximum value (step S3-3). Specifically, the position adjustment unit 212 of the control unit 21 specifies the maximum value among the mutual information amounts stored in the memory, and specifies the affine transformation condition associated with the maximum value.
Next, the control unit 21 of the image processing apparatus 20 executes a registration result recording process (step S3-4). Specifically, the position adjustment unit 212 of the control unit 21 acquires the specified affine transformation condition, completes the alignment of the multimodal image, generates a composite image, and records it in the composite image storage unit 23.

(Cell recognition process)
Next, with reference to FIG. 5B, the cell recognition process in the two-dimensional cell counting process (step S1-5) of the designated color will be described.
Here, first, the control unit 21 of the image processing apparatus 20 executes HSV conversion processing (step S4-1). Specifically, the count unit 214 of the control unit 21 projects the composite image on an HSV color space composed of hue (H), saturation (S), and brightness (V). Then, the count unit 214 displays the composite image on the output unit 15.

  Next, the control unit 21 of the image processing apparatus 20 executes HSV designation processing (step S4-2). Specifically, the count unit 214 of the control unit 21 outputs a color designation screen to the output unit 15. In this color designation screen, setting icons (for example, sliders) for designating hue (H), saturation (S), and brightness (V) are provided. In this color designation screen, a representative value of a color scheme (“yellow”) that is expressed by synthesis of designated colors (for example, “red”, “green”) is set as an initial value. Using this slider, a color area (HSV value) can be specified. The count unit 214 that has acquired the HSV value designated on the color designation screen changes the composite image displayed on the output unit 15 according to the HSV value.

  Next, the control unit 21 of the image processing apparatus 20 performs a color region extraction process in a specified range (step S4-3). Specifically, the count unit 214 of the control unit 21 extracts a color region (target region) in a specified range in the composite image according to the HSV value.

  Next, the control unit 21 of the image processing apparatus 20 executes a closing process based on a morphological operation (step S4-4). Specifically, the count unit 214 of the control unit 21 interpolates missing color regions in the specified range.

  As shown in an image 520 in FIG. 5C, a part of the stained region may be missing depending on the overlapping state of the multimodal images. In this case, the region lacking the colored region is joined by the closing process. Thereby, as shown to the image 521, a dyeing | staining area | region becomes a ring shape with a hole.

  Next, the control unit 21 of the image processing apparatus 20 executes a hole closing process (step S4-5). Specifically, the count unit 214 of the control unit 21 fills the inside of the ring-shaped color region with the same color. As a result, as shown in the image 523, the stained region is formed into a circular shape.

(Example)
6 and 7 show an example in which the present embodiment is applied to a tissue slice specimen.
FIGS. 6A to 6C are slice images that are stained with different antibodies and photographed in the same slice specimen. Here, FIG. 6 (a) is the first image stained with the protein α antibody, FIG. 6 (b) is the second image stained with the protein β antibody, and FIG. 6 (c) is the protein γ. It is the 3rd image dye | stained using the antibody. 6D to 6F are images obtained by coloring each image with a designated color. FIG. 6D is a fourth image colored by assigning red to the first image, FIG. 6E is a fifth image colored by assigning green to the second image, and FIG. It is the 6th image colored by assigning blue to the 3rd image.

  FIG. 7 is an explanatory diagram of a composite image of the fourth image to the sixth image colored with each specified color. FIG. 7A is a composite image that has not been aligned, and FIG. 7B is a composite image that has been aligned. As shown in FIG. 7A, when the alignment is not performed, each color area varies, and it is difficult to identify the target area. On the other hand, as shown in FIG. 7B, when the alignment is performed, the target area determined by the overlapping of the colored areas can be identified.

According to the image processing system of the present embodiment, the following effects can be obtained.
(1) In this embodiment, the control unit 21 of the image processing apparatus 20 executes a color assignment process (step S1-2), a registration process (step S1-3), and a composition process (step S1-4). . Thereby, in a plurality of stained images, a common existing area can be visualized and easily confirmed based on the composite color. In addition, the number of target areas can be counted using the composite color area.

  (2) In the color assignment process (step S1-2) of the present embodiment, any one of the three primary colors is assigned for each staining method. Thereby, based on RGB of three primary colors, the area | region where the color scheme was superimposed can be easily visually recognized in a synthesized image.

  (3) In the alignment process of this embodiment, a two-dimensional histogram h (ai, bj) is used. Thereby, based on the characteristic distribution of a multimodal image, a plurality of images can be aligned.

  (4) In the cell recognition process of the present embodiment, the control unit 21 of the image processing apparatus 20 executes an HSV conversion process (step S4-1). Accordingly, it is possible to adjust the color while checking the color change in consideration of the hue. When the RGB values are adjusted, the hue may change, but by using the HSV color space, color adjustment considering the three primary colors can be performed efficiently.

  Further, the control unit 21 of the image processing apparatus 20 performs a closing process by a morphological operation (step S4-4). This makes it possible to correct staining unevenness and misalignment in consideration of the shape of a specific cell (ellipsoidal shape).

Moreover, you may change the said embodiment as follows.
In the above embodiment, it is assumed that the number of specific cells included in the tissue slice specimen is counted using a multimodal image. The application target of the present invention is not limited to this, and can be applied to multimodal images obtained by photographing the same subject by a plurality of different methods. For example, the present invention can be used for magnetic resonance imaging (MRI), X-ray CT images, a plurality of inspection methods for structures (for example, buildings), and the like.

  Alternatively, multimodal images taken at different times on the same subject may be used. For example, the same subject is photographed at different times, and a different designated color is assigned to each photographed image. Thereby, the change with time of the target region can be confirmed.

  In the above embodiment, the control unit 21 of the image processing apparatus 20 executes the graying process (step S2-1) and the black and white reversal process (step S2-2). Here, it is sufficient that the background region and the stained region can be identified, and the present invention is not limited to the combination of the above processes. For example, a background color and a stained region may be identified using a predetermined color threshold, and color conversion may be performed so that the background region is black and the stained region is white.

  In the above embodiment, the control unit 21 of the image processing apparatus 20 executes the count process for the two-dimensional cells of the designated color (Step S1-5). Furthermore, the control unit 21 of the image processing apparatus 20 executes a count process for the three-dimensional cells of the designated color (step S1-6). The count process is not essential, and a process for generating a composite image in which a predetermined area is visualized in the multimodal image may be executed by the composite process (step S1-4).

  In the above embodiment, the control unit 21 of the image processing apparatus 20 executes HSV designation processing (step S4-2) and color range extraction processing (step S4-3) of the designated range. Here, the color area of the color scheme designated on the color designation screen is extracted. The extraction of the color region is not limited to this, and for example, a color scheme that is expressed by the synthesis of a designated color assigned in advance may be predicted, and the color region may be extracted based on this color scheme.

  In the above embodiment, the control unit 21 of the image processing apparatus 20 assigns one of the three primary colors to each staining area for each staining technique. The colors assigned to each area are not limited to the three primary colors. A composite color of a plurality of designated colors may be assigned a color that can be clearly distinguished from other color schemes. In this case, the target area is specified by predicting the combined color of each designated color.

DESCRIPTION OF SYMBOLS 10 ... Input part, 15 ... Output part, 20 ... Image processing apparatus, 21 ... Control part, 210 ... Management part, 211 ... Color allocation part, 212 ... Position adjustment part, 213 ... Image composition part, 214 ... Count part, 22 ... a captured image storage unit, 23 ... a composite image storage unit, 24 ... a calculation information storage unit.

Claims (8)

An image processing system including a captured image storage unit that stores a plurality of captured images captured using a plurality of methods and a control unit connected to an output unit,
In the photographed image storage unit, a first photographed image obtained by photographing the object by the first technique and a second photographed image obtained by photographing the object by the second technique are recorded.
The control unit is
Assigning a first color scheme to the first method, assigning a second color scheme to a second method different from the first method,
The target areas included in the recorded first captured image in the captured image storage unit, assigned colored with the first color, to generate a first image obtained by blackening the background,
The target areas included in the recorded second captured image in the captured image storage unit, colored in different second color from said first color to generate a second image obtained by blackening the background,
Mutual information amount when one of the first and second images is transformed using the pixel value of the pixel included in the first image and the pixel value of the pixel included in the second image To calculate
Using said modified as mutual information is maximized, the first, the second image, overlay target regions are colored respectively in register position, by said first color and said second color An image processing system for generating and outputting a composite image colored with the generated third color scheme.   The image processing system according to claim 1, wherein the first color scheme and the second color scheme are selected from three primary colors. Obtaining luminance information in the first and second captured images;
Based on the luminance information, first and second inverted images in which the target area is whitened are generated, and the values of the first color scheme and the second color scheme are set for the first and second inverted images as a whole. The image processing system according to claim 1, wherein multiplication is performed. The composite image is represented in an HSV color space using hue, saturation, and brightness,
Using the values specified in the HSV color space, the image processing system according to any one of claims 1 to 3, wherein the identifying the target area. In the composite image, by performing the closing process by morphological operation, the deformed target area, according to any one of claims 1 to 4, wherein counting the number of the target area image Processing system. The first and second photographed images are images obtained by photographing slice samples of the evaluation target tissue,
In the first method, the expression region of the first substance contained in the evaluation target tissue is colored, and in the second method, the expression region of the second substance contained in the evaluation target tissue is colored, the first material and the image processing system according to any one of claims 1 to 5, the second region material has been expressed, and detects as the target area. A method of performing image processing on an object using an image processing system including a captured image storage unit that stores a plurality of captured images captured using a plurality of methods and a control unit connected to an output unit Because
In the photographed image storage unit, a first photographed image obtained by photographing the object by the first technique and a second photographed image obtained by photographing the object by the second technique are recorded.
The control unit is
Assigning a first color scheme to the first method, assigning a second color scheme to a second method different from the first method,
The target areas included in the recorded first captured image in the captured image storage unit, assigned colored with the first color, to generate a first image obtained by blackening the background,
The target areas included in the recorded second captured image in the captured image storage unit, colored in different second color from said first color to generate a second image obtained by blackening the background,
Mutual information amount when one of the first and second images is transformed using the pixel value of the pixel included in the first image and the pixel value of the pixel included in the second image To calculate
Using said modified as mutual information is maximized, the first, the second image, overlay target regions are colored respectively in register position, by said first color and said second color An image processing method comprising: generating and outputting a composite image colored with the generated third color scheme. To perform image processing on an object using an image processing system including a captured image storage unit that stores a plurality of captured images captured using a plurality of methods and a control unit connected to an output unit The program of
In the photographed image storage unit, a first photographed image obtained by photographing the object by the first technique and a second photographed image obtained by photographing the object by the second technique are recorded.
The control unit
Assigning a first color scheme to the first method, assigning a second color scheme to a second method different from the first method,
The target areas included in the recorded first captured image in the captured image storage unit, assigned colored with the first color, to generate a first image obtained by blackening the background,
The target areas included in the recorded second captured image in the captured image storage unit, colored in different second color from said first color to generate a second image obtained by blackening the background,
Mutual information amount when one of the first and second images is transformed using the pixel value of the pixel included in the first image and the pixel value of the pixel included in the second image To calculate
Using said modified as mutual information is maximized, the first, the second image, overlay target regions are colored respectively in register position, by said first color and said second color An image processing program that functions as means for generating and outputting a composite image colored with the generated third color scheme.