JP2022112542A - Production device, image processing device, production method, image processing method, and program - Google Patents

Abstract

To solve the problem in which conventionally, it has been difficult to obtain information for accurately detecting an area of a purpose category from a target image including two or more categories.SOLUTION: An image processing device includes: a useful key area information storage unit that stores one or more pieces of useful key area information; an image division unit that divides a target image and acquires two or more small-image fragments; a key area determination unit that determines one or more key areas for each of the two or more small-image fragments acquired by the image division unit; a small-image fragment information amount acquisition unit that inspects whether key area information about each of the one or more key areas is similar to each piece of useful key area information stored in the useful key area information storage unit, and acquires a small-image fragment information amount according to the inspection result for each of the two or more small-image fragments; an image construction unit that constructs an output image that reflects the two or more small-image fragment information amounts; and an image output unit that outputs the output image. The image processing device allows an image for accurately detecting an area of a purpose category to be obtained from the target image including two or more categories.SELECTED DRAWING: Figure 2

Description

The present invention relates to an image processing apparatus and the like for detecting a target category area from a target image.

Conventionally, there has been a technology related to a discriminator that identifies a binary value indicating whether or not it is a cancerous region (see, for example, Patent Document 1).

IEEE TRANSACTION MEDICAL IMAGING VOL.38, NO.2 FEBRUARY 2019, From Detection of Individual Metastases to Classification of Lymph Node Status at the Patient Level: The CAMELYON17 Challenge

However, in the prior art, it is difficult to obtain information for accurately detecting a target category area from a target image containing two or more categories.

More specifically, in the prior art, since only binary discrimination of whether or not the cancerous area was possible, it is possible to force the doctor to look at the detected area, thereby avoiding erroneous detection and non-detection. Due to lack of control, physicians were often forced to diagnose areas that could be ignored from medical experience.

The apparatus for producing useful key region information according to the first aspect of the present invention comprises an image storage unit for storing images, a key region determining unit for determining one or more key regions that satisfy key conditions from the images, and regions for target categories. an information amount acquisition unit for acquiring an amount of information that is information on the ability to determine and is information on each of the one or more key areas determined by the key area determination unit; a useful key region determination unit for determining one or more useful key regions satisfying selection conditions for selecting a key region of interest; and useful key region information relating to the one or more useful key regions determined by the useful key region determination unit. and an accumulation unit.

With such a configuration, it is possible to acquire useful information for obtaining information for accurately detecting an area of a target category from a target image including two or more categories.

In addition, in the production apparatus of the second invention, in contrast to the first invention, there are one or more target reference images that are images including the target category area and one or more images that are images that do not include the target category area. and a reference image storage unit for storing the other purpose reference image, and the information amount acquisition unit stores the area corresponding to each of the one or more key areas determined by the key area determination unit in each of the one or more purpose reference images. The first inspection is to inspect whether or not the key area exists, and the second inspection is to inspect whether or not the area corresponding to each of the one or more key areas determined by the key area determination unit exists in each of the one or more other reference images. 2. The production apparatus according to claim 1, wherein two inspections are performed, and the information amount of each of the one or more key areas is acquired using the results of the first inspection and the second inspection.

With such a configuration, it is possible to acquire useful information for obtaining information for accurately detecting an area of a target category from a target image including two or more categories.

Further, in the production apparatus of the third invention, in contrast to the first or second invention, the useful key area determination unit uses the amount of information acquired by the information amount acquisition unit to determine one or more key areas. A production device that obtains an identification value and determines one or more useful areas where the identification value satisfies a selection condition.

With such a configuration, it is possible to acquire useful information for obtaining information for accurately detecting an area of a target category from a target image including two or more categories.

Further, the image processing apparatus of the fourth aspect of the present invention includes a useful key area information storage unit storing one or more pieces of useful key area information, a target image storage unit storing a target image which is an image to be processed, an image dividing unit that divides a target image and obtains two or more small image pieces; a key region determination unit that determines one or more key regions for each of the two or more small image pieces obtained by the image dividing unit; It is checked whether or not the key area information about one or more key areas determined by the area determination unit is similar to each useful key area information stored in the useful key area information storage unit, and a determination is made according to the inspection result. A small image piece information amount acquisition unit that acquires the small image piece information amount for each of two or more small image pieces, and information that reflects the two or more small image piece information amounts acquired by the small image piece information amount acquisition unit. and an output unit for outputting.

With such a configuration, information for accurately detecting a target category area from a target image including two or more categories can be obtained.

Further, in the image processing apparatus of the fifth invention, in contrast to the fourth invention, an image configuration for forming an output image reflecting the two or more small image piece information amounts acquired by the small image piece information amount acquisition unit and the information reflecting the two or more small image piece information amounts acquired by the small image piece information amount acquisition unit is an output image.

With such a configuration, an image for accurately detecting an area of a target category from a target image including two or more categories can be obtained.

Further, in the image processing apparatus of the sixth aspect of the present invention, in contrast to the fourth or fifth aspect, the small image piece information amount is obtained by using the two or more small image piece information amounts acquired by the small image piece information amount acquisition unit. a small image piece identification value obtaining unit for obtaining a small image piece identification value, which is an identification value; An image processing apparatus for outputting information reflecting two or more small image piece identification values acquired by a small image piece identification value acquiring unit instead of the small image piece information amount.

With such a configuration, information for more accurately detecting the region of the target category from the target image including two or more categories is obtained.

Further, in the image processing apparatus of the seventh invention, in the image processing apparatus of any one of the fourth to sixth inventions, when a change of the similarity condition is received, the small image piece information amount acquisition unit Based on the similarity condition, it is determined whether or not the key region information related to one or more key regions determined by the key region determination unit is similar to each useful key region information stored in the useful key region information storage unit. a new small image piece information amount corresponding to the inspection result is obtained for each of two or more small image pieces, and an output unit outputs information reflecting the new small image piece information amount; processing equipment.

With such a configuration, it is possible to dynamically change the output image according to the change of the received similarity condition.

Further, in the image processing apparatus of the eighth invention, in contrast to any one of the fourth to seventh inventions, the useful key area information storage unit stores one or more purpose useful key area information and one or more other useful key area information. Key region information is stored, and a small image piece information amount acquisition unit stores key region information related to one or more key regions determined by the key region determination unit, and each purpose useful key stored in the useful key region information storage unit It inspects whether or not the area information is similar, acquires the target small image piece information amount according to the inspection result for each of the two or more small image pieces, and obtains one or more each of the key area determination unit Inspecting whether or not the key area information on the key area is similar to each other useful key area information stored in the useful key area information storage unit, and calculating the other small image piece information amount according to the inspection result, Each of two or more small image pieces is obtained, and the output unit calculates the target small image piece information amount and the other small image piece information amount obtained by the small image piece information amount obtaining unit for each small image piece with respect to the target image. It is an image processing device that outputs information reflecting the above.

With such a configuration, information for detecting the region of the target category with higher accuracy from a medical image containing two or more categories can be obtained.

In addition, the image processing apparatus of the ninth invention compares the small image piece information amount acquired by the small image piece information amount acquiring unit with a threshold, further comprising a detection signal acquisition unit that acquires a detection signal when the image piece information amount is greater than a threshold;
The output unit is an image processing device that outputs a detection signal.

With such a configuration, information for detecting the region of the target category with higher accuracy from a medical image containing two or more categories can be obtained.

Further, the image processing apparatus of the tenth invention is an image processing apparatus, in contrast to any one of the fourth to ninth inventions, wherein the target image is a medical image.

With such a configuration, it is possible to obtain an image for accurately detecting a region of a target category (for example, cancer) from a medical image containing two or more categories.

According to the image processing apparatus of the present invention, it is possible to obtain an image for accurately detecting an area of a target category from a target image including two or more categories.

Block diagram of production apparatus 1 according to Embodiment 1 Block diagram of the same image processing device 2 Flowchart for explaining an operation example of the production apparatus 1 Flowchart for explaining an example of the same amount of information acquisition processing Flowchart for explaining an example of the same identification value acquisition process Flowchart for explaining an operation example of the image processing device 2 A diagram for explaining a processing example of the same key area determination unit 131 A diagram for explaining a processing example of the same key area determination unit 131 A diagram for explaining a processing example of the same key area determination unit 131 A diagram for explaining a processing example of the same key area determination unit 131 A diagram for explaining a processing example of the same key area determination unit 131 Diagram explaining the concept of correspondence between same key region information and purpose useful key region information A diagram for explaining a processing example of the same small image fragment information amount acquisition unit 233. A diagram for explaining a processing example of the same small image fragment information amount acquisition unit 233. A diagram for explaining a processing example of the same small image fragment information amount acquisition unit 233. A diagram for explaining a processing example of the same small image fragment identification value acquisition unit 234. Schematic diagram of a computer system in the above embodiment Block diagram of the same computer system

Embodiments of an image processing apparatus and the like will be described below with reference to the drawings. It should be noted that, since components denoted by the same reference numerals in the embodiments perform similar operations, repetitive description may be omitted.

(Embodiment 1)
In the present embodiment, useful key area information for determining a key area from an image, obtaining the amount of information of the key area, determining the useful key area using the amount of information, and accumulating the useful key area information production equipment will be described. In the present embodiment, it is preferable to obtain an identification value using the amount of information, and determine a useful key region using the identification value based on the amount of information.

Further, in the present embodiment, the image to be processed is divided, one or more key regions are determined for each of two or more acquired small image pieces, and the key region information of each of the one or more key regions is useful. An image processing apparatus that determines whether or not the image is similar to the key area information, constructs and outputs an output image including the image of each small image piece according to the determination result will be described.

The output image including the image of each small image piece according to the judgment result is, for example, an output image reflecting the amount of information for each small image piece according to the number of similar useful key region information. Also, the output image including the image of each small image piece according to the judgment result is, for example, an output image reflecting the identification value obtained using the amount of information for each small image piece. Also, the output image including the image of each small image piece according to the determination result is, for example, an output image reflecting the amount of information and the identification value.

FIG. 1 is a block diagram of a production apparatus 1 according to this embodiment. The production device 1 is a device that acquires and stores useful key area information, which will be described later. Also, the useful key region information is information normally used by the image processing apparatus described in the second embodiment.

The production apparatus 1 includes a first storage section 11 , a first reception section 12 , a first processing section 13 and a first output section 14 . The first storage section 11 includes an image storage section 111 , a reference image storage section 112 , a condition storage section 113 and a useful key area information storage section 114 . The first processing unit 13 includes a key area determination unit 131 , an information amount acquisition unit 132 , a useful key area determination unit 133 and an accumulation unit 134 .

FIG. 2 is a block diagram of the image processing device 2 in this embodiment. The image processing device 2 includes a second storage section 21 , a second reception section 22 , a second processing section 23 and a second output section 24 . The second storage section 21 includes a useful key area information storage section 114 and a target image storage section 211 . The second processing unit 23 includes an image segmentation unit 231 , a key region determination unit 232 , a small image piece information amount acquisition unit 233 , a small image piece identification value acquisition unit 234 , and an image construction unit 235 . The second output section 24 has an image output section 241 .

Note that the production device 1 and the image processing device 2 may be integrated. Further, the processing realized by the production device 1 and the image processing device 2 may be shared by three or more devices.

Various types of information are stored in the first storage unit 11 that constitutes the production apparatus 1 . The various types of information are, for example, images to be described later, determination conditions to be described later, and useful key area information to be described later.

The image storage unit 111 stores one or more images. The image here is an image from which useful key region information, which will be described later, is obtained. The image is, for example, a medical image. The image is, for example, an fMRI image, a PET image, an ultrasound image, etc., and its type and data structure are not limited.

The reference image storage unit 112 stores two or more reference images. A reference image is an image used when obtaining the information content of a key region. Each of the two or more reference images is associated with one of the two or more categories. Each of the two or more reference images is associated with a category identifier that identifies the category. For example, if the category is "cancer" or "normal", each of the two or more reference images is an image of "cancer" or an image that is "normal".

A key area is an area in an image and has a large amount of information. A key region is, for example, a keypoint obtained using a SIFT feature obtained from an image. A region is one pixel or a set of two or more pixels. The key region may be a small region useful for distinguishing the categories of the target image, and may be, for example, a mosaic feature image of the Viola-Jones method used for pedestrian and vehicle recognition.

A category is a type of some area in an image. The categories are, for example, a “cancer” region and a “normal” cell region. Also, the number of categories may be two, or three or more. If there are three or more categories, for example, they are "cancer", "liver cirrhosis", and "normal". Here, the category to be detected (for example, the area of "cancer") is appropriately called a target category. In addition, categories other than the detection target (for example, “normal” regions) are appropriately referred to as other categories.

The reference image is, for example, a target reference image or other reference image. The target reference image is an image of the target category (for example, an image consisting of a “cancer” area) and is a reference image associated with the target category. The other reference image is an image of another category (for example, an image consisting of a “normal” area), and is a reference image associated with the other category.

One or more conditions are stored in the condition storage unit 113 . Conditions are, for example, key conditions, selection conditions, and similar conditions described later.

A key condition is a condition for determining a key area from an image. The key condition is, for example, a condition for determining that the keypoint is acquired using the SIFT feature amount acquired from the image.

The selection condition is a condition for selecting a useful key area, which will be described later, from the key areas. The selection condition is, for example, that the later-described identification value is equal to or greater than a threshold, or that the later-described identification value is greater than the threshold.

The useful key area information storage unit 114 stores one or more pieces of useful key area information. The useful key region information is information accumulated by the accumulation unit 134 .

The useful key area information is information about useful key areas. The useful key region information is, for example, a feature quantity vector that is a set of two or more feature quantities of useful key regions. The two or more feature amounts forming the feature amount vector include, for example, one or more SIFT feature amounts obtained from the image. The two or more feature amounts forming the feature amount vector include, for example, one or more HOG feature amounts obtained from the image. A useful key region is a region that is useful for identifying a target category region (eg, a cancer region) among key regions.

It is preferable that the useful key area information storage unit 114 stores one or more purpose useful key area information and one or more other useful key area information. Note that useful key area information of a useful key area having a high ability to identify a target category is referred to as useful target key area information. Also, useful key area information of a useful key area having a high ability to identify other categories is referred to as other useful key area information.

The first reception unit 12 receives various instructions and information. Various instructions and information are, for example, start instructions, images, reference images, and similarity conditions. A start instruction is an instruction to start acquisition of useful key region information.

Acceptance of a similarity condition may be acceptance of a threshold for determining whether or not two vectors are similar. Acceptance of a similarity condition is, for example, acceptance of a numerical threshold value, acceptance of a threshold value by a slider bar, or the like, and any acceptance method is acceptable.

Here, reception means reception of information input from an input device such as a keyboard, mouse, or touch panel, reception of information transmitted via a wired or wireless communication line, optical disk, magnetic disk, semiconductor memory, etc. This is a concept including acceptance of information read from the recording medium.

Also, any means such as a touch panel, a keyboard, a mouse, or a menu screen may be used as means for inputting various instructions and information.

The first processing unit 13 performs various types of processing. The various types of processing are, for example, processing performed by the key region determination unit 131, the information amount acquisition unit 132, the useful key region determination unit 133, and the storage unit 134.

The key area determination unit 131 determines one or more key areas that satisfy the key conditions in the condition storage unit 113 from the image. Note that the image is an image in the image storage unit 111 . Determining a key area means obtaining key area information, and obtaining information specifying the key area (for example, one or more coordinate values in the image). Key area information is information about a key area. The key region information is, for example, a feature quantity vector that is a set of two or more feature quantities of the key region. The two or more feature amounts forming the feature amount vector include, for example, one or more SIFT feature amounts obtained from the image. The two or more feature amounts forming the feature amount vector include, for example, one or more HOG feature amounts obtained from the image.

The key region determination unit 131 determines one or more keypoints from the image. Note that the process of determining keypoints from an image is a well-known technique, so detailed description thereof will be omitted here. Also, the key region determination unit 131 acquires key region information of key points from the target image, for example.

The information amount acquisition unit 132 acquires the information amount of each of the one or more key areas determined by the key area determination unit 131 .

The amount of information is information about the key area. The amount of information is information about the ability to determine the domain of the target category.

The information amount acquisition unit 132, for example, performs a first inspection and a second inspection, and uses the results of the first inspection and the result of the second inspection to acquire the information amount of each of the one or more key regions.

The first inspection is an inspection as to whether or not an area corresponding to each of the one or more key areas determined by the key area determination unit 131 exists in each of the one or more target reference images.

The second inspection is an inspection of whether or not an area corresponding to one or more key areas determined by the key area determination unit 131 exists in one or more other reference images.

Note that the area corresponding to the key area is an area for area information similar to the key area information. The area corresponding to the key area is, for example, the area for the area information that satisfies the key area information and the similarity condition. The similarity condition is that the degree of similarity between two pieces of region information (for example, vectors) is equal to or greater than a threshold. Similar conditions are stored in the condition storage unit 113 .

For example, the information amount acquisition unit 132 acquires a larger amount of information for the key area as the number of target reference images in which an area corresponding to the key area exists as a result of the first inspection.

For example, the information amount acquisition unit 132 acquires a larger amount of information for the key area as the result of the second inspection shows that there are more other reference images in which an area corresponding to the key area exists. For example, the information amount acquisition unit 132 may acquire a smaller amount of information for the key area as the number of other reference images in which an area corresponding to the key area exists as a result of the second inspection increases.

For example, the information amount acquisition unit 132 acquires region information that is most similar to the key region information for each of one or more target reference images, and uses an increasing function (for example, addition) with the degree of similarity as a parameter to obtain information get the quantity.

The information amount acquisition unit 132 acquires, for example, the area information most similar to the key area information for each of the one or more other reference images, and acquires the amount of information by an increasing function with the degree of similarity as a parameter. For example, the information amount acquisition unit 132 acquires the region information most similar to the key region information for each of the one or more other reference images, and uses a decreasing function (for example, subtraction) with the degree of similarity as a parameter to obtain the information get the quantity.

The information amount acquisition unit 132 calculates the amount of information corresponding to the key area using, for example, Equations 1, 2, and 3 below.

Here, in the above formula, 'K _i ' and 'K _i' ' are the i-th key region (for example, the i-th key point). "h(K _i )" is the amount of information in the i-th key area. "C" and "C'" are categories. Here, the categories are, for example, "C=1" for "cancer" and "C=2" for "normal". Also, ``p(K _i , C)'' is the key region
??
??
is the probability distribution for category C of "q(C)" is the probability distribution for category C over the key region. In Equation 3, "q(C)" is the probability that one reference image is an image of category C when it is selected. " _Nc " is the number of categories of interest. For example, for cancer/normal discrimination, "N _c =2". is also a category. “N(K _i , C)” is the number of images among reference images of category C that have key regions similar to the key region.

Note that the information amount acquisition unit 132 may calculate “q(C)” using Equations 4 and 5 instead of Equation 3.

A method of calculating 'q(C)' using Equations 4 and 5 is a method of calculating using the number of images having a key region similar to the key region 'K _i '. In Equations 4 and 5, instead of "n(C)", the number of images "N ^k (C )” (see formula 4), the information amount acquisition unit 132 calculates “q(C)” (see formula 5).

The useful key region determination unit 133 uses the amount of information acquired by the information amount acquisition unit 132 to determine one or more useful key regions that satisfy selection conditions for selecting useful key regions.

The useful key region determination unit 133 acquires the identification value of each of the one or more key regions using, for example, the amount of information acquired by the information amount acquisition unit 132, and selects one or more useful regions whose identification value satisfies the selection condition. decide. Note that the identification value is information related to the ability to identify the target category. The amount of information used when acquiring the identification value of one key area is the amount of information of the one key area.

The selection conditions are, for example, that the identification value is equal to or greater than a threshold value, that the identification value is greater than the threshold value, that the identification value is among the top N key regions, and the like.

Note that the useful key region determining unit 133 calculates the identification value "v(K _i ,C)" using, for example, the following formula 6.

Here, if there are two possible categories C, ``cancer'' and ``normal'', the identification value ``v(K <sub>i</sub> , cancer)'' for identifying ``cancer'' in the useful key region ``K _i '' is as follows: It can be calculated by Equation 7.

For example, the useful key region determination unit 133 acquires a larger identification value as the purpose reference information is larger. The useful key region determination unit 133 acquires the identification value by, for example, an increasing function with the purpose reference information as a parameter. Note that the target reference information is information relating to a target reference image having an area corresponding to the key area. The target reference information is, for example, a number calculated by an increasing function whose parameter is the number of target reference images having a region corresponding to the key region. The purpose reference information is, for example, "p(K _i , purpose category)" (Formula 2) described above.

For example, the useful key region determination unit 133 acquires a larger identification value as the other reference information is smaller. The useful key region determination unit 133 acquires the identification value by, for example, a decreasing function with other reference information as a parameter. Note that the other reference information is information related to another reference image having an area corresponding to the key area. The other reference information is, for example, a number calculated by an increasing function whose parameter is the number of other reference images having regions corresponding to the key regions. Other reference information is, for example, "p(K _i , other category)" (Formula 2) described above.

The useful key region determination unit 133 may determine one or more useful regions whose information amount acquired by the information amount acquisition unit 132 satisfies the selection condition. In such a case, the selection conditions are, for example, that the amount of information is greater than or equal to the threshold, and that the amount of information is greater than the threshold. In this case, the amount of information increases as the number of target reference images having areas corresponding to the key areas increases, and decreases as the number of other reference images including areas corresponding to the key areas increases. Information.

The useful key area determining unit 133 has determined the useful key areas with high ability to identify the target category. However, the useful key area determination unit 133 may determine a useful key area that is highly capable of identifying other categories. If the useful key area determination unit 133 replaces the target category with the other category and performs the above-described processing, the useful key area determination unit 133 can determine a useful key area that has a high ability to identify the other category. If there are three or more categories, it is preferable that the useful key area determination unit 133 determines, for each category, a useful key area that is highly capable of identifying the category.

The accumulation unit 134 accumulates useful key area information regarding one or more useful key areas determined by the useful key area determination unit 133 . The accumulation unit 134 accumulates useful key area information in the useful key area information storage unit 114, for example.

It is preferable that the storage unit 134 stores the useful key region information in association with the amount of information acquired by the information amount acquisition unit 132 . Moreover, it is preferable that the storage unit 134 stores the useful key region information in association with the identification value acquired by the useful key region determination unit 133 .

The accumulation unit 134 preferably accumulates one or more purpose useful key region information and one or more other useful key region information. In such a case, it is preferable that the accumulation unit 134 accumulates the useful key region information in association with the category identifier. Note that the number of categories may be three or more.

The first output unit 14 outputs various information. The various types of information are, for example, useful key area information and information indicating that processing has ended.

Here, output means display on a display, projection using a projector, printing on a printer, sound output, transmission to an external device, storage on a recording medium, and transmission to another processing device or other program. This is a concept that includes delivery of processing results.

Various types of information are stored in the second storage unit 21 that constitutes the image processing apparatus 2 . Various types of information are, for example, a target image and useful key area information, which will be described later. One or more pieces of useful key area information in useful key area information storage unit 114 are preferably useful key area information acquired by production apparatus 1 .

One or more target images are stored in the target image storage unit 211 . A target image is an image to be processed. The target image is an image from which an output image, which will be described later, is based. The target image is, for example, a medical image. The target image is, for example, a medical image including a cancer region. The target image is, for example, an fMRI image, a PET image, an ultrasound image, or the like, and any type of image can be used. The target image is, for example, an image of TIFF, PNG, JPEG, etc., but its data structure is not limited.

The second reception unit 22 receives various instructions and information. Various instructions and information are, for example, a start instruction, a target image, useful key area information, and similarity conditions to be changed. A start instruction is an instruction to start output processing of an output image.

Any means such as a touch panel, a keyboard, a mouse, or a menu screen may be used as means for inputting various instructions and information.

The second processing unit 23 performs various types of processing. The various types of processing are, for example, processing performed by the image segmentation unit 231, the key region determination unit 232, the small image piece information amount acquisition unit 233, the small image piece identification value acquisition unit 234, and the image construction unit 235.

For example, the second processing unit 23 compares the small image piece information amount acquired by the small image piece information amount acquiring unit 233 described later with a threshold, and acquires a detection signal when the small image piece information amount is greater than the threshold. . Such detection signal acquisition processing may be performed by a detection signal acquisition unit (not shown). Also, the detection signal is, for example, information indicating that the small image fragment corresponds to the target category. Note that the threshold is stored in the second storage unit 21 .

The image dividing unit 231 divides the target image and acquires two or more small image pieces. It does not matter how the target image is divided. A small image piece is a portion of the segmented result of the target image. The small image piece preferably has a fixed size (for example, a size of x dots×y dots (where x and y are natural numbers)), but does not have to have a fixed size. The size of the small image piece does not matter. A small image piece has two or more pixels. Also, although it is preferable that the small image piece is rectangular, its shape does not matter.

The key region determination unit 232 determines one or more key regions for each of the two or more small image pieces acquired by the image division unit 231 . The process of determining the key area from the image by the key area determining section 232 may be the same as that of the key area determining section 131 .

The small image piece information amount acquisition unit 233 determines that the key area information related to each of the one or more key areas determined by the key area determination unit 232 is similar to each useful key area information stored in the useful key area information storage unit 114. Then, the information amount of the small image piece corresponding to the inspection result is obtained for each of two or more small image pieces. The small image piece information amount is the information amount of the small image piece.

It should be noted that being similar usually means satisfying a similarity condition. The similarity condition here is, for example, "the degree of similarity is equal to or greater than the threshold" and "the degree of similarity is greater than the threshold".

In addition, the small image piece information amount acquisition unit 233 normally calculates the degree of similarity between the key region information and one or more pieces of useful key region information stored in the useful key region information storage unit 114, and the degree of similarity It is determined whether or not they are similar depending on whether or not the conditions are satisfied.

The small image piece information amount acquiring unit 233 acquires a larger small image piece information amount as a result of the above examination, for example, as the number of similar useful key region information increases.

For example, the small image piece information amount acquisition unit 233 obtains a large small size as the number of key area information items similar to any useful key area information among the key area information of one or more key areas in the small image piece increases. Get the image piece information amount.

For example, the small image piece information amount acquisition unit 233 determines that one or more pieces of key region information are similar to any useful key region information among the key region information of one or more key regions in the small image pieces. The small image piece information amount is calculated by an increasing function with the information amount of as a parameter. Note that such an increasing function is, for example, addition.

For example, the small image piece information amount acquisition unit 233 obtains the information amount paired with the one or more useful key area information similar to the key area information of the one or more key areas in the small image piece as useful key area information. The small image piece information amount is calculated by an increasing function (for example, addition) obtained from the storage unit 114 and using the information amount as a parameter.

It is preferable that the small image piece information amount acquisition unit 233 acquires the target small image piece information amount and the other small image piece information amount for each of two or more small image pieces. That is, the small image piece information amount acquisition unit 233 obtains key area information related to one or more key areas determined by the key area determination unit 232 and useful key area information for each purpose stored in the useful key area information storage unit 114. are similar to each other, the target small image piece information amount corresponding to the inspection result is obtained for each of the two or more small image pieces, and the one or more keys determined by the key area determination unit 232 It is checked whether or not the key area information about the area is similar to each other useful key area information stored in the useful key area information storage unit 114, and the other small image piece information amount according to the inspection result is It is preferable to acquire every two or more small image pieces.

The small image piece identification value acquisition unit 234 uses the small image piece information amount acquired by the small image piece information amount acquisition unit 233 for each small image piece to obtain the small image piece identification value, which is the identification value of the small image piece. get.

The small image piece identification value acquisition unit 234, for example, determines that one or more pieces of key region information similar to any useful key region information out of the key region information of one or more key regions in the small image piece. The information amount of the small image piece is calculated by an increasing function whose parameter is the identification value of . Note that such an increasing function is, for example, addition.

In addition, the small image fragment identification value acquisition unit 234, for example, obtains the identification value paired with the one or more useful key region information similar to the key region information of each of the one or more key regions in the small image piece. A small image piece identification value is calculated by an increasing function (for example, addition) obtained from the area information storage unit 114 and using the identification value as a parameter.

In addition, the small image piece identification value acquisition unit 234 calculates the small image piece identification value using, for example, an increasing function with the small image piece information amount as a parameter.

For example, for each small image piece, the small image piece identification value acquisition unit 234 calculates information (e.g., difference, ratio) regarding the difference between the target small image piece information amount and the other small image piece information amount, and The information is used to obtain a small image strip identification value. Note that the small image piece identification value may be information about the difference. For each small image piece, the small image piece identification value acquisition unit 234 uses, for example, an increasing function with the target small image piece information amount as a parameter and a decreasing function with the other small image piece information amount as a An image piece identification value is calculated.

The image construction unit 235 constructs an output image reflecting the two or more small image piece information amounts acquired by the small image piece information amount acquisition unit 233 . The image construction unit 235, for example, constructs an output image reflecting the information amount of the small image piece for each small image piece.

The output image reflecting the small image piece information amount may be an output image reflecting the small image piece identification value. The image constructing unit 235 constructs an output image reflecting the two or more small image piece identification values acquired by the small image piece identification value acquiring unit 234, for example.

The image construction unit 235 reflects, for example, two or more small image piece information amounts acquired by the small image piece information amount acquisition unit 233 and two or more small image piece identification values acquired by the small image piece identification value acquisition unit 234. compose the output image. The output image reflecting the small image piece information amount of the small image piece and/or the small image piece identification value is an image parameter corresponding to the small image piece information amount of the small image piece and/or the small image piece is an output image of arranged image fragments having image parameters corresponding to the image fragment identification values of . Image parameters are parameters of an image, such as brightness, brightness, saturation, and color.

The image constructing unit 235 may construct an output image in which the small image piece information amount or the small image piece identification value for each small image piece is reflected in the target image.

The second output unit 24 outputs various information. The various types of information are, for example, information reflecting two or more small image piece information amounts acquired by the small image piece information amount acquisition unit 233 . The information reflecting the two or more small image piece information amounts acquired by the small image piece information amount acquisition unit 233 may be information reflecting the small image piece identification value described above. Further, the information reflecting the two or more small image piece information amounts acquired by the small image piece information amount acquisition unit 233 is, for example, an output image. Further, the information reflecting the two or more small image piece information amounts acquired by the small image piece information amount acquisition unit 233 is, for example, a detection signal. Note that if the second output unit 24 does not output an output image, the image construction unit 235 and the image output unit 241 are unnecessary.

The image output section 241 outputs the output image configured by the image configuration section 235 . Here, output means display on a display, projection using a projector, printing on a printer, transmission to an external device, accumulation on a recording medium, processing results to another processing device or other program, etc. This is a concept that includes handing over.

The first storage unit 11, the image storage unit 111, the reference image storage unit 112, the condition storage unit 113, the useful key area information storage unit 114, the second storage unit 21, and the target image storage unit 211 are stored in non-volatile recording media. Although preferred, a volatile recording medium is also feasible.

It does not matter how the information is stored in the first storage unit 11 or the like. For example, information may be stored in the first storage section 11 or the like via a recording medium, and information transmitted via a communication line or the like may be stored in the first storage section 11 or the like. Alternatively, information input via an input device may be stored in the first storage unit 11 or the like.

The first reception unit 12 and the second reception unit 22 can be realized by a device driver for input means such as a touch panel or a keyboard, control software for a menu screen, or the like.

First processing unit 13, key region determination unit 131, information amount acquisition unit 132, useful key region determination unit 133, accumulation unit 134, second processing unit 23, image division unit 231, key region determination unit 232, small image piece information The amount acquisition unit 233, the small image piece identification value acquisition unit 234, and the image construction unit 235 are usually implemented by a processor, memory, or the like. The processing procedure of the first processing unit 13 and the like is normally realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit). Note that the processor is, for example, a CPU, MPU, GPU, or the like, and any type of processor does not matter.

The first output unit 14, the second output unit 24, and the image output unit 241 may or may not include output devices such as displays and speakers. The first output unit 14 can be realized by output device driver software, or by output device driver software and an output device.

Next, an operation example of the production apparatus 1 will be described with reference to the flowchart of FIG. Note that, for example, when the first reception unit 12 receives a start instruction, the processes after step S301 are performed.

(Step S301) The second processing unit 23 substitutes 1 for the counter i.

(Step S302) The second processing unit 23 determines whether the i-th image exists in the image storage unit 111 or not. If the i-th image exists, the process goes to step S303, and if the i-th image does not exist, the process ends.

(Step S<b>303 ) The key area determination unit 131 acquires the i-th image from the image storage unit 111 .

(Step S304) The key region determination unit 131 acquires one or more pieces of key region information from the i-th image acquired in step S303, and temporarily stores the information in a buffer (not shown). Note that processing for acquiring key region information from an image can be performed using a known technique.

(Step S305) The second processing unit 23 substitutes 1 for the counter j.

(Step S306) The second processing unit 23 determines whether j-th key region information exists in a buffer (not shown). If the j-th key area information exists, go to step S307; if not, go to step S312.

(Step S307) The information amount acquisition unit 132 acquires the information amount of the j-th key region information. An example of such information amount acquisition processing will be described with reference to the flowchart of FIG.

(Step S308) The useful key region determination unit 133 acquires the identification value of the j-th key region information. An example of such identification value acquisition processing will be described with reference to the flowchart of FIG.

(Step S<b>309 ) The useful key area determination unit 133 acquires selection conditions from the condition storage unit 113 . The useful key region determination unit 133 determines whether or not the identification value acquired in step S308 satisfies the selection condition. When the selection condition is satisfied, the process goes to step S310, and when the selection condition is not satisfied, the process goes to step S311.

(Step S310) The storage unit 134 associates the j-th key area information, the amount of information of the j-th key area information, and the identification value of the j-th key area information, and stores them in the useful key area information storage unit 114. accumulate in

(Step S311) The second processing unit 23 increments the counter j by 1. Return to step S306.

(Step S312) The second processing unit 23 increments the counter i by 1. Return to step S302.

Next, an example of information amount acquisition processing in step S307 will be described using the flowchart of FIG.

(Step S401) The information amount acquisition unit 132 performs initialization processing. That is, the information amount acquisition unit 132 substitutes 0 for the variable “purpose reference information” and the variable “other reference information”.

Note that the variable “purpose reference information” is information acquired from the purpose reference image, and serves as a basis for acquiring the amount of information of the target key region. The variable “target reference information” is, for example, the number of target reference images having key region information similar (satisfying a similarity condition) to the target key region information. The variable “target reference information” is, for example, a value obtained by adding the similarity of similar key region information in a target reference image having key region information similar to the target key region information.

The variable “other reference information” is information acquired from other reference images, and is information that is the basis for acquiring the information amount of the target key region. The variable “other reference information” is, for example, the number of other reference images having key region information similar (satisfying a similarity condition) to the target key region information. The variable “other reference information” is, for example, a value obtained by adding the similarity of similar key region information in another reference image having key region information similar to the target key region information.

(Step S402) The information amount acquisition unit 132 substitutes 1 for the counter i.

(Step S<b>403 ) The information amount acquisition unit 132 determines whether or not the i-th target reference image exists in the reference image storage unit 112 . If the i-th target reference image exists, go to step S404, otherwise go to step S413.

(Step S<b>404 ) The information amount acquisition unit 132 acquires the i-th target reference image from the reference image storage unit 112 .

(Step S405) The key region determination unit 131 acquires one or more pieces of reference key region information, which are key region information, from the i-th target reference image acquired in step S404.

(Step S406) The information amount acquisition unit 132 substitutes 1 for the counter j.

(Step S407) The information amount acquisition unit 132 determines whether or not j-th reference key area information exists among the reference key area information acquired in step S405. If the j-th reference key region information exists, go to step S408, otherwise go to step S411.

(Step S408) The information amount acquisition unit 132 calculates the degree of similarity between the target key region information and the j-th reference key region information.

(Step S409) The information amount acquisition unit 132 determines whether or not the degree of similarity calculated in step S408 satisfies a similarity condition. If the similarity condition is satisfied (similar), the process goes to step S410, and if the similarity condition is not satisfied (not similar), the process goes to step S412.

(Step S410) The information amount acquisition unit 132 updates the variable “purpose reference information”. Note that the information amount acquisition unit 132 normally increases the variable “purpose reference information”. The information amount acquisition unit 132 adds 1 to the variable “purpose reference information”, for example. For example, the information amount acquisition unit 132 adds the similarity calculated in step S408 to the variable “purpose reference information”.

Next, without proceeding to step S411, the degree of similarity between the key region information and all the reference key region information of the i-th target reference image is obtained, and the degree of similarity is used to obtain the key region information You may acquire the purpose reference information of

(Step S411) The information amount acquisition unit 132 increments the counter i by 1. Return to step S402.

(Step S412) The information amount acquisition unit 132 increments the counter j by 1. Return to step S407.

(Step S413) The information amount acquisition unit 132 substitutes 1 for the counter k.

(Step S<b>414 ) The information amount acquisition unit 132 determines whether or not the k-th other reference image exists in the reference image storage unit 112 . If the k-th other reference image exists, go to step S415, otherwise go to step S424.

(Step S<b>415 ) The information amount acquisition unit 132 acquires the k-th other reference image from the reference image storage unit 112 .

(Step S416) The key region determination unit 131 acquires one or more pieces of reference key region information, which are key region information, from the k-th other reference image acquired in step S415.

(Step S417) The information amount acquisition unit 132 substitutes 1 for the counter l.

(Step S418) The information amount acquisition unit 132 determines whether or not l-th reference key region information exists among the reference key region information acquired in step S416. If the l-th reference key region information exists, go to step S419, otherwise go to step S422.

(Step S419) The information amount acquisition unit 132 calculates the degree of similarity between the target key region information and the l-th reference key region information.

(Step S420) The information amount acquisition unit 132 determines whether or not the degree of similarity calculated in step S419 satisfies a similarity condition. If the similarity condition is satisfied, the process goes to step S421, and if the similarity condition is not satisfied, the process goes to step S423.

(Step S421) The information amount acquisition unit 132 updates the variable "other reference information". Note that the information amount acquisition unit 132 normally increases the variable “other reference information”. The information amount acquisition unit 132 adds 1 to the variable “other reference information”, for example. For example, the information amount acquisition unit 132 adds the similarity calculated in step S408 to the variable "other reference information".

Next, without proceeding to step S422, the degree of similarity between the key region information and all the reference key region information of the i-th other reference image is acquired, and the degree of similarity is used to obtain the key region information Other reference information may be acquired.

(Step S422) The information amount acquisition unit 132 increments the counter k by 1. Return to step S414.

(Step S423) The information amount acquisition unit 132 increments the counter l by 1. Return to step S418.

(Step S424) The information amount acquisition unit 132 calculates the amount of information using the variable "purpose reference information" and the variable "other reference information". Return to upper process. Note that the information amount acquisition unit 132 normally calculates the amount of information using an increasing function with the value of the variable “purpose reference information” as a parameter. In addition, the information amount acquisition unit 132 normally calculates the information amount using a decreasing function using the value of the variable "other reference information" as a parameter. Note that the "purpose reference information" and the "other reference information" have the property that the larger one becomes, the smaller the other becomes. The amount of information may be calculated by an increasing function with the value of "other reference information" as a parameter.

Next, an example of the identification value acquisition process in step S408 will be described using the flowchart of FIG.

(Step S501) The useful key area determination unit 133 acquires the value of the variable "purpose reference information".

(Step S502) The useful key region determination unit 133 acquires the value of the variable "other reference information".

(Step S503) The useful key region determining unit 133 calculates an identification value using the value of the variable 'purpose reference information' and the value of the variable 'other reference information', and assigns the identification value to the target key region. It is associated with information and temporarily stored in a buffer (not shown). Return to upper process.

Note that the useful key region determination unit 133 always calculates the identification value by an increasing function using the value of the variable “purpose reference information” as a parameter. Also, the useful key region determination unit 133 calculates the identification value by a decreasing function with the value of the variable “other reference information” as a parameter, for example.

Next, an operation example of the image processing device 2 will be described using the flowchart of FIG. Here, processing for obtaining an output image from one target image will be described.

(Step S<b>601 ) The image division unit 231 acquires the target image from the target image storage unit 211 .

(Step S602) The image dividing unit 231 divides the target image obtained in step S601 to obtain two or more small image pieces.

(Step S603) The second processing unit 23 substitutes 1 for the counter i.

(Step S604) The second processing unit 23 performs initialization processing. The initialization process is a process of assigning 0 to the variable "small image piece information amount" and a process of assigning 0 to the variable "small image piece identification value".

(Step S605) The key region determination unit 232 determines whether or not the i-th small image piece exists among the small image pieces acquired in step S602. If the i-th small image fragment exists, go to step S606, otherwise go to step S620.

(Step S606) The key region determination unit 232 acquires one or more key region information in the i-th small image piece.

(Step S607) The second processing unit 23 substitutes 1 for the counter j.

(Step S608) The second processing unit 23 determines whether or not j-th key region information exists in the key region information acquired in step S606. If the j-th key area information exists, go to step S609; if not, go to step S617.

(Step S609) The second processing unit 23 substitutes 1 for the counter k.

(Step S<b>610 ) The second processing unit 23 determines whether or not the k-th useful key area information exists in the useful key area information storage unit 114 . If the k-th useful key area information exists, go to step S611, otherwise go to step S616.

(Step S611) The small image piece information amount acquisition unit 233 calculates the degree of similarity between the j-th key region information and the k-th useful key region information.

(Step S612) The small image piece information amount acquisition unit 233 determines whether or not the degree of similarity calculated in step S611 satisfies a similarity condition. If the similarity condition is satisfied, the process goes to step S613, and if the similarity condition is not satisfied, the process goes to step S615.

(Step S613) The small image piece information amount acquisition unit 233 updates the variable "small image piece information amount". For example, the small image piece information amount acquisition unit 233 adds the information amount paired with the k-th useful key region information to the variable “small image piece information amount”. The small image piece information amount acquisition unit 233 adds 1 to the variable “small image piece information amount”, for example.

(Step S614) The small image piece identification value acquisition unit 234 updates the variable “small image piece identification value”. The small image piece information amount acquisition unit 233 adds, for example, the identification value paired with the k-th useful key region information to the variable “small image piece identification value”. The small image piece identification value acquisition unit 234 adds 1 to the variable “small image piece identification value”, for example.

(Step S615) The counter k is incremented by 1. Return to step S610.

(Step S616) The counter j is incremented by 1. Return to step S608.

(Step S617) The image construction unit 235 uses the values of the variable "small image piece information amount" and the variable "small image piece identification value" for the i-th small image piece to determine the i-th small image in the target image. Get an image of the piece. Note that the image construction unit 235, for example, sets attribute values such as brightness, brightness, and color of the i-th small image piece in the target image to the variable "small image piece information amount" and/or the variable "small image piece identification value". , and constructs the image of the i-th small image piece.

(Step S618) The image constructing unit 235 arranges the image of the i-th small image piece acquired in step S617 at the position of the i-th small image piece.

(Step S619) The second processing unit 23 increments the counter i by 1. Return to step S605.

(Step S620) The image output unit 241 outputs the output image configured by the processing in step S618. End the process.

In the flowchart of FIG. 6, the small image piece information amount acquisition unit 233 may acquire the target small image piece information amount and the other small image piece information amount for each small image piece.

Further, in the flowchart of FIG. 6, the small image piece identification value acquisition unit 234 acquires the small image piece identification value for each small image piece using the target small image piece information amount and the other small image piece information amount. can be

A specific operation example of the production apparatus 1 according to the present embodiment will be described below.

It is assumed that one or more medical images are stored in the image storage unit 111 of the production apparatus 1 . Assume that the medical image includes cancerous areas and normal areas.

It is also assumed that the reference image storage unit 112 stores two or more reference images associated with the category identifier “cancer”. Such a reference image is an image of an area of cancer cells. It is also assumed that the reference image storage unit 112 stores two or more reference images associated with the category identifier “normal”. Such a reference image is an image of a region of normal cells.

Assume that the user inputs a start instruction to the production apparatus 1 in this situation. Next, the first reception unit 12 of the production device 1 receives the start instruction.

Then, one or more medical images in the image storage unit 111 are processed as follows, and a large amount of useful key region information is accumulated in the useful key region information storage unit 114 .

That is, first, the key region determination unit 131 acquires a medical image (hereinafter referred to as “image” as appropriate) from the image storage unit 111 .

Next, the key region determination unit 131 determines one or more key points (an example of key regions) from the acquired image. Using SIFT as a preferred image feature quantity, the key region determination unit 131 applies different scales (σ, k ¹ σ, k ² σ, k ³ σ, k ⁴ σ (σ, k ^x are positive numbers, x is a natural number) Smoothed images are acquired Next, the key region determination unit 131 determines two smoothed images whose scales are adjacent to each other. A Doc image, which is a differential image, is obtained (see FIG. 7).

Next, the key region determination unit 131 detects extreme values from each Doc image. The key region determiner 131 compares the DoG value of the pixel of interest with 26 neighbors in the image scale space. If the point is greater or less than the 26 neighbors, that point is taken as a keypoint candidate (see FIG. 8). Note that in FIG. 8, the key region determining unit 131 determines the DoG value of a pixel of interest (the central pixel of the colored pixels in the middle Doc image of a set of three Doc images) and The DoG value of the pixel and the Doc image of the corresponding pixel in the top and bottom Doc images of the Doc image triplet (the colored pixels in the top and bottom Doc images) are compared, and if the DoG value of the pixel of interest is greater or smaller than the DoG values of other pixels to be compared, the pixel of interest is determined as a keypoint candidate.

In addition, the key region determination unit 131 narrows down the keypoint candidates by excluding points on edges and small DoG values (for example, 0.03 or less) from the keypoint candidates. . Note that the key region determination unit 131 preferably performs sub-pixel position estimation and contrast threshold processing.

Next, the key region determination unit 131 calculates the gradient direction from the region corresponding to the scale of each of the remaining one or more keypoints. Such a conceptual diagram is shown in FIG. The key region determining unit 131 also adds the weight obtained by multiplying the Gaussian window by the gradient strength to the histogram of gradient orientations in the 36 directions. Also, the key region determination unit 131 assigns a peak, which is, for example, 80% or more from the maximum value of the histogram, as the orientation of the keypoint (see FIG. 10). Such assigned keypoints are selected keypoints. Depending on the key point, there may be two orientations.

Next, the key region determination unit 131 acquires a 128-dimensional feature amount vector as key region information based on the detected orientation. First, the key region determination unit 131 rotates the region describing the feature quantity in accordance with the orientation of the keypoint. The vector of the feature quantity has gradient information around the keypoint as elements. This gradient information is information obtained from a circle centered at a keypoint and having a radius corresponding to the scale of the keypoint. This area is divided into 4×4 blocks, each of which is a block unit, and one block consists of eight directions, so that the area has a total of 128-dimensional feature amounts. The key region determination unit 131 uses this feature amount to create a histogram of gradient directions as shown in FIG. At that time, the key region determining unit 131 normalizes the length of the 128-dimensional feature vector by the sum of the vectors. As a result, the key region determination unit 131 acquires a SIFT feature vector that is resistant to changes in illumination.

A large amount of key region information is obtained by performing the above processing or other processing capable of extracting useful small image regions for evaluating differences in categories on a plurality of medical images. The key region information here is a 128-dimensional feature vector calculated as a SIFT feature amount.

Next, the information amount acquisition unit 132 calculates the information amount of each of the one or more key area information acquired by the key area determination unit 131 using Equations 1, 2, and 3 described above.

Next, the useful key region determination unit 133 substitutes the information amount of each key region information of one or more into the above-described formula 7 (also referred to as formula 6), and executes the formula 7 to obtain one or more of each key An identification value "v(K _i , cancer)" for identifying "cancer" in the region information is calculated.

Next, the useful key region determining unit 133 determines whether or not the identification value of each of the one or more pieces of key region information matches the selection condition, and selects one or more pieces of key region information that match the selection condition. Acquired as useful key region information.

Next, the accumulation unit 134 associates one or more pieces of useful key region information, information amount of each useful key region information, and identification value of each useful key region information, and stores them in the useful key region information storage unit 114. accumulate.

By the above processing, the production apparatus 1 was able to accumulate a large amount of useful key region information with high ability to identify "cancer". Here, the useful key area information having a high ability to identify "cancer" is referred to as purpose useful key area information.

Further, it is assumed that the same algorithm processing as the above processing is performed, and that the production apparatus 1 has accumulated a large amount of useful key region information with a high ability to identify "normal". Here, if the "cancer" image and the "normal" image are exchanged and the production apparatus 1 operates according to the same algorithm as above, a large amount of useful key region information with a high ability to distinguish "normal" can be obtained. can be accumulated. Here, the useful key area information with high ability to identify "normal" is referred to as other useful key area information.

Next, the image processing apparatus 2 uses a large amount of useful key area information accumulated by the production apparatus 1 to compose and output an output image from the medical image including the cancer area by the following processing. It is assumed that the useful key area information storage unit 114 of the image processing apparatus 2 stores a large amount of purpose useful key area information and a large amount of other useful key area information acquired by the production apparatus 1 . It is also assumed that the target image storage unit 211 stores a medical image including a cancer region (hereinafter referred to as a “target image” as appropriate).

Assume that the user inputs a start instruction to the image processing apparatus 2 in such a situation. Then, the second reception unit 22 of the image processing device 2 receives the start instruction.

Next, the image dividing section 231 acquires the target image from the target image storage section 211 . Next, the image dividing unit 231 divides the obtained target image to obtain two or more small image pieces.

Next, the key region determining unit 232 acquires key region information of key points, for example, for each of two or more small image pieces. Here, it is assumed that the key region information is a vector of SIFT features.

Next, the small image piece information amount acquisition unit 233 determines useful key area information corresponding to the one or more pieces of key area information acquired by the key area determination unit 232 for each of the two or more small image pieces. Then, the small image piece information amount acquisition unit 233 calculates the number of pieces of key area information corresponding to the useful key area information for each of the two or more small image pieces. The useful key area information corresponding to the key area information is the useful key area information that satisfies the similarity condition with respect to the key area information. FIG. 12 shows the concept of correspondence between key area information and purpose-useful key area information. (a) of FIG. 12 is an image including purpose-useful key regions having a high ability to identify "cancer", and (b) is a target image. It can be judged that a region in the target image that has a good correspondence with the image of (a) is a region with "cancer" cells.

Also, FIG. 13 shows an image of processing performed by the small image piece information amount acquisition unit 233 . Reference numeral 1301 in FIG. 13 denotes an image (an image of a "cancer" area) corresponding to purpose-useful key area information (a vector of keypoints) with a high ability to identify "cancer". In addition, 1301 ◯ is purpose useful key region information. A target image 1302 has a large number of small image pieces. 1303 is the number of pieces of key area information corresponding to purposeful key area information for each small image piece. This number is the small image piece information amount. In other words, the small image piece information amount acquisition unit 233 calculates the number of pieces of key region information corresponding to the useful key region information for each small image piece.

Here, FIG. 14 shows an example of an image configured using purpose useful key region information for each small image piece acquired by the small image piece information amount acquisition unit 233 . Reference numeral 1401 in FIG. 14 denotes an image corresponding to the useful key area information, which is a sample image of the cancer area. 1402 is a target image. A target image includes a cancer region and a normal region. An image 1403 is an image in which small image pieces of 1303 in FIG. Reference numeral 1403 denotes an image in which a small image fragment that looks like cancer is clearly shown.

Next, the small image piece information amount acquisition unit 233 determines other useful key area information corresponding to the one or more pieces of key area information acquired by the key area determination unit 232 for each of the two or more small image pieces. Then, the small image piece information amount acquisition unit 233 calculates the number of pieces of key area information corresponding to other useful key area information for each of two or more small image pieces.

Here, FIG. 15 shows an example of an image configured using purpose useful key region information for each small image piece acquired by the small image piece information amount acquisition unit 233 . Reference numeral 1501 in FIG. 15 denotes an image corresponding to other useful key area information, which is a specimen image of a normal area. 1502 is a target image. A target image includes a cancer region and a normal region. 1503 is an image in which small image pieces are arranged such that the color becomes darker as the numerical value of each small image piece (the number of key region information corresponding to other useful key region information) increases. Reference numeral 1503 denotes an image in which a small image fragment recognized as normal is specified.

Next, the small image piece identification value acquisition unit 234 calculates the number of key area information corresponding to the purpose useful key area information and the number of key area information corresponding to other useful key area information for each of the two or more small image pieces. Calculate the difference between This difference is the small image piece identification value for each small image piece.

Next, the image construction unit 235 constructs an output image using the small image piece identification value for each small image piece.

Next, the image output unit 241 outputs the output image.

FIG. 16 is a diagram for explaining the operations of the small image fragment identification value acquisition section 234 and the image output section 241. As shown in FIG. 1601 in FIG. 16 is an image in which a small image piece having cancer-likeness is clearly indicated, which is the same as 1403 . 1602 is an image in which a small image fragment recognized as normal is clearly indicated, which is the same as 1503 . 1603 is a diagram in which normality is removed from cancer-likeness, and the color of the small image pieces in the cancer-like region is darker than in 1403, making the image easier for people to recognize the cancerous region.

It is also possible to employ a simple configuration in which the output of the image processing device 2 is not the image but the position of the small image piece in the target image having the information amount of the small image piece larger than the threshold value. Note that the threshold is, for example, information stored in advance or information that is input and received by the second receiving unit 22 . Also, the position of the small image piece may be one coordinate, the coordinates of two diagonal points of a rectangle representing the area, the center and radius of a circle, or the like.

As described above, according to the production apparatus 1 of the present embodiment, it is possible to acquire useful information for obtaining an image for accurately detecting an area of a target category from a target image including two or more categories.

Further, according to the image processing apparatus 2 of the present embodiment, an image for accurately detecting the area of the target category from the target image including two or more categories is obtained.

Furthermore, according to the image processing apparatus 2 of the present embodiment, it is possible to dynamically change the output image according to the received similarity condition change.

Note that the processing in this embodiment may be realized by software. Then, this software may be distributed by software download or the like. Also, this software may be recorded on a recording medium such as a CD-ROM and distributed. Note that this also applies to other embodiments in this specification. The software that implements the production apparatus 1 in this embodiment is the following program. In other words, this program comprises a key area determining unit for determining one or more key areas that satisfy key conditions from an image, and information on the ability to determine the area of the target category, which is determined by the key area determining unit. 1 satisfying a selection condition for selecting a useful key region by using an information amount acquisition unit that acquires an amount of information that is information about each of the one or more key areas; and using the amount of information acquired by the information amount acquisition unit: It is a program for functioning as a useful key area determination section for determining the useful key area and as an accumulation section for accumulating useful key area information regarding one or more useful key areas determined by the useful key area determination section.

Also, the software that implements the image processing apparatus 2 is the following program. That is, the program provides a computer capable of accessing a useful key region information storage section storing one or more pieces of useful key region information and a target image storage section storing a target image, which is an image to be processed. an image dividing unit that divides a target image and obtains two or more small image pieces; a key region determination unit that determines one or more key regions for each of the two or more small image pieces obtained by the image dividing unit; inspecting whether or not the key area information related to one or more key areas determined by the key area determination unit is similar to each useful key area information stored in the useful key area information storage unit; a small image piece information amount acquiring unit for acquiring a small image piece information amount according to the result for each of two or more small image pieces; This is a program for functioning as an output unit that outputs reflected information.

Also, FIG. 17 shows the appearance of a computer that executes the programs described in this specification and implements the production apparatus 1 and the image processing apparatus 2 of the various embodiments described above. The embodiments described above may be implemented in computer hardware and computer programs running thereon. FIG. 17 is an overview diagram of this computer system 300, and FIG. 18 is a block diagram of the system 300. As shown in FIG.

In FIG. 17, computer system 300 includes computer 301 including a CD-ROM drive, keyboard 302 , mouse 303 and monitor 304 .

18, a computer 301 includes a CD-ROM drive 3012, an MPU 3013, a bus 3014 connected to the CD-ROM drive 3012, a ROM 3015 for storing programs such as a boot-up program, It includes a RAM 3016 connected and for temporarily storing application program instructions and providing temporary storage space, and a hard disk 3017 for storing application programs, system programs and data. Although not shown here, computer 301 may also include a network card that provides connection to a LAN.

A program that causes the computer system 300 to execute the functions of the image processing apparatus 2 of the embodiment described above may be stored in the CD-ROM 3101, inserted into the CD-ROM drive 3012, and transferred to the hard disk 3017. . Alternatively, the program may be transmitted to computer 301 via a network (not shown) and stored in hard disk 3017 . Programs are loaded into RAM 3016 during execution. The program may be loaded directly from CD-ROM 3101 or network.

The program does not necessarily include an operating system (OS) or a third party program that causes the computer 301 to execute the functions of the image processing apparatus 2 of the above-described embodiment. A program need only contain those parts of instructions that call the appropriate functions (modules) in a controlled manner to produce the desired result. How the computer system 300 operates is well known and will not be described in detail.

In the above program, the step of transmitting information, the step of receiving information, etc. are performed by hardware. processing) are not included.

Also, the number of computers that execute the above programs may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

Further, in each of the above embodiments, each process may be implemented by centralized processing by a single device, or may be implemented by distributed processing by a plurality of devices.

An example of distributed processing is as follows. In other words, when distributed processing is performed on a remote island where it is difficult to secure a sufficient communication band and a base diagnosis center, a remote input unit (not shown) that inputs the target image to the remote island and an abnormal image piece position reception unit (not shown) are used. , and the output of the image processing device 2 is limited to the position of the abnormal image piece, the output result can be quickly transmitted from the base diagnostic center to the remote island.

It goes without saying that the present invention is not limited to the above-described embodiments, and that various modifications are possible and are also included within the scope of the present invention.

INDUSTRIAL APPLICABILITY As described above, the image processing apparatus according to the present invention has the effect of obtaining an image for accurately detecting an area of a target category from a target image including two or more categories, and is useful as an image processing apparatus or the like. is.

1 production device 2 image processing device 11 first storage unit 12 first reception unit 13 first processing unit 14 first output unit 21 second storage unit 22 second reception unit 23 second processing unit 24 second output unit 111 image storage Unit 112 Reference image storage unit 113 Condition storage unit 114 Useful key region information storage unit 131, 232 Key region determination unit 132 Information amount acquisition unit 133 Useful key region determination unit 134 Storage unit 211 Target image storage unit 231 Image division unit 233 Small image Piece information amount acquisition unit 234 Small image piece identification value acquisition unit 235 Image construction unit 241 Image output unit

Claims (14)

an image storage unit in which images are stored;
a key region determination unit that determines one or more key regions satisfying key conditions from the image;
an information amount acquisition unit that acquires an amount of information that is information related to the ability to determine an area of a target category and that is information related to each of the one or more key areas determined by the key area determination unit;
a useful key region determination unit that determines one or more useful key regions that satisfy a selection condition for selecting a useful key region using the amount of information acquired by the information amount acquisition unit;
a useful key area information production apparatus comprising: an accumulation unit for accumulating useful key area information relating to one or more useful key areas determined by the useful key area determination unit. further comprising a reference image storage unit for storing one or more target reference images that are images containing target category regions and one or more other-purpose reference images that are images that do not include target category regions;
The information amount acquisition unit
a first inspection for inspecting whether or not an area corresponding to each of the one or more key areas determined by the key area determination unit exists in each of the one or more target reference images; and determination by the key area determination unit. performing a second inspection for inspecting whether or not an area corresponding to each of the one or more key areas exists in each of the one or more other reference images, and using the results of the first inspection and the second inspection 2. The production apparatus according to claim 1, wherein the amount of information of each of the one or more key areas is obtained by The useful key region determining unit
2. The identification value of each of the one or more key areas is obtained using the information amount obtained by the information amount obtaining unit, and the one or more useful areas whose identification value satisfies the selection condition are determined. 3. The production apparatus according to claim 2. a useful key region information storage unit storing one or more useful key region information;
a target image storage unit that stores a target image that is an image to be processed;
an image dividing unit that divides the target image and acquires two or more small image pieces;
a key region determining unit that determines one or more key regions for each of the two or more small image pieces acquired by the image dividing unit;
inspecting whether or not the key area information related to one or more key areas determined by the key area determination unit is similar to each useful key area information stored in the useful key area information storage unit; a small image piece information amount acquisition unit that acquires a small image piece information amount according to the result for each of two or more small image pieces;
and an output unit that outputs information reflecting the two or more small image piece information amounts acquired by the small image piece information amount acquisition unit. further comprising an image construction unit that constructs an output image that reflects the two or more small image piece information amounts acquired by the small image piece information amount acquisition unit;
5. The image processing apparatus according to claim 4, wherein the output image is the information reflecting the two or more small image piece information amounts acquired by the small image piece information amount acquisition unit. further comprising a small image piece identification value obtaining unit for obtaining a small image piece identification value, which is an identification value of the small image piece, using the two or more small image piece information amounts obtained by the small image piece information amount obtaining unit. ,
The output unit
two or more small image piece identification values obtained by the identification value obtaining unit in addition to the two or more small image piece information amounts or in place of the two or more small image piece information amounts for the target image; 6. The image processing apparatus according to claim 4, which outputs information reflecting the above. The small image piece information amount acquisition unit
When a change of the similarity condition is accepted, the key region information about one or more key regions determined by the key region determination unit and stored in the useful key region information storage unit based on the changed similarity condition. inspecting whether or not each useful key region information is similar, and obtaining a new small image piece information amount according to the inspection result for each of the two or more small image pieces;
The output unit
7. The image processing apparatus according to any one of claims 4 to 6, wherein information reflecting the new small image piece information amount is output. In the useful key region information storage unit,
One or more purpose useful key area information and one or more other useful key area information are stored, and the small image piece information amount acquiring unit
It is checked whether or not the key area information related to one or more key areas determined by the key area determination unit is similar to each purpose useful key area information stored in the useful key area information storage unit, and A target small image piece information amount corresponding to an inspection result is obtained for each of two or more small image pieces, and key area information on each of the one or more key areas determined by the key area determining unit, and the useful key area information. inspecting whether each other useful key region information stored in the storage unit is similar, and acquiring the other small image piece information amount corresponding to the inspection result for each of the two or more small image pieces;
The output unit
5. Outputting information reflecting the target small image piece information amount and the other small image piece information amount acquired by the small image piece information amount acquiring unit for each small image piece, with respect to the target image. 8. The image processing apparatus according to any one of claims 7 to 7. a detection signal acquisition unit that compares the small image piece information amount acquired by the small image piece information amount acquisition unit with a threshold value and acquires a detection signal when the small image piece information amount is greater than the threshold value;
The image processing apparatus according to any one of claims 4 to 8, wherein the output section outputs the detection signal. The image processing apparatus according to any one of claims 4 to 9, wherein the target image is a medical image. A method for producing useful key region information realized by a key region determination unit, an information amount acquisition unit, a useful key region determination unit, and a storage unit,
a key region determination step in which the key region determination unit determines one or more key regions satisfying key conditions from an image;
an information amount acquisition step of acquiring an information amount, which is information related to the ability of the information amount acquisition unit to determine the area of the target category and information related to each of the one or more key areas determined in the key area determination step; ,
A useful key area determination step in which the useful key area determination unit determines one or more useful key areas that satisfy a selection condition for selecting a useful key area, using the information amount acquired in the information amount acquisition step. When,
A method for producing useful key area information, comprising an accumulation step in which the accumulation unit accumulates useful key area information relating to one or more useful key areas determined in the useful key area determination step. A useful key region information storage unit storing one or more pieces of useful key region information, a target image storage unit storing a target image that is an image to be processed, an image dividing unit, a key region determination unit, and a small image. An image processing method implemented by a piece information acquisition unit, an image construction unit, and an output unit,
an image dividing step in which the image dividing unit divides the target image to obtain two or more small image pieces;
a key region determination step in which the key region determination unit determines one or more key regions for each of the two or more small image pieces acquired in the image division step;
The small image piece information amount acquisition unit acquires key area information related to one or more key areas determined in the key area determination step and useful key area information stored in the useful key area information storage unit. a small image piece information amount acquiring step of inspecting whether or not there is similarity, and acquiring the small image piece information amount corresponding to the inspection result for each of two or more small image pieces;
and an output step in which the output unit outputs information reflecting the two or more small image piece information amounts obtained in the small image piece information amount obtaining step. the computer,
a key region determination unit that determines one or more key regions that satisfy key conditions from an image;
an information amount acquisition unit that acquires an amount of information that is information related to the ability to determine an area of a target category and that is information related to each of the one or more key areas determined by the key area determination unit;
a useful key region determination unit that determines one or more useful key regions that satisfy a selection condition for selecting a useful key region using the amount of information acquired by the information amount acquisition unit;
A program for functioning as an accumulation section for accumulating useful key area information relating to one or more useful key areas determined by the useful key area determination section. A computer capable of accessing a useful key region information storage unit storing one or more pieces of useful key region information and a target image storage unit storing a target image that is an image to be processed,
an image dividing unit that divides the target image and acquires two or more small image pieces;
a key region determining unit that determines one or more key regions for each of the two or more small image pieces acquired by the image dividing unit;
inspecting whether or not the key area information related to one or more key areas determined by the key area determination unit is similar to each useful key area information stored in the useful key area information storage unit; a small image piece information amount acquisition unit that acquires a small image piece information amount according to the result for each of two or more small image pieces;
A program for functioning as an output unit for outputting information reflecting two or more small image piece information amounts acquired by the small image piece information amount acquiring unit.

Images