JP2012187139A - Image processing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus for objectively classifying a subject into multiple regions and evaluating the classified regions.SOLUTION: The image processing apparatus includes a first calculation part, a first management part, a first evaluation part, a second calculation part, a second management part, and a second evaluation part. The first calculation part calculates feature amounts 0 from image data (S6). The first management part generates and manages an discriminator 0. The first evaluation part evaluates the feature amounts 0 by using the discriminator 0, classifies the image data into multiple region image data, and outputs them (S7-S9). The second calculation part calculates feature amounts i from the region image data. The second management part generates and manages an discriminator i. The second evaluation part evaluates the feature amounts i by using the discriminator i and outputs them.

Description

  Embodiments described herein relate generally to an image processing apparatus.

As an image processing apparatus, an ultrasonic diagnostic apparatus or the like is known. An ultrasonic diagnostic apparatus is a device that non-invasively obtains a tomographic image from the surface to the inside of a subject by an ultrasonic pulse reflection method. When the object is a part, internal information (degree of rust, defects, etc.) can be grasped from the tomogram without cutting the part. In medical use, by using an ultrasonic diagnostic apparatus, a doctor can read various diagnostic information such as the severity of a disease targeting an organ such as a liver from an acquired image.
As described above, the following techniques have been proposed in addition to the method for visually extracting the above-described regions (rust, defects, lesions, etc.).

  Type 1: A technology that uses three-dimensional information. Continuous information inside the subject is extracted from the three-dimensional information, and fatigue cracks inside the metal, blood vessels inside the liver, and the like are detected. Specifically, this is a method of extracting a region other than a substantial region by extracting a region from pre-photographed three-dimensional volume data based on a difference in echo intensity.

  Type 2: A technique that uses continuous information of a two-dimensional image. By evaluating continuous information and the like, a crack inside the material and a blood vessel inside the liver are detected. Specifically, the echo signal reflected from the boundary uses a feature that is continuously connected.

JP 2010-220775 A JP 2004-215701 A

By the way, since the type 1 technique is based on three-dimensional volume data, there is a problem that it cannot be applied to a general two-dimensional image.
The type 2 technique has a problem that a thin region cannot be detected because the determination is made based on crack, blood vessel thickness, and continuous echo information. If the above cannot be detected, there is a feature that the substantial area of the liver looks rough due to the influence of the capillary, so it is very important to extract the non-essential area such as the capillary. In industrial ultrasonic inspection, it is difficult and problematic to detect minute defects and the like.
For this reason, there is a need for an image processing apparatus that can objectively classify a subject into a plurality of regions and can evaluate the classified regions.

  The present invention has been made in view of the above points, and an object of the present invention is to provide an image processing apparatus capable of objectively classifying a subject into a plurality of areas and evaluating the classified areas. is there.

An image processing apparatus according to an embodiment includes:
A first arithmetic unit that receives image data obtained by imaging a subject including a plurality of regions, and calculates a feature amount that serves as an index for classifying the image data into the plurality of region image data;
Creating and managing a discriminator having an index for classifying the image data into the plurality of area image data based on a feature quantity serving as an index for classifying the image data into the plurality of area image data; The management department of
An identification having an index for classifying the image data into the plurality of region image data, and a feature amount serving as an index for classifying the image data calculated by the first arithmetic unit into the plurality of region image data A first evaluation unit that classifies the image data into the plurality of area image data according to the evaluation result, and outputs the classified area image data;
A second calculation unit that calculates a feature amount serving as an index for identifying a state of a region corresponding to the region image data from the region image data classified by the first evaluation unit;
Creating and managing a discriminator having an index for identifying the state of the region corresponding to the region image data based on a feature quantity serving as an index for identifying the state of the region corresponding to the region image data; The management department of
An identification having an index for identifying the state of the region corresponding to the region image data, as a feature quantity that is an index for identifying the state of the region corresponding to the region image data calculated by the second arithmetic unit And a second evaluation unit that outputs information on the result of the evaluation.

FIG. 1 is a block diagram illustrating an ultrasonic diagnostic system according to an embodiment. FIG. 2 is a flowchart for explaining the flow in the case where a subject is evaluated using the ultrasonic diagnostic system. FIG. 3 is a flowchart for explaining the flow when the subject is evaluated using the ultrasonic diagnostic system, following FIG. FIG. 4 is a diagram showing the screen of the display unit shown in FIG. 1, and is a schematic diagram showing a state in which a region of interest is selected from a part of the image data. FIG. 5 is a diagram showing a screen of the display unit, and is a schematic diagram displaying blocks on the blood vessel wall and blood. FIG. 6 is a histogram showing an image of the feature distribution in the feature region shown in FIGS. 4 and 5. FIG. 7 is a conceptual diagram showing the discriminator 0 created and managed by the management unit shown in FIG. 1, and is a diagram showing feature quantities of three classes in the higher-order feature space. FIG. 8 is another conceptual diagram illustrating the discriminator 0 created and managed by the management unit, and is a diagram in which the three classes of feature amounts illustrated in FIG. 7 are projected in one dimension. FIG. 9 is a diagram showing a screen of the display unit, and is a diagram showing a tomographic image of a part. It is a conceptual diagram which shows the discriminator i which the said management part produces and manages. It is another conceptual diagram which shows the discriminator i which the said management part produces and manages. It is a conceptual diagram which shows the example of a local binary pattern. It is a conceptual diagram which shows the other example of a local binary pattern. It is a conceptual diagram which shows the other example of a local binary pattern. It is a conceptual diagram which shows the other example of a local binary pattern. It is a conceptual diagram which shows the other example of a local binary pattern. It is a conceptual diagram in the case of displaying the information of the result of having evaluated the substantial area | region (internal tissue) of the liver in one dimension. It is a conceptual diagram in the case of displaying the information of the result of having evaluated the substantial area | region of the said liver in two dimensions.

  Hereinafter, an image processing apparatus according to an embodiment will be described in detail with reference to the drawings. The image processing apparatus of this embodiment is an ultrasonic diagnostic apparatus. For this reason, in this embodiment, the case where an ultrasonic diagnostic apparatus (image processing apparatus) is incorporated and used in an ultrasonic diagnostic system (image processing system) will be described as an example.

  As shown in FIG. 1, the ultrasound diagnostic system 1 includes an ultrasound diagnostic apparatus 2, an imaging unit 3, a setting unit 4, and a display unit 5. The ultrasonic diagnostic apparatus 2 includes a calculation unit 6, a management unit 7, and an evaluation unit 8.

  The imaging unit 3 obtains image data by imaging a subject using ultrasonic waves. When the ultrasonic diagnostic apparatus 2 is used, the subject is subject to ultrasonic waves such as a part of a living body including muscles and bones, an organ including blood vessels, tumors, and organ parenchyma, and parts including defects and rust. Any image can be used as long as it can be imaged and includes portions with different ultrasonic image characteristics.

  Image data obtained by imaging the subject is input to the calculation unit 6. The image data includes a plurality of areas. For example, image data mainly of a patient's liver includes a plurality of liver real areas (hepatocytes), non-parenchymal areas of the liver such as blood vessels, blood, and tumors, and organs other than the liver such as kidneys. Includes area. The image data of a part includes a normal area and a plurality of defect areas.

Next, the configuration (function) of the ultrasonic diagnostic system 1 for classifying image data into a plurality of area image data will be described.
The calculation unit 6 calculates a feature amount (hereinafter referred to as a feature amount 0) as an index (parameter) for classifying the image data into a plurality of area image data from the input image data. The management unit 7 creates and manages a discriminator having an index for classifying image data into a plurality of area image data (hereinafter referred to as discriminator 0) based on the feature amount 0 calculated by the calculation unit 6. Is.

  Here, the above-described index for classifying the image data into a plurality of area image data may be information that can identify the area (texture identification (texture identification)) included in the image data. For example, as an index for image data mainly composed of a patient's liver, the shape of the liver and the luminance level and color of blood vessels and blood existing in the liver can be exemplified. In addition, various indicators such as a kidney shape, a heart shape, and a muscle pattern may be used. Examples of the index for the image data of the component include the luminance level and color of the corrosion area and the defect area.

  When the evaluation target image data is input to the calculation unit 6, the evaluation unit 8 evaluates the feature amount 0 calculated by the calculation unit 6 from the evaluation target image data using the classifier 0 of the management unit 7. Then, the evaluation unit 8 classifies the image data into a plurality of area image data according to the evaluation result, and outputs the classified area image data.

The setting unit 4 enables various settings by the operator of the ultrasonic diagnostic system 1. Some examples of settings using the setting unit 4 are as follows.
The setting unit 4 can be set to cause the calculation unit 6 to calculate an index corresponding to input image data. The setting unit 4 can be set to cause the evaluation unit 8 to evaluate the index corresponding to the image data to be evaluated.

  The setting unit 4 can be set so as to select a region of interest from the image data and cause the calculation unit 6 to calculate the region of interest of the image data. The setting unit 4 can be set to cause the evaluation unit 8 to evaluate the attention area of the image data to be evaluated. Note that the attention area corresponds to at least one pixel group including a plurality of adjacent pixels.

The setting unit 4 can be set so that the evaluation unit 8 outputs all or part of the plurality of classified area image data.
The setting unit 4 can be set to cause the evaluation unit 8 to output information indicating the feature amount 0, and the display unit 5 can display information indicating the feature amount 0.

Next, the configuration (function) of the ultrasonic diagnostic system 1 for evaluating the state of the classified region (region image data) will be described.
The calculation unit 6 calculates a feature amount (hereinafter referred to as a feature amount i) that serves as an index for identifying the state of the region corresponding to the region image data from the region image data input from the evaluation unit 8. is there. The management unit 7 creates and manages a discriminator (hereinafter referred to as discriminator i) having an index for identifying the state of the region corresponding to the region image data based on the feature quantity i.

  When the evaluation target region image data is input to the calculation unit 6, the evaluation unit 8 evaluates the feature quantity i calculated from the evaluation target image data by using the classifier i. And the evaluation part 8 outputs the information of the result of evaluation.

The setting unit 4 enables various settings by the operator of the ultrasonic diagnostic system 1. Some examples of settings using the setting unit 4 are as follows.
The setting unit 4 can be set to cause the calculation unit 6 to calculate an index corresponding to the input region image data. The setting unit 4 can be set to cause the evaluation unit 8 to evaluate the index corresponding to the area image data to be evaluated.

  The setting unit 4 can be set to select a region of interest from the region image data and cause the calculation unit 6 to calculate the region of interest of the region image data. The setting unit 4 can be set to cause the evaluation unit 8 to evaluate the attention area of the area image data to be evaluated. Note that the attention area corresponds to at least one pixel group including a plurality of adjacent pixels.

  The setting unit 4 can be set to cause the evaluation unit 8 to output various types of information related to the evaluation target region image data (the region of interest of the region image data), as well as the evaluation result information. For example, when the evaluation unit 8 evaluates a plurality of types of indicators and there are a plurality of pieces of evaluation result information, the setting unit 4 causes the evaluation unit 8 to output one or more types of evaluation result information. The display unit 5 can display one type or a plurality of types of evaluation result information. In addition, the setting unit 4 may cause the evaluation unit 8 to output the evaluation target region image data (original data) together with the evaluation result information, and display it on the display unit 5.

The setting unit 4 can be set to cause the evaluation unit 8 to output information indicating the feature quantity i, and the display unit 5 can display information indicating the feature quantity i.
The setting unit 4 can be set so that the evaluation unit 8 integrates and outputs the evaluation result information regarding the plurality of classified area image data, and the display unit 5 displays the integrated information. be able to.

The display unit 5 is input with information on the result of evaluation from the evaluation unit 8. The display unit 5 can display information on the result of evaluation by the evaluation unit 8.
As described above, an operator (an ultrasonic examiner, a doctor, or the like) can visually recognize information as a result of evaluation (diagnosis) by the evaluation unit 8. In addition, the display unit 5 can also display an image of the subject using image data obtained by the computing unit 6 imaging the subject.

  Note that a setting unit 4 for classifying image data into a plurality of region image data, a calculation unit 6, a management unit 7, an evaluation unit 8, and a setting unit 4 for evaluating the state of the classified region (region image data). Since the calculation unit 6, the management unit 7, and the evaluation unit 8 can be shared, they are described with the same reference numerals, but they may be provided separately.

Next, a flow when classifying and evaluating a subject using the ultrasonic diagnostic system 1 will be described.
As shown in FIGS. 1 and 2, when the classification and evaluation of the subject using the ultrasonic diagnostic system 1 is started, first, in step S1, the imaging unit 3 calculates the image data of the captured subject. Input to part 6. Needless to say, it is assumed that the image data includes a plurality of regions.

  In subsequent step S2, as necessary, the setting unit 4 displays the captured image data on the display unit 5 by setting the calculation unit 6 and the evaluation unit 8, and selects a region of interest from the image data. May be. Thereby, the location to be classified and evaluated can be selected in advance.

  Next, in step S3, as necessary, the setting unit 4 causes the calculation unit 6 to calculate the index corresponding to the image data (the attention area of the image data) and cause the evaluation unit 8 to evaluate the index. And the evaluation part 8 may be set. At this time, the setting unit 4 sets the calculation unit 6 and the evaluation unit 8 so that the calculation unit 6 calculates and evaluates the index suitable for the image data (the attention area of the image data). Is preferred.

  For example, when it is assumed that the same type of image data is classified in the same manner with respect to the same index, the step S3 may not be used. On the other hand, if the type of image data is not determined to be the same every time classification is performed, or the index is changed, it is effective to use the step S3.

  Thereafter, in step S4, the setting unit 4 classifies the image data (the attention area of the image data) into a plurality of area image data, and then outputs all or part of the classified area image data as necessary. As described above, the evaluation unit 8 may be set. For example, if the image data is mainly the liver and you want to evaluate only the blood vessels inside the liver, the image data (regions) of multiple regions, such as the real region of the liver, blood vessels and blood inside the liver, organs other than the liver, etc. It is effective to set by the setting unit 4 so that only the image data (region image data) of blood vessels inside the liver is output after being classified into (image data).

  In subsequent step S5, the setting unit 4 calculates the feature amount 0 as necessary, and then causes the evaluation unit 8 to output information indicating the feature amount 0 to the evaluation unit 8 and display the information on the display unit 5. May be set. Thereby, not only the subject is objectively classified using the ultrasonic diagnostic system 1 but also the subject can be subjectively classified by the operator, or the subject can be objectively classified by the ultrasonic diagnostic system 1. The classification can be reviewed and done.

  Next, in step S6, the calculation unit 6 calculates the feature amount 0 from the image data (the attention area of the image data), and then in step S7, the evaluation unit 8 identifies the calculated feature amount 0 by the management unit 7. Evaluate using vessel 0. Subsequently, in step S8, the evaluation unit 8 classifies the image data (the attention area of the image data) into a plurality (n) of area image data according to the evaluation result. Note that n is a natural number of 2 or more.

  Thereafter, in step S9, the evaluation unit 8 outputs all the plurality of classified area image data. At this time, the evaluation unit 8 can output all or only part of the classified area image data according to the setting contents using the setting unit 4 in step S4.

  As shown in FIGS. 1 and 3, in the subsequent step S <b> 10, the setting unit 4 displays the region image data i on the display unit 5 by setting the evaluation unit 8 as necessary, so that the region image data i is displayed. A region of interest may be selected from the list. Thereby, the location to be evaluated can be selected in advance.

  Next, in step S11, as necessary, the setting unit 4 causes the calculation unit 6 to calculate and evaluate the evaluation unit 8 with respect to the index corresponding to the region image data i (the region of interest of the region image data). You may set the calculating part 6 and the evaluation part 8. FIG. At this time, the setting unit 4 causes the calculation unit 6 and the evaluation unit 8 to calculate the index suitable for the region image data i (the region of interest of the region image data) and cause the evaluation unit 8 to evaluate the index. It is preferable to set.

  For example, when the calculation unit 6 can calculate the index corresponding to the region image data i (the attention area of the region image data) by the automatic recognition processing of the calculation unit 6 and the evaluation unit 8 and the evaluation unit 8 can evaluate, step S11 It is not necessary to use this process. On the other hand, it is effective to use the step S11, such as when an index corresponding to the region image data i (region of interest of the region image data) is customized every time it is calculated and evaluated.

  In subsequent step S <b> 12, the setting unit 4 calculates the feature quantity i and outputs the information indicating the feature quantity i to the evaluation unit 8 and displays it on the display unit 5 as necessary. May be set. Thereby, not only the region image data i (region of interest of region image data) is objectively evaluated using the ultrasonic diagnostic system 1, but also the subjective evaluation of the region image data i (region of interest of region image data) by the operator. Evaluation can be performed, and objective evaluation of the region image data i (region of interest of region image data) by the ultrasonic diagnostic system 1 can be reviewed.

  Next, after calculating the feature amount i from the region image data i (region of interest of the region image data) in step S13, the evaluation unit 8 uses the management unit to calculate the calculated feature amount i in step S14. 7 is used for evaluation. Subsequently, in step S <b> 15, the evaluation unit 8 outputs information on the result of evaluation regarding the region image data i (region of interest of the region image data), and displays the information on the evaluation result on the display unit 5.

  Thereby, the area | region image data i (region of interest of area | region image data) can be objectively evaluated using the ultrasonic diagnostic system 1. FIG. In step S15, the setting unit 4 may set the display unit 5 to display information indicating the feature amount i calculated from the region image data i (the region of interest of the region image data).

  Further, in step S15, the setting unit 4 causes the evaluation unit 8 to output various types of information related to the evaluation target region image data (the attention region of the region image data) as well as the evaluation result information. It can be set. Thereby, the operator can perform multi-faceted evaluation of the area image data i (the attention area of the area image data).

  Thereafter, in step S16, the management unit 7 and the evaluation unit 8 determine whether the evaluation of all the region image data i to be evaluated (the attention region of the region image data) has been completed, that is, whether i = m. To do. Note that i is set to 1 at first (i = 1). Further, m is the number of region image data i to be evaluated (region of interest in region image data), and is a natural number equal to or less than n.

  When all the evaluations are not completed, i.e., when i <m (step S16), the management unit 7 and the evaluation unit 8 increment i (i ++), proceed to step S10, and perform steps S10 to S15. repeat. When all the evaluations are completed, i.e., i = m (step S16), the process proceeds to step S17.

  For example, the image data is classified into five area image data (areas), and a plurality (n = 5) of which two area image data (areas) are to be evaluated (m = 2). Advances step S10 to S15 for region image data 1 (i = 1), and then advances to steps S10 to S15 for region image data 2 (i = 2), and then proceeds to step S17.

Subsequently, in step S <b> 17, the setting unit 4 integrates information on the results of evaluation regarding the plurality of classified region image data i (region of interest of region image data) into the evaluation unit 8 as necessary. You may set so that it may output, and, thereby, the information integrated on the display part 5 can be displayed. Thus, the operator can collectively evaluate a plurality of area image data i (a plurality of attention areas of the area image data).
From the above, the classification and evaluation of the subject using the ultrasonic diagnostic system 1 are completed.

  Next, a case will be described in which image data mainly including the liver is classified into a plurality of region image data using the ultrasonic diagnostic system 1. Here, the captured image data is displayed on the display unit 5, and a region of interest is selected from the image data. Further, the evaluation target area is a liver real area, and only image data (area image data) of the liver real area is output. For this reason, regions (region image data) other than the substantial region of the liver are excluded.

  FIG. 4 is a diagram showing a screen of the display unit 5, and is a schematic diagram showing a state in which a region of interest is selected from a part of the image data. In order to select a region of interest, the liver internal tissue is displayed on the display unit 5 using image data obtained by imaging the liver internal tissue using ultrasound in advance.

  When a patient's liver internal tissue is imaged, the displayed image (image data) includes an organ other than the liver, a background, and the like. The liver is an organ rich in blood vessels. For this reason, the setting is made so that the doctor selects a substantial region of the liver from the image (image data) excluding organs other than the liver, background, blood vessels of the liver, and the like based on the image displayed on the display unit 5. Operate the settings in section 4. Thereby, the image data (region image data) of the substantial region of the liver to be diagnosed (evaluated) can be automatically classified and output.

The automatic classification of the liver parenchymal region and non-parenchymal region using the ultrasonic diagnostic system 1 is performed as follows.
In the ultrasonic tomogram of the liver, there is a feature that the echo luminance (brightness level) of the softer part of the liver such as blood is lower and the echo brightness of the harder part of the liver such as the blood vessel wall is higher. For this reason, such features are extracted by texture analysis.

  For example, feature values based on luminance are extracted (calculated) for each small block. Each block corresponds to a pixel group including a plurality of adjacent pixels. The shape of the block is not limited to a rectangular shape and can be variously modified. For example, it may be a fan shape, a circular shape, or an elliptical shape. The size of the block (pixel group) is not limited and can be variously modified. Then, an average value and a variance value of luminance levels in each block are calculated.

  Since the blood portion shows a characteristic that the luminance value is low and smooth, it can be estimated that a block having a low luminance value and a low variance value is a blood region. In addition, since the blood vessel wall portion and the capillary blood vessel portion have a high luminance value and a characteristic in which the luminance difference from surrounding pixels is severe, a block having a high luminance value and a high variance value is a blood vessel wall or a capillary vessel. Can be estimated.

  FIG. 5 is a diagram showing a screen of the display unit 5, and is a schematic diagram displaying blocks overlaid on the blood vessel wall and blood in the region of interest. As shown in FIG. 5, blocks can be displayed so as to overlap the non-substantial region of the liver, which is a blood vessel wall or blood. Here, the block having a low variance value is a block (pixel group) having a small variation in luminance value between pixels. A block having a high variance value is a block (pixel group) having a large variation in luminance value between pixels.

Next, a method for classifying (extracting) a substantial region and a non-substantial region of the liver, such as classifying (extracting) a blood vessel region based on the calculated feature amount 0, will be described.
FIG. 6 is a histogram showing an image of the feature distribution in the feature region shown in FIGS. 4 and 5. As shown in FIG. 6, if a region having a high luminance value and a high variance value, an intermediate region, and a region having a low luminance value and a low variance value can be separated, the liver real region and non-liver such as blood vessels and blood The real area can be separated (classified).

  Examples of the separation (classification) method include principal component analysis (PCA), Otsu method, and P-tile method. For example, in the case of separation (classification) by the principal component analysis method, an identification feature axis that maximizes the degree of separation is extracted from the calculated feature amount by the principal component analysis method.

  FIG. 7 is a conceptual diagram showing the discriminator 0 created and managed by the management unit 7. FIG. 8 is another conceptual diagram showing the discriminator 0 created and managed by the management unit 7. FIG. 7 shows feature amounts of three classes in the higher-order feature space. Further, when projected in one dimension, the distribution shown in FIG. 8 is obtained.

  Here, the distance to each class can be calculated by projecting the feature amount calculated from the image data (the attention area of the image data) onto the identification feature axis calculated above. In calculating the distance to each class, the Euclidean distance may be calculated, or the Mahalanobis distance may be calculated. It is possible to classify into classes based on the calculated nearest distance.

  Further, in the feature distribution image shown in FIG. 6, a region having a high luminance value and a high variance value can be separated using a threshold value set by the P-tile method, and the luminance value is low and the variance value is low. It is also possible to isolate low areas. Further, the mode value of the intermediate feature value can be obtained by using the Otsu method, and a threshold value having the highest degree of separation can be calculated in order to separate a region having a high luminance value and a high variance value. Similarly, it is possible to calculate a threshold value for separating an area having a low luminance value and a low variance value. Using the two calculated thresholds, three classes can be classified, and non-substantial regions of the liver such as blood vessels can be extracted (classified).

  After the extraction (classification), the state of the extracted blood vessel region (blood vessel) is evaluated, and the state of the substantial region of the liver after the removal of blood vessels and the like is evaluated. Alternatively, the evaluation is performed only on the designated (focused) target area. As a result of the evaluation, the blood vessel region and the substantial region of the liver can be handled as separate evaluation indexes, and the whole liver function can be evaluated as a separate evaluation index.

  For example, the blood vessel wall can be detected by extracting edge information or the like for the extracted blood vessel region. When the liver function is normal, the blood vessel wall can be detected neatly. On the other hand, the liver where the lesion progresses considerably, such as cirrhosis, has a characteristic that the blood vessel wall is burnt. The edge information of the blood vessel wall is useful for evaluating the entire liver function as an auxiliary evaluation index. Therefore, by integrating the evaluation results for the substantial region of the liver after removing the blood vessel region and the blood vessel wall information of the blood vessel region, it is possible to make it closer to a doctor's judgment standard.

  The case where the subject is classified and evaluated when the subject is a patient has been described above. However, the subject is not limited to a patient and can be variously modified. For example, the subject may be a component.

  FIG. 9 is a diagram showing a screen of the display unit 5, and is a diagram showing a tomographic image of the imaged part. As shown in FIG. 9, even if the subject is a part, by using the ultrasonic diagnostic system 1, the image data of the part, the image data of the defect area (area image data), and the image of the normal area are displayed. Data (region image data) can be automatically classified.

After the above classification, the classified normal area (area image data) can be automatically evaluated by using the ultrasonic diagnostic system 1, and the degree of corrosion progress (no corrosion) from the normal area. It is also possible to evaluate a part, a lightly corroded part, a corroded part, etc.).

Next, a case where region image data of a substantial region of the liver is evaluated using the ultrasonic diagnostic system 1 will be described.
First, the discriminator i of the management unit 7 when diagnosing (evaluating) a substantial region of the liver will be described. In clinical practice, doctors classify the roughness of the substantial area of the liver into four stages based on subjective judgment. Therefore, the management unit 7 can create and manage a discriminator i having an index for classifying the roughness of the substantial area of the liver into four stages by subjective judgment. Thus, the evaluation unit 8 can objectively determine the roughness of the substantial area of the liver by using the discriminator i.

  Further, every time image data to be evaluated is input to the calculation unit 6, the management unit 7 updates an index for evaluating the feature quantity of the classifier based on information on the result of evaluation by the evaluation unit 8. It is possible. That is, the subjective judgment of the doctor can be quantified by taking the diagnosis result as learning data into the discriminator of the management unit 7.

  Here, when diagnosis is performed on an organ such as the liver using ultrasound, there are indices such as the smoothness of the liver surface, the angle of the liver margin, and the roughness of the internal tissue of the liver. Among them, the roughness of the liver internal tissue is the index that is most difficult to evaluate objectively. However, as described above, the management unit 7 creates and manages the discriminator i, thereby reducing the roughness of the liver internal tissue. It can be evaluated objectively. In addition, the management unit 7 creates and manages the discriminators 0 and i corresponding to the smoothness of the liver surface, the angle of the liver margin, and the organs other than the liver, without limiting to the internal tissue of the liver. It can be evaluated objectively.

Next, a substantial method for diagnosing a substantial region of the liver, such as a method for displaying information on the evaluation results, will be described.
When the pixel group index is set, the calculation unit 6 calculates a feature amount (texture feature amount) i of the region image data. Then, the evaluation unit 8 evaluates each calculated feature quantity i using the classifier i of the management unit 7. The management unit 7 updates the index for evaluating the feature quantity i of the discriminator i based on the information obtained as a result of the evaluation by the evaluation unit 8, and causes the discriminator i to learn in this way.

  When the management unit 7 forms and manages the classifier i using the principal component analysis method based on the feature quantity i, the management unit 7 calculates the feature quantity i from the classifier i, and calculates the calculated feature quantity. An evaluation axis that maximizes the degree of separation can be calculated (extracted) by i using principal component analysis.

FIG. 10 is a conceptual diagram showing the classifier i. As shown in FIG. 10, the feature quantity i of the discriminator i is projected on a higher-order space. An evaluation axis calculated by the management unit 7 is also shown in the higher-order space. From this, it can be seen that the feature quantity i is high-order information. The feature quantity i is classified into four classes from class 1 to class 4.
Here, the evaluation axis may be selected from a plurality of axes that form a higher-order space, or may be a newly generated axis.

  FIG. 11 is another conceptual diagram showing the discriminator i. As shown in FIG. 11, the feature quantity i of the discriminator i shown in FIG. 10 is projected and shown only on the evaluation axis. Here, the feature quantity i classified into class 1 is the feature quantity i calculated from the region image data of the normal region of the normal liver, and the feature quantity i classified into classes 2 to 4 is non-normal (lesional) ) Of the feature amount calculated from the region image data of the substantial region of the liver.

  From the above, the distance to each class can be calculated by projecting the feature quantity i calculated from the area image data to be evaluated onto the evaluation axis, and the calculated distance is classified into the class with the shortest distance. can do. For this reason, the evaluation unit 8 can evaluate the feature quantity i by classifying the feature quantity i into a class, and thus can diagnose a substantial region of the liver. In calculating the distance to each class, calculation of Euclidean distance or calculation of Mahalanobis distance can be used.

  As described above, since the evaluation axis is the axis with the highest degree of separation of the feature quantity i, projecting the feature quantity i calculated from the region image data to be evaluated onto the evaluation axis is the evaluation unit 8. It is suitable for the evaluation performed.

Next, an example in which the management unit 7 updates an index for evaluating the feature quantity i of the discriminator i every time the evaluation target region image data is input to the calculation unit 6 will be described.
The area image data is further classified based on the result (diagnosis result) evaluated by the evaluation unit 8 regarding the roughness of the substantial area of the liver. A feature amount is calculated from the classified data (image data), and a discriminator is created. For example, when the management unit 7 creates a discriminator using a linear discriminating method based on an 8-dimensional feature amount calculated from data classified into two stages with respect to roughness, the discriminator (learning data) linearly The coefficient terms (eight) and constant terms of the discriminant can be calculated. In this case, the calculated linear discriminant can be used as a discriminator.

Next, the setting unit 4 will be described.
Although the setting unit 4 is set to output all the information of the results evaluated by the evaluation unit 8, the setting unit 4 is not limited thereto, and may be set to output only part of the information of the results evaluated by the evaluation unit 8. It is.

  For example, when a doctor designates a certain position on the screen of the display unit 5 and sets an index, an attention area having the designated position as a central coordinate can be set, whereby a liver real area within the attention area can be set. The result (diagnosis result) of evaluating the roughness of the image can be output (displayed). In this case as well, the index of the attention area can be set freely. For example, the shape of the attention area may be rectangular or circular. In addition, when outputting (displaying) the result of evaluating the roughness of the liver real area, it is output (displayed) with a numerical value within the normalized range, or the numerical value is converted into color information and the color information is output (displayed) ).

Next, the feature amount will be described.
The feature quantity may be a higher-order local feature quantity or a lower-order local feature quantity calculated for each pixel group. Examples of the higher-order local feature amount include a local binary pattern (LBP) feature amount and a higher-order local autocorrelation (HLAC) feature amount.

  For example, local binary pattern feature quantities as one of the higher-order local feature quantities (texture feature quantities) are characterized by high robustness against image brightness changes, rotation and scale conversion, and low calculation costs. , Often used in face recognition and other fields.

  The local binary pattern feature amount is a method of extracting texture feature information of an image by calculating a pattern histogram based on a luminance difference between a point of interest (point of interest) in the image and a surrounding point. . Examples of local binary patterns when the positions and scales of the attention point and the surrounding points are different are shown in FIGS.

  In FIG. 12, there are four surrounding points around the point of interest (P = 4), and the distance (radius) from the point of interest to the surrounding point is 1.0 (R = 1.0). 13 to 16 are the same as FIG. 12, P = 8 and R = 1.0 in FIG. 13, P = 12 and R = 1.5 in FIG. 14, P = 16 in FIG. R = 2.0, P = 24 and R = 3.0 in FIG.

  For example, the calculation unit 6 can calculate a local binary pattern feature amount without shifting for each pixel group with respect to an image of a substantial region of the liver imaged using ultrasound. And the evaluation part 8 can evaluate the property (roughness) of the substantial area | region of a liver based on the calculated feature-value.

Next, the management unit 7 will be described.
The management unit 7 can create and manage the discriminator i using a linear discrimination method or a nonlinear discrimination method based on the feature quantity i. Examples of the linear discriminating method include linear discriminant analysis (LDA), support vector machine (SVM), principal component analysis method and the like. Nonlinear discriminant analysis (QDA: Quadratic Discriminant Analysis) etc. can be mentioned as a nonlinear discriminating method.

  From the above, it can be seen that it is possible to extract the evaluation axis that maximizes the degree of separation by compressing the dimension by the principal component analysis method for the feature quantity i calculated from the learning data. It is also possible to calculate a support vector machine training model using learning data in advance.

Next, the evaluation unit 8 will be described.
The evaluation unit 8 can perform evaluation based on the length of the spatial distance of the feature quantity i or the degree of similarity. In addition, the evaluation part 8 can also evaluate with another parameter | index. From the above, when information (new information) as a result of evaluation by the evaluation unit 8 is given, the result of determination can be calculated by substituting the feature quantity i calculated from the information into the identification formula.

  When the discriminator i is created and managed using the linear discriminant analysis method, it is possible to classify the evaluated result by detecting whether it is greater than 0. It is also possible to calculate Mahalanobis distance and classify the evaluation results in ascending order of distance. It is also possible to classify the results evaluated by other methods.

Next, the setting unit 4 will be described.
The setting unit 4 can set the information of the result of evaluation by the evaluation unit 8 so that the evaluation unit 8 outputs the information as information of one or more dimensions. Therefore, the evaluation result information can be displayed as one-dimensional information (for example, a numerical value or a color corresponding to the numerical value), and can also be displayed in two dimensions or multidimensional.

  For example, FIG. 17 is a conceptual diagram in a case where information on a result of evaluating a substantial area of the liver is displayed in one dimension. As shown in FIG. 17, when the parenchymal region of the liver is evaluated and classified into three stages of normal, chronic hepatitis, and cirrhosis, the distance from normal to cirrhosis is normalized to 0 to 1, and information on the evaluation results (Evaluation data) can be displayed quantitatively.

FIG. 18 is a conceptual diagram in a case where information on the result of evaluating the liver internal tissue is displayed in two dimensions. As shown in FIG. 18, by displaying in two dimensions, it is possible to display the results of evaluating the liver internal tissue in a more diversified manner. In the figure, data indicating normality is indicated by circles, data indicating chronic hepatitis is indicated by squares, and data indicating cirrhosis is indicated by crosses.
The case where the region image data of the substantial region of the liver is evaluated using the ultrasonic diagnostic system 1 has been described above.

  According to the ultrasonic diagnostic system 1 configured as described above, the ultrasonic diagnostic apparatus 2 includes the calculation unit 6, the management unit 7, and the evaluation unit 8. The calculation unit 6 calculates a feature amount 0 from the image data, and the evaluation unit 8 can evaluate the feature amount 0 using the discriminator 0, classify the image data into a plurality of area image data, and output the image data. . Thereby, the ultrasound diagnostic apparatus 2 can objectively classify the image of the subject into a plurality of regions.

  Further, the evaluation unit 8 uses the discriminator i, for example, region image data without depending on the setting of the index (contrast or the like) of the ultrasonic diagnostic apparatus 2 and without performing threshold setting or optimization processing. Can be objectively evaluated.

  In addition, each time image data or region image data is input to the calculation unit 6, the management unit 7 is based on information obtained as a result of classification or evaluation by the evaluation unit 8. It is possible to update an index for evaluating That is, the evaluation result can be taken into the discriminator of the management unit 7 as learning data. The discriminator of the management unit 7 can also incorporate know-how of doctors and experts. Thereby, the subjective judgment of the operator (such as a doctor) can be quantified.

  From the above, it is possible to obtain the ultrasonic diagnostic system 1 that can objectively classify a subject into a plurality of areas and can evaluate the classified areas. And the ultrasonic diagnostic system 1 useful for an operator can be obtained.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the components without departing from the scope of the invention in the implementation stage. Various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiments. For example, some components may be deleted from all the components shown in the embodiment.

  For example, the ultrasonic diagnostic apparatus 2 itself may include the imaging unit 3 and the display unit 5. The image data input to the calculation unit 6 and the display unit 5 may be obtained by means other than the imaging unit 3.

  For example, a solar panel or a turbine may be used as another example of the subject. In a solar panel, it is possible to evaluate whether a normal panel or a deteriorated panel by using a thermographic image. Further, in the turbine, the roughness of the turbine surface can be evaluated.

  The image processing system of the present invention is not limited to the ultrasonic diagnostic system 1 but can be variously modified, and can be applied to various image processing systems such as other ultrasonic diagnostic systems.

  DESCRIPTION OF SYMBOLS 1 ... Ultrasonic diagnostic system, 2 ... Ultrasonic diagnostic apparatus, 3 ... Imaging part, 4 ... Setting part, 5 ... Display part, 6 ... Calculation part, 7 ... Management part, 8 ... Evaluation part.

Claims (7)

A first arithmetic unit that receives image data obtained by imaging a subject including a plurality of regions, and calculates a feature amount that serves as an index for classifying the image data into the plurality of region image data;
Creating and managing a discriminator having an index for classifying the image data into the plurality of area image data based on a feature quantity serving as an index for classifying the image data into the plurality of area image data; The management department of
An identification having an index for classifying the image data into the plurality of region image data, and a feature amount serving as an index for classifying the image data calculated by the first arithmetic unit into the plurality of region image data A first evaluation unit that classifies the image data into the plurality of area image data according to the evaluation result, and outputs the classified area image data;
A second calculation unit that calculates a feature amount serving as an index for identifying a state of a region corresponding to the region image data from the region image data classified by the first evaluation unit;
Creating and managing a discriminator having an index for identifying the state of the region corresponding to the region image data based on a feature quantity serving as an index for identifying the state of the region corresponding to the region image data; The management department of
An identification having an index for identifying the state of the region corresponding to the region image data, as a feature quantity that is an index for identifying the state of the region corresponding to the region image data calculated by the second arithmetic unit An image processing apparatus comprising: a second evaluation unit that performs evaluation using a container and outputs information of the evaluation result.   An index for calculating a feature value corresponding to the image data is set in the first calculation unit, and an index for calculating a feature value corresponding to the region image data is set in the second calculation unit. The image processing apparatus according to claim 1, further comprising a setting unit.   The image processing apparatus according to claim 1, further comprising a setting unit that sets a region of interest from the image data or the region image data, and classifies or evaluates the region of interest.   The image processing apparatus according to claim 1, further comprising a setting unit that can be set so that the first evaluation unit outputs all or only a part of the plurality of classified area image data.   The image according to claim 1, further comprising a setting unit that can be set to cause the first evaluation unit to output not only information on the evaluation result but also information on the region image data to be evaluated. Processing equipment.   The first evaluation unit further outputs information indicating a feature amount serving as an index for classifying the image data into the plurality of region image data, and the second evaluation unit further corresponds to the region image data. The image processing apparatus according to claim 1, further comprising: a setting unit that can be set to output information indicating a feature amount that serves as an index for identifying the state of the region.   The setting part which can be set so that the said 2nd evaluation part may integrate and output the information of the said evaluation result regarding the said several classified area | region image data is provided. Image processing device.

Images