JPH08293025A - Image sorting device - Google Patents

Abstract

PURPOSE: To provide an image sorting device which can sort such parts that are hard to be extracted out of the input images only through the binarization or the three-dimensional display and then can give the instructions that are visually and easily understood. CONSTITUTION: An image sorting device is provided with an image input device 1 which fetches the image data, a filing device 2 which stores the input image data and then reads out, retrieves and edits the image data through every prescribed operation, and a normalization part 3 which corrects the variance caused by various conditions of an image input mode or the device 1 and secures the coincidence of input conditions between the execution and learning modes. Furthermore, a feature extraction part 4 is added to extract the feature value that is effective to the sorting of images, together with a sorting decision part 5 which decides the sorting based on the feature value extracted by the part 4, a display part 6 which synthesizes the sorting decision result and displays it on the input image, and a learning control part 7 which controls both parts 4 and 5 based on the prescribed learning data.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for inputting image information and making a classification decision.

[0002]

2. Description of the Related Art Generally, photographing devices such as cameras and infrared cameras, endoscope images, microscope images, ultrasonic image input devices,
X-ray image input device, MRI (Magnetic Resonance Imagi
ng) device, CT (Computer Tomography) device, and the like, there are image input devices that classify and display images from several predetermined types.

For example, as a first example, "image laboratory"
(February 1993, p. 19) describes an example applied to a device for extracting a malignant tumor by binarization. In addition, in this example, in the case where the primary liver cancer is displayed three-dimensionally, the judgment is made from the irregularity in the projected image.

In addition, as a second example, "Konik"
a Technical Report ", Vol.
6, 1993, P55, an example of performing a feature analysis of a difference between an image in which a specific pattern of a lesion part is emphasized and an image in which the pattern is attenuated is described.

[0005]

However, the above-mentioned first problem
In the example of a device for extracting a malignant tumor from a binarized image according to the above example, it is difficult to set the binarization level to detect as malignant, and it is effective only for a tumor that originally has a clear difference.

Also, in the second example, the shape of the lesion in the image is limited, but in reality, there are various shapes. Therefore, the pattern that can be detected is fixed, and there is a problem that the detection accuracy is lowered even if the pattern is slightly different.

Therefore, the present invention uses 2 in the input image.
An object of the present invention is to provide an image classification device capable of accurately classifying even a portion that is difficult to extract only by binarization or three-dimensional display and giving an instruction visually easily understandable.

[0008]

In order to achieve the above object, the present invention provides an image classification device having the following configuration. First, at least information input means for inputting information to be classified, feature extraction means for dividing the effective area of the information input by the information input means and other areas, and extracting the feature information, There is provided an image classification device including a classification determination unit that determines the type of information based on the characteristic information extracted by the characteristic extraction unit and classifies the information for each type.

Secondly, at least one of a power spectrum, a difference statistic, a co-occurrence matrix and a run length matrix is selected from the information input means for inputting information to be classified and the information input by the information input means. Using a feature extraction means for extracting feature information, and at least a classification determination means for determining the type of information based on the feature information extracted by the feature extraction means and classifying the information for each type. An image classification device is provided.

Thirdly, information input means for inputting information to be classified, feature extraction means for extracting feature information from the information input by the information input means, and feature information extracted by the feature extraction means An image classifying apparatus is provided which is configured to classify a specific region a plurality of times by using the classification result and classify the final classification based on the classification result.

Fourth, information input means for inputting information to be classified, characteristic extraction means for extracting characteristic information from the information input by the information input means, and at least characteristic information extracted by the characteristic extraction means. A classification determining unit that determines the type of information based on the above, and a learning unit that reconstructs the feature extracting unit and / or the classification determining unit by using the classification result of the classification determining unit. There is provided an image classification device configured by.

Fifth, information input means for inputting information to be classified, characteristic extraction means for extracting characteristic information from the information input by the information input means, and at least characteristic information extracted by the characteristic extraction means. An image classification device is provided that is configured with a classification determination unit that determines the information type based on the above and repeats feedback of a plurality of types.

[0013]

In the image classifying apparatus having the above structure,
With the first configuration, the characteristic information for dividing the input image information into the effective area and the area other than the effective area is extracted, and the information type is determined based on the extracted characteristic information and the image information. It is determined and classified according to this type.

According to the second configuration, at least the difference statistic or the Lang length matrix is used from the power spectrum, the difference statistic, the co-occurrence matrix, and the Lang length matrix in the input image information, and the image information is extracted by the feature extraction means. The type of information is determined based on the characteristic information and / or the information input from the information input means, and the information is classified for each type.

With the third configuration, the characteristic information is extracted from the input image information, the specific area is classified and determined a plurality of times using the characteristic information, and the characteristic information to be finally classified according to the classification result and The type of information is determined based on the image information, the information is classified for each type, and the classification determination result is displayed.

With the fourth configuration, the characteristic information is extracted from the input image information, the type of information is determined based on the characteristic information and the image information, and the information is classified for each type, and these classification results are used. Then, the feature extraction means and / or the classification determination means are reconstructed.

According to the fifth configuration, the characteristic information is extracted from the input image information, the type of the information is determined based on the characteristic information and the image information, and a plurality of types are repeatedly fed back to obtain each type. Classify.

[0018]

Embodiments of the present invention will be described below in detail with reference to the drawings. First, in the image classification apparatus of the present invention, an outline will be described by taking an image classification apparatus that provides diagnostic support for a lesion portion of a tumor whose difference is difficult to visually recognize.

This image classification device analyzes the texture pattern of the lesion by learning using a statistical method,
The feature is that feature quantities effective for classification are extracted. Then, if the various conditions at the time of image input are different, even the same lesion will have different images, so that the conditions at the time of image input and the variations of the image input device are also corrected (normalized).

Particularly, when the input image is an ultrasonic image, frequency, waveform, and TGC (Time Gain Control) are used for normalization.
, DR (dynamic range), gain, and how to cut out a determination region based on pixel position.

Further, since it is difficult to capture the features of the lesion by merely binarizing or three-dimensional display, many features are subjected to multivariate analysis to select the features or processing method effective for determining the lesion. Examples are also included. In order to make this selection, there is a mode in which the apparatus has two modes, a learning mode and an execution mode.

In the learning mode, an experienced doctor or the like designates a normal area and a lesion area on an image, creates learning data, performs various preprocessings on the learning data, and analyzes the statistics with multivariate analysis. By doing so, a feature amount effective for determining a lesion part is extracted.

Then, in the execution mode, the feature quantity determined in the learning mode is extracted to judge the lesion area.
If necessary, feedback is provided to the feature extraction unit and the classification determination unit according to the determination result to improve the precision of the determination. It is possible to perform the above-described normalization in both the learning mode and the execution mode.

Then, an example in which the determination result is combined with the original image and displayed is included. For example, in an ultrasonic image or an X-ray image, the original image is monochrome, and the area determined to be a lesion is displayed with a color.

FIG. 1 shows an example of the image classification device of the present invention. FIG. 1A shows the configuration of the image classification apparatus when performing classification, and FIG. 1B is a diagram showing the configuration of classification learning performed prior to executing classification.

The structure of the image classification device shown in FIG.
An image input device 1 for taking in image data and CT image data by an ultrasonic image input device, an X-ray image input device, and the like, and image data input from the image input device 1 are stored, read out by a predetermined operation, and searched. , A filing device 2 for editing, a normalizing unit 3 for correcting various conditions at the time of image input and a variation due to the image input device, and for performing a normalization process for matching input conditions at the time of execution and learning, and effective for classification Feature extraction unit 4 for extracting various feature amounts, and feature extraction unit 4
The classification determination unit 5 that performs classification determination described below from the obtained feature amount and the display unit 6 that displays the result of the classification determination combined with the input image are displayed. In FIG. The feature extracting unit 4 based on the determined learning data.
And a learning control unit 7 that controls the classification determination unit 5, respectively.

Further, the feature extraction unit 4 is obtained by the pre-processing unit 8 for extracting the feature amount effective for diagnosis, for example, the first-order statistic, etc. from the input image as a pre-stage, and the pre-processing unit 8. The multivariate analysis unit 9 is configured to statistically analyze the feature amount and convert or reduce the feature amount (decrease in the number of dimensions).

In the image classifying device thus constructed, the image data input from the image inputting device 1 is stored in the filing device 2 as necessary, and thereafter learning and classification judgment are performed. Then, the normalization unit 3 improves the classification determination accuracy at the time of execution.

From the image data subjected to predetermined processing by the filing device 2 and the normalizing section 3, a plurality of characteristic quantities effective for diagnosis are extracted from the input image by the preprocessing section 8, and these are calculated by the multivariate analysis section 9. A feature amount effective for classification is selected from the feature amounts to improve the classification accuracy, and the classification determination is performed at high speed by reducing the number of dimensions.
Further, the preprocessing unit 8 can select the optimal classification feature amount according to the object by using the multivariate analysis unit 9 together.

The feature extracting unit 4 extracts the feature amount for automatically classifying the image data, and the classification determining unit 5 makes a determination based on the feature amount. The classification determination result determined in this way is visually displayed on the display unit 6 and displayed in an easy-to-understand manner.

Further, the learning control section 7 incorporates highly reliable classification judgment data by experience into the normalizing section 3, the feature extracting section 4 and the classification judging section 5, respectively, so that even a beginner can make a high-level classification judgment. it can.

In the image classifying apparatus thus configured, when the execution mode shown in FIG. 1A and the learning mode shown in FIG. 1B are not selected, the image input from the image input apparatus is directly displayed on the display unit. 6 is displayed.

The execution mode will be described with reference to FIG. First, by the image input device 1, DR, Gai
Input an image in which n, magnification, etc. are normalized. Coordinate information is sequentially given from a control unit (not shown), the cutout direction of the region is normalized, and the feature extraction unit 4 performs preprocessing and multivariate analysis to perform classification determination. Feedback calculation is performed as necessary, and the result is combined with the original image and displayed on the display unit 6. Coordinate information within the designated range is sequentially given from the control unit, and this series of processing is performed, and the designated range is displayed in color, for example.

Next, FIGS. 2 and 3 show an example in which a convex ultrasonic device is used as an image input device of the image classification device of the first embodiment according to the present invention. In this convex type ultrasonic device 10, 128 transducers 11 which are divided into eight groups and sequentially select transmission and reception of ultrasonic waves, and transmission and reception from the transducers 11 are switched to a predetermined group. An element selection switch 12 that simultaneously turns on and off by a number (the predetermined number is called an aperture, eight in this embodiment), and eight transducers connected to one of the switching terminals of the element selection switch 12. And a transmission circuit 13 for transmitting a pulse from which a waveform that converges to a predetermined focus point is formed, and the element selection switch 1
The rotary switch 14 which is connected to the other end of the switching terminal 2 for sending the eight received signals to the predetermined delay line 15 and the received wave is a spherical wave, the signals are received according to the position of each transducer. The delay time is different, and the delay line 15 for temporally correcting the received signal, the adder 16 for adding the output signal from the delay line 15, and the distance in the present embodiment are the same as the time TGC (Time).
It is called a "Gain Control", and is composed of an amplifier 17 that amplifies the amplitude attenuated according to the distance.

This TGC is a TGC fixed waveform generating circuit 132 which performs a predetermined amplification by the action of a changeover switch 130 whose switching is switched by a normalization control section 43 described later, or a TGC waveform which amplifies based on an external setting signal. It is connected to the generation circuit 131. In the learning mode and the execution mode, the TGC fixed waveform generation circuit 132 is switched and normalized to a predetermined amplification value.

Further, the output signal from the amplifier 17 (the signal received by the transducer 11) is once transmitted to a high frequency (f ₀
+ F _H ), a multiplier 18, a BPF 19 for extracting a signal in a predetermined band, multipliers 20 and 21 for shifting to a zero frequency, and a double conversion circuit 24 including LPFs 22 and 23, and outputs from the LPFs 22 and 23. A / D converter 2 for A / D converting each signal
5 and 26, and the waveform of the ultrasonic wave shown in FIG. 6A, which is the waveform of the output signal (ultrasonic wave) that has been A / D converted, is shown in FIG.
Two digital filters 27, 2 for correcting to the waveform shown in
8, a waveform correction circuit (normalization unit) 29, square detection circuits 30 and 31 for calculating output values from the outputs of the digital filters 27 and 28, and a detection circuit 33 including an adder 32.
A logarithmic amplifier 34 for compressing a dynamic range with respect to the detection output; a scan converter 35 capable of performing scanning line conversion in accordance with a display unit of an image (not shown) and controlling the display magnification on the display unit; , Or in the execution mode, it controls the transmission circuit 13 and the normalization unit 36 that sets the DR (dynamic range), the gain (Gain), and the magnification to predetermined values Vref1, Vref2, and Vref3, respectively.
A control unit 37 for setting the DR, the gain, and the magnification to desired values of the user.

In the convex type ultrasonic device having such a configuration, when the frequency of the vibrator 11 is f ₀ , a transmission / reception signal having a waveform as shown in FIG. 6A is input. Then, these signals are selected by switching the element selection switch 12 and the rotary switch 14,
The delay line 15 temporally corrects and aligns the signals.
The corrected signal is once converted to a high frequency (f ₀ +) by the double conversion circuit 24, as shown in FIG. 6B.
f _H ), and efficiently extract only the signal band,
Then it is shifted to the lower frequencies. This processing reduces the load on the circuit.

Further, a waveform correction circuit (normalization section)
According to 29, when the characteristic is different from the learned oscillator, the correction is performed by the filter. It is also corrected when the learned oscillator changes over time. It is also possible to correct the frequency attenuation with respect to the distance (time).

The two output signals from the waveform correction circuit 29 are subjected to predetermined detection by the detection circuit 33. Next, the detected signal is normalized by the normalizing unit 36 only in the learning mode and the execution mode, and the DR, gain, and magnification in the learning mode and the execution mode are normalized to predetermined values. Further, the scan converter 35 performs scanning line conversion adapted to a display unit (not shown) and also controls the display magnification on the display unit.

FIG. 4 shows the structure of the image classification apparatus of the first embodiment incorporating the above-mentioned convex type ultrasonic wave apparatus. The image classification apparatus shown in FIG. 4 shows the configuration at the time of executing the classification determination (execution mode), and the constituent members are the same as those shown in FIG. To do. In this embodiment, an image classification device classifies a pathological form such as a tumor occurring in a human body as an image.

In this image classifying apparatus, roughly, the ultrasonic wave apparatus 10 of the convex type, the range for performing the classification judgment in the image, and the size of the extraction region used for the classification judgment are designated, and the coordinate information is sequentially output. Range designation section 41
And a normalization control unit 43 that controls the normalization units 29 and 36, the changeover switch 130, and the normalization unit 42 in the ultrasonic device 10 to switch between normal diagnosis and determination classification processing (learning mode, execution mode). A frame memory 44 that temporarily stores the image signal from the ultrasonic device 10 on a frame-by-frame basis, a filing device 2, a feature extraction unit 4, a classification determination unit 5, and a display unit 6.

For example, the predetermined values Vref1, Vref2, Vref3 in the normalize section 36 and the amplification value of the TGC fixed waveform generating circuit 132 are set to desired values by the condition setting means (not shown). The normalizing section 4
2 is the tilt angle θ at that position from the designated coordinate value P
And the distance d is calculated by the equation (1). (See Figure 7)

[0043]

[Equation 1]

The normalization group 42 includes a θ and d calculator 45a which can easily calculate θ and d from the coordinates of four points around the convex type image, and θ and d from the θ and d calculator 45a. The extraction area is deformed in accordance with. That is, it is rotated in a rectangle (Fig. 8 (a)) or cut out in a trapezoid (Fig. 8).
By performing (b)), the region extracting unit 45b corrects the difference in the characteristics of the convex type ultrasonic image having different imaging characteristics (frequency characteristics) depending on the image position.

Further, the range designation section 41 allows the user to easily designate the area in the image to be classified and determined, and designates the range to be classified and the size of the extraction area used for classification determination, and coordinates Specify the information.

Next, the feature extraction section 4 comprises a preprocessing section 46 and a multivariate analysis section 47. In this preprocessing unit 46,
When a power spectrum P (u) having a lot of information effective for classification of an image, which will be described later, is detected by multiplying by a window function such as Hamming or Hanning, a window circuit 48 for removing the influence of the edge of the image peripheral portion, and FIG. Power spectrum P using Fourier transform as shown in (a)
And a power spectrum detector 49 for detecting (u). Here, FFT means a fast Fourier transform, and the output real number term and imaginary number term are squared and added. In the power normalization unit, the zero frequency is processed so as to have a constant value, and the difference in image brightness is normalized. In addition, the power processing unit converts the values as shown in FIGS.

The multivariate analysis unit 47 has the power spectrum P detected by the power spectrum detector 49.
By storing (u), it is composed of a classification data memory 50 and a dimension conversion unit 51 that are repeatedly used in feedback control. The dimension conversion unit 51 stores a classification vector calculated at the time of learning, and a classification vector memory 52 in which the value of the classification vector is selected according to the value of d calculated by the θ, d calculator 45a and the classification number, By performing an inner product operation of the power spectrum P (u) read from the classification data memory 50 and the selected classification vectors C1 (u) and C2 (u) and outputting the results t1 and t2,
It is composed of a projection calculator 53 that converts into a feature space effective for classification, reduces the number of dimensions without reducing the classification performance, and reduces the input to the classification determination unit. This projection calculator 53 is shown in FIG.
As shown in, consists of a multiplier and a cumulative adder in parallel,
Classification vector C1 selected for each multiplier
(U) and C2 (u) are input, the inner product operation is performed, and the results t1 and t2 are output. With this dimension conversion unit, the input unit of the classification determination unit 5 can be reduced, and the classification determination unit 5 can be downsized, that is, the processing speed can be increased.

Next, the classification judgment unit 5 converts the input signal into a form suitable for logical judgment, and the ambiguity judgment unit 5
4 and a logical judgment unit 55 which is a logical judgment circuit such as AI.
And a control section 58 for instructing the classification number to the weight memory 57 and the classification vector memory 52 based on the output of the logic determination section 55.

Further, the use of the neural network has an advantage that the classification boundary can be easily constructed even if the classification boundary is complicated. This ambiguity determination unit 54
In the above, by outputting the judgment values J1, J2 and J3 from t1 and t2 from the projection computing unit 53, the correct evaluation value based on the learning data can be obtained even for the delicate judgment of normal part or abnormal part. (J1, J2, J3) can be output, and is composed of a three-layered neural network 56 capable of analog (intuitive) evaluation as performed by a person, and a weight memory 57 for storing the weighting coefficient of this neural network 56. It

The weight memory 57 is used by the control unit 5
8 selects the weighting coefficient according to the classification number and the value of the distance d, and sets the weighting coefficient as the weighting coefficient of the neural network 56, so that the weighting memory value suitable for the classification purpose can be selected, and the imaging characteristics different depending on the distance are corrected. It is possible to improve the classification accuracy. However, in the output of the neural network 56, J1 indicates a normal portion, J2 indicates an abnormal portion (malignant), and J3 indicates an abnormal degree (benign), and values of 0 to 1.0 are output. In addition, the logic determination unit 55
Selects a process according to the values of J1, J2, and J3 as in [Judgment Table 1] described later. In the case of Case 4, it is abnormal but it is not known whether it is malignant or benign, and the classification vector and weight learned so as to classify only malignant or benign are selected and re-executed.

When the judgment of the logic judgment unit 55 is ambiguous in this way, the accuracy of the classification judgment is improved by performing a feedback process to the classification judgment unit 5 or the preceding stage thereof to make the judgment. Whereas the neural network 56 makes an analog (intuitive) evaluation, the logic judging section 55 can make a digital (logical) evaluation, and can also make a more complicated classification judgment. When the number of times of feedback is equal to or greater than the predetermined number, the determination value 5 (meaning that classification is unknown) is output.

The judgment value output from the logic judgment unit 55 is
It is sent to the display unit 6. The display unit 6 includes a display image generation unit 59 and a monitor 60 that is a display such as a CRT or a flat panel, and displays, for example, an input image in different colors according to a determination value. For example, judgment value 1 (normal) is green, judgment value 2 (malignant) is red, judgment value 3
By coloring the area in blue for (benign) and yellow for the judgment value 5 (unknown), the classification judgment result can be displayed in a visually easy-to-understand manner, and useful diagnosis support can be provided.

The value of the classification vector and the value of the weight memory are selected according to the control information (distance d here) and / or the classification number output from the normalizing section 42 to improve the classification accuracy. You can

Next, the learning mode will be described with reference to FIG. First, with the image input device 1, the TGC,
An image in which the waveform, DR, Gain, and magnification are normalized is input, appropriately frozen, and stored in the filing device 2. Then, under the control of the learning control unit 7, a designated image is read out from the images to be filed, displayed on the display unit 6, and an operator (experienced person) designates a desired area in the display to give teacher information. . The area specified in this way is normalized such as the cutting direction,
The feature extraction unit performs preprocessing and multivariate analysis to create learning data, and the classification determination unit performs learning.

Then, FIG. 5 shows a configuration at the time of classification determination learning (learning mode) in the image classification apparatus shown in FIG. The same members as the members shown in FIG. 4 among the members configured here are designated by the same reference numerals, and the description thereof will be omitted.

The image classification device at the time of this classification judgment learning is
The coordinate information P is output to the normalization controller 43 and the display image generation unit 59, and the multivariate analysis unit 47 and the ambiguity determination unit 54 are output.
A learning control unit 61 for learning a predetermined classification number is provided for each.

The learning control unit 61 includes a teacher data input unit 62 for inputting data such as a case by an experienced person such as a doctor, and a learning data memory 63 for recording the results t1 and t2 from the projection calculator 53. , A learner 64 that generates a value of a weight memory from the data from the learning data memory 63 and the teacher data, and a classification vector calculation unit 65 that calculates a classification vector from the classification data from the classification data memory 50.
And a learning controller 66 that controls the entire learning. The feature extraction unit 4 uses the calculated classification vector.
Will be set.

The teacher data input section 62 is a user interface with an experienced person such as a doctor, and while the experienced person is looking at the screen displayed on the monitor 60, the teacher data input section 62 is displayed in the monitor 60 by a mouse, a keyboard or the like (not shown). By moving the window, the teacher data such as the positions of the normal part, the abnormal part (malignant), and the abnormal part (benign) is input. In addition, by displaying the image area at the designated position in different colors, the area input as the teacher data can be easily visually recognized. Further, θ and d are calculated from the coordinate information of the designated position. In this way, the useful information of the experienced person can be easily input, so that data such as new cases can always be easily input later and learning can be performed again. Further, as the teacher data, not only the information on the normal part, the abnormal part (malignant), and the abnormal part (benign), but also the name of the abnormal part may be input.

The learning controller 66 controls the learning as a whole according to the instruction from the teacher data input unit 62 to calculate the values of the classification vector memory 52 and the weight memory 57 according to the distance d and the classification number. Output information. This d is, for example, as shown in FIG.
The case is divided into a plurality of four stages such as 1, d2, d3 and d4. The classification numbers are 2 of classification numbers 1 and 2 as described later.
Set in multiple stages. As a result, the accuracy of classification determination can be improved.

The learning device 64 is a learning device for learning the neural network 56, and the learning data memory 63 is a memory for storing learning data input to the neural network 56.

The classification data memory 50 stores the power spectrum P calculated by the power spectrum detector 49.
Memorize (u). For each teacher data (normal or abnormal (malignant))
Or abnormal (benign)), for each image distance d (d1, d2, d
3, d4).

The classification vector calculator 65 calculates a classification vector from the power spectrum P (u) stored in the classification data memory 50. The details of this calculation, as described in Japanese Patent Application No. 5-264781 proposed by the present applicant, calculate a vector for classifying two classes by a statistical method using FS conversion or the like. Classification vector 1 (classification number 1) for classifying two classes that mix normal and abnormal (malignant) and abnormal (benign), and two classes used for feedback: abnormal (malignant) and abnormal (benign) Then, a classification vector 2 for calculating is calculated (classification number 2). The classification vectors 1 and 2 are distances d1, d2,
It is also calculated for each of the distances d3 and d4 and is written in a predetermined address of the classification vector memory 52. Similarly, the value of the weight memory 57 is calculated according to each classification number and each distance, and is written in a predetermined address.

The operation of the thus-configured image classifying apparatus will be described. In this image classification device, when the learning mode and the execution mode are not selected, the changeover switch 130 of the ultrasonic device serving as the input device and the switch A side (FIG. 3) of the normalize unit 36 are selected and the TGC amplification specified by the user is performed. The value, DR, Gain, and magnification are selected, input is made, and displayed. When the learning mode or the execution mode is selected, the switch B is selected and T
The GC amplification value, DR, Gain, and magnification are normalized to predetermined values and input, and feature extraction and classification determination are performed.

In this embodiment, the normal part and the abnormal part (malignant),
The three types of abnormal parts (benign) are classified. That is, these three types of areas are designated in the learning mode. Also, there are 1 and 2 as classification numbers, which are switched depending on the purpose of classification. The purpose of classifying the normal part and the abnormal part (both malignant and benign) is classification number 1, and the classification of the abnormal part (malignant) and abnormal part (benign) is classification number 2. Learning is performed for each classification number. Further, in the execution, the classification number 1 is executed, and when the judgment is delicate, the feedback control is performed by selecting the classification number 2.

The learning mode will be described with reference to FIG. The image in which the TGC amplification value, waveform, DR, Gain, and magnification are normalized is appropriately frozen and stored in the filing device 2. The image designated by the control of the learning control unit 61 is read from the filing device 2,
The monitor 60 of the display unit 6 is stored in the frame memory 44.
Is displayed in.

Then, an experienced person such as a doctor specifies a region such as a lesion with a mouse or the like not shown, and gives teacher information such as the type of disease. The specified area is normalized such as the cutting direction, the power extraction unit 4 first obtains the power spectrum P (u), and the power spectrum P (u) is stored in the classification data memory 50. The learning control unit 61 selects and uses data from the classification data memory 50 according to the classification number to calculate the weight of the classification vector and the neural network 56. This calculation is performed not only for each classification number but also for each distance d.

Instead of temporarily recording in the filing device 2, a region may be directly designated by an image from the ultrasonic device 10 to give teacher information. Further, the correction of the magnification may be performed at the same time when the region is cut out (region extraction unit 45b). In addition, this learning control unit
It may be detachable, or the value of the weight memory and the classification vector memory may be calculated by another computer as the off-line processing. Further, in this embodiment, the FS transform is used as a statistical method for obtaining a classification vector, but HT known as a method for classifying multiple classes
You may use C (Hotelling Trace Criterion).

Next, the execution mode will be described with reference to FIG. With the ultrasonic device 10 serving as an image input device,
An image in which the TGC amplification value, waveform, DR, Gain, and magnification are normalized is input. The coordinate information is sequentially given from the range designation unit 41, the cutout direction of the region is normalized, and the feature extraction unit 4 performs preprocessing (power spectrum calculation) and multivariate analysis to obtain a value t1,
t2 is input to the classification determination unit 5. In this classification determination unit 5, the neural network 56 is used to select J1, J2, J3.
Is output, the logic determination unit 55 makes a determination according to this value, and feedback calculation is performed as necessary.
The result is combined with the original image and displayed on the display unit 6. The above processing is performed over all the corresponding areas in the range designation unit 41.

The input is an image input device (ultrasonic device 1
0) may be directly input, or may be a frozen image or a moving image sequentially sent. Furthermore,
The image recorded in the filing device 2 may be read out.

In this embodiment, the neural network has a three-layer hierarchical structure, but the present invention is not limited to this, and of course, two layers or multiple layers may be used.
A feedback type (fully connected type) may be used. Further, in this embodiment, the power is used by using the Fourier transform, but other orthogonal transforms such as cosine transform and wavelet transform may be used.

[Judgment Table 1] Case 1: J1 ≧ 0.7 and J2, J3 ≦ 0.3 Judgment is made as a normal portion, and judgment value 1 is output. Case 2: J2 ≧ 0.7 and J1, J3 ≦ 0.3 The abnormal part (malignant) is judged and the judgment value 2 is output. Case 3: J3 ≧ 0.7 and J1, J2 ≦ 0.3 The abnormal part (benign) is judged and the judgment value 3 is output. Case 4: J1 <0.5 and (J2> 0.5) It is judged as an abnormal part, and the judgment value 4 is output.

Or J3> 0.5) Select a classification spectrum and a weight memory (classification number 2) for identifying an abnormal part and execute again (feedback). Case 5: Other than the above It is judged that the classification is unknown and the judgment value 5 is output.

Next, as a second embodiment, a modification of the above-mentioned first embodiment will be described. In this modification, the θ, d calculator 45a shown in FIG. 4 is configured as shown in FIG. The density histogram calculation unit 71 obtains a histogram relating to the density of the entire ultrasonic image. The adaptive binarization processing unit 72 binarizes the image according to the histogram obtained by the density histogram calculation unit 71. When performing the binarization, the binarization may be performed by using the information that the concentration value of the portion other than the diagnostic organ, which is a characteristic of the ultrasonic waves, is almost zero. By this adaptive binarization processing, the region can be divided into a diagnostic organ and other portions as shown in FIG.

Next, the contour line extraction unit 73 performs contour line extraction for dividing the image into the diagnostic organs and other regions. A straight line is detected by the line segment detection 74 from the image from which the contour line has been extracted. A well-known technique such as hough transform or graph research can be used for the line segment detector to easily detect the line segment. 12
Also displays the detected line segment.

Assuming that each of the two detected line segments is y = ex + f y = gx + h, Cx and Cy in FIG. 7 can be calculated as Cx = (f−h) / (g -E) Cy = (gf-eh) / (g-e).

By thus configuring the θ, d calculator 45a as described above, Cx and Cy can be automatically calculated even when different ultrasonic images are input, and the operator can The labor of processing can be reduced. Also, since the processing amount does not require non-linear calculation, Cx and Cy can be obtained very quickly.

Next, FIG. 13 shows the structure of an image classification apparatus as a third embodiment according to the present invention and will be described. here,
The same members as those of the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted. In this example,
The pre-processing unit 46 of the feature extraction unit 4 is different in configuration from the first embodiment.

In this embodiment, a difference statistic, a run length matrix, and a co-occurrence matrix are used as the feature quantity instead of the power spectrum. The difference statistic is the probability Pδ (k), (k = 0, 1, ..., N−1) that the density difference between two points separated by a constant displacement δ = (r, θ) in the image of interest is k. Seeking
Then four types of features are calculated and the ultrasound image is characterized by those features. It can be satisfactorily extracted in comparison with a fine power spectrum of an image such as a texture.

Next, the run-length matrix has a frequency Pθ (i, j), where j points of density i in the angle θ direction in the image continue.
A run-length matrix having (i = 1, 2, ..., N-1, j = 1, 2, ..., L) as elements is calculated, and five types of feature quantities are calculated from the matrix, and the ultrasonic image is calculated using them. Characterize. Speckle patterns unique to ultrasonic images can be expressed well. In the co-occurrence matrix, the density difference between two points separated by a constant displacement δ = (r, θ) in the image of interest is i,
Probability of being P j (i, j), (i, j = 0, 1, ...,
n-1) is calculated, and then 14 kinds of features are calculated, and the ultrasonic image is characterized by them. For details, the difference statistic is described in P521 of the “Image Analysis Handbook” of the University of Tokyo Press, the run-length matrix is described in P522, and the co-occurrence matrix is described in P518, and the description thereof is omitted here.

FIG. 13 shows the configuration of the preprocessing section 46. The preprocessing unit 46 includes a difference statistic detector 75 and a difference statistic S (x, y) obtained by the difference statistic detector 75.
From the run length matrix detector 77a, the run length matrix detector 77, and the run length matrix R (x, y) obtained by the run length matrix detector 77. And a feature amount calculation unit 76b.

By using the difference statistic and the run length matrix as described above, the texture pattern and the speckle pattern of the image can be detected more effectively as compared with the power spectrum of the first embodiment, and the classification accuracy is greatly improved. it can.

FIG. 14 shows another structural example. The pre-processing unit 46 uses a co-occurrence matrix instead of the run-length matrix described above. The pre-processing unit 46 calculates a difference statistic detector 75, a feature amount calculator 76a, a co-occurrence matrix detector 78, and a feature amount for calculating 14 types of feature amounts obtained by the co-occurrence matrix detector 78. It is composed of a calculator 76c. Since the co-occurrence matrix extracts various features that characterize the texture, the co-occurrence matrix can be used together with the difference statistic to further improve the classification accuracy.

It should be noted that only one of the above-mentioned three types of feature quantities (coincidence matrix, difference statistic, run-length matrix) may be selected for processing, or two or more feature quantities may be combined. It goes without saying that it may be performed in combination with the power spectrum described above.

A fourth embodiment, which is a modification of the first embodiment described above, will now be described with reference to FIGS.
Here, the same members as those in the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted.

When making a diagnosis, the doctor first finds a suspicious area, and then pays attention to the suspicious area and closely observes it. Therefore, the present embodiment proposes a method of processing a predetermined predetermined attention area with higher accuracy.

FIG. 15A shows a region of interest in the ultrasonic image for which classification determination is desired. It corresponds to the area of interest that the doctor first suspected. In this embodiment, the accuracy is improved by repeatedly processing the attention area multiple times.

This will be specifically described with reference to FIGS. 16 and 17. The range to be classified shown in FIG. 15A is input from the range designation unit 41. This range corresponds to the attention area described above. Next, the coordinate designation unit 79 sequentially designates a processing region for classification determination from this attention region. This designation is performed randomly, for example. In this embodiment, areas P1 to P8 are designated areas as shown in FIG. Then, the classification of the processing area P1 is determined, and then P
2, P3, ..., P8 are determined, and a final determination is made based on the determination result of each processing area. The processing outline of the feature amount calculation unit of the present embodiment is the same as that of the feature amount calculation unit of the first embodiment, and the description thereof is omitted.

The logic judgment unit 80 of the classification judgment unit 5 is composed of binarizers 81a to 81c, cumulative adders 82a to 82c, a maximum value detector 83, and a final judgment unit 84. The output of the neural network 56 is the binarizer 8
Binarization is performed in 1a to 81c, for example, the output is 0 or 1.
Is converted to When the portion to be subjected to the classification determination is the normal portion, the output of the binarizer 81a becomes "1" and the output of the binarizer 81b becomes "0", and the cumulative adders 82a to 82a.
It is configured to be added by c.

Then, classification determination processing is performed on the sequentially designated areas, and cumulative additions 82a to 82c are cumulatively added according to the output of the neural network.
At the end of processing of all areas P1 to P8, the maximum value detection unit 83 calculates the maximum value from the outputs of the cumulative adders 82a to 82c. Based on this maximum value, the final determination unit 84
In, normal, abnormal part benign, abnormal part malignant, etc. are classified.

As described above, by using the maximum value of the results of repeated classification determination for the attention area,
It is more robust against noise and can be more reliable.
In this embodiment, eight processing areas are designated from the attention area, but it is natural that more processing areas may be designated. Further, in the present embodiment, a plurality of processing regions are specified from the attention region, but the maximum value of the results of processing with a plurality of different feature amounts of combinations such as a histogram, co-occurrence matrix, and fractal dimension in the preprocessing unit is set. You may use it.

Next, FIGS. 18, 20, and 21 show the structure of an image classification apparatus as a fifth embodiment according to the present invention, and will be described. Since this image classifier can buffer the classification data as needed during the execution mode in the image classification apparatus shown in the first embodiment, it is possible to add buffered data and re-learn. Characterize.

FIG. 18 shows a basic processing flow. In the execution mode, the operator can relearn as needed. 20 and 2
The same members as those shown in FIGS. 4 and 5 among the members shown in FIG. 1 are designated by the same reference numerals and the description thereof will be omitted.

The operator uses the range designating section 41 to specify the range in which classification determination is desired. Next, in this range, a predetermined feature amount is calculated by the preprocessing unit 46 of the feature extraction unit 4. The calculated feature amount P (u) is used as the multivariate analysis unit 4
7 and the additional teacher data buffer 85. Since the processing contents after the multivariate analysis unit 47 are the same as the contents shown in FIG. 4, description thereof will be omitted.

The feature amount P (u) sent to the additional teacher data buffer 85 is managed by the buffer controller 86. Next, the operator makes another range of classification determination.
By repeating this operation, a large number of feature quantities are buffered in the additional teacher data buffer 85. When the operator updates the classification vector and the teacher data, the processing signal is sent from the buffer controller 86 to the learning controller 87, and the feature amount filed in the additional teacher data buffer 85 is sent to the teacher data input unit 88. Then, the classification vector and the teacher data are updated. The updated classification vector and teacher data are stored in the classification vector memory 52 or the weight memory 57 according to an operator's instruction, and are used for subsequent classification determination. The learning control unit 87 includes a teacher data input unit 88, a learning controller 89, a classification vector calculation unit 90, a learning device 91, and a learning data memory 92.

As described above, it is possible to improve the classification accuracy by being able to re-learn during the execution mode. Also,
If the number of samples of the classification vector that has been learned in advance is small, a new classification vector can be created by this function and learning can be redone. Furthermore, the data that has been subjected to the classification determination in the execution mode can be effectively used.

Next, an image classification apparatus as a sixth embodiment according to the present invention will be described. In this example, 4
The classification of two abnormal parts (lesion parts α, β, γ, δ) is performed. In the above-described first embodiment, the normal part and the two abnormal parts (benign and malignant) are classified. In this case, normal part and abnormal part (malignant, benign) are classified as classification number 1, abnormal part (malignant) and abnormal part (benign) are classified as classification number 2, and when the result of classification number 1 is abnormal Only the classification number 2 was executed.

On the other hand, in this embodiment, the normal part and the normal part are
All abnormalities (lesions α, β. Γ, δ) are selected from 2 types of targets, and all combinations are selected by feedback, and the results are comprehensively judged and finally classified. It is characterized by judging. As shown in FIG. 19A, the classification process of the normal part and the abnormal part α is set to the classification number 1. The classification process of the normal part and the abnormal part β is classified as classification number 2, and similarly, the classification of the abnormal part γ and the abnormal part δ is defined as classification number 10. Classification numbers 1 to 10 are executed by feedback, and the final judgment is made from the result.

FIG. 19B shows 2 out of 5 types of targets.
One of them is selected and the result of classification is shown. As for the number of classifications, the classification target is a normal part and four abnormal parts (lesion parts α, β.
γ, δ) represents the number of times of classification, and in this example, the number of normal parts is four, which is the largest, so the classification target is determined to be a normal part. In this embodiment, since the classification vectors of two classes are used, the classification performance is very high, and further feedback is performed, so that the circuit scale can be small.

Further, in the present embodiment, a total of five classes, that is, a normal part and four abnormal parts, are classified and judged, but it is natural that it is possible to correspond to five or more classes. Although the description has been given based on the above embodiment, the present invention also includes the following inventions.

(1) Information input means for inputting information to be classified, and characteristic extraction means for extracting characteristic information by dividing the effective area of the information input by the information input means and the other area. An image classification device is provided, which is configured to include at least a classification determination unit that determines the type of information based on the characteristic information extracted by the characteristic extraction unit and classifies the information for each type.

As a result, since the effective area can be automatically extracted, the processing of the operator can be reduced and the processing efficiency can be improved. (2) Using at least one of a power spectrum, a difference statistic, a co-occurrence matrix, and a run-length matrix from the information input means for inputting information to be classified and the information input by the information input means. An image classification device including a feature extraction unit that extracts feature information, and a classification determination unit that determines at least the type of information based on the feature information extracted by the feature extraction unit and classifies the information for each type. provide.

As a result, fine features of the image can be extracted, so that the accuracy of the classification determination can be improved. (3) Information input means for inputting information to be classified, feature extraction means for extracting feature information from the information input by the information input means, and specific area using the feature information extracted by the feature extraction means There is provided an image classification device configured by a classification determination unit that classifies a plurality of times and finally classifies based on the classification result.

This makes it robust against noise, so that the reliability of classification determination can be improved. (4) Information input means for inputting information to be classified, feature extraction means for extracting feature information from the information input by the information input means, and at least information based on the feature information extracted by the feature extraction means. It is configured by a classification determining unit that determines a type and classifies each type, and a learning unit that reconstructs the feature extracting unit and / or the classification determining unit using the classification result of the classification determining unit. An image classification device is provided.

As a result, the accuracy of the classification determination can be improved, and the data for which the classification determination has been performed can be effectively used. (5) Information input means for inputting information to be classified, feature extraction means for extracting feature information from the information input by the information input means, and information based on at least the feature information extracted by the feature extraction means. There is provided an image classification device configured by a classification determination unit that classifies each type and repeats feedback of a plurality of types. As a result, the classification performance can be improved, and the circuit scale can be reduced because feedback is performed.

[0105]

As described above in detail, according to the present invention, it is possible to accurately classify even a portion of an input image that is difficult to extract only by binarization or three-dimensional display, and give a visually easy-to-understand instruction. It is possible to provide an image classification device capable of performing the above.

[Brief description of drawings]

FIG. 1 (a) is a diagram showing a configuration at the time of performing classification in an image classification device according to the present invention, and FIG. 1 (b) is a diagram showing a configuration of classification learning performed prior to performing classification. is there.

FIG. 2 is a diagram showing a configuration of a first half portion of a convex type ultrasonic device used in the image input device of the image classification device according to the first embodiment of the present invention.

FIG. 3 is a diagram showing the configuration of the latter half of the convex type ultrasonic device used in the image input device of the image classification device according to the first embodiment of the present invention.

FIG. 4 is a diagram showing a configuration example (at the time of performing classification determination) of an image classification device of a first embodiment according to the present invention, in which the convex type ultrasonic wave device shown in FIG. 1 is incorporated.

5 is a diagram showing a configuration during classification determination learning (learning mode) in the image classification device shown in FIG.

FIG. 6 is a diagram for explaining correction of an ultrasonic waveform.

FIG. 7 is a diagram showing a form of an ultrasonic image of a convex type ultrasonic device.

FIG. 8 is a diagram showing a shape of an extraction region of an ultrasonic image.

FIG. 9 is a diagram showing a configuration example of a projection calculator.

FIG. 10 is a diagram showing a configuration example of a power spectrum detector for detecting a power spectrum P (u) and a classification method.

FIG. 11 is a diagram showing a configuration example of a θ, d calculator in an image classification device as a second embodiment.

FIG. 12 is a diagram showing a state in which a contour line extraction for performing region division into a diagnostic organ and other portions is performed by adaptive binarization processing.

FIG. 13 is a diagram illustrating a configuration example of a preprocessing unit of a feature extraction unit in an image classification device as a third embodiment.

FIG. 14 is a diagram showing a configuration example in which a preprocessing unit of the feature extraction unit in the third embodiment is modified.

FIG. 15A is a diagram showing a region of interest in an ultrasonic image for which classification determination is to be performed in an image classification apparatus according to a fourth embodiment, and FIG. 15B is a diagram showing a specified region for which classification determination is desired. FIG.

FIG. 16 is a diagram showing a configuration of one of two divided image classification devices as a fourth embodiment.

FIG. 17 is a diagram showing the other two-divided configuration of an image classification device as a fourth embodiment.

FIG. 18 is a flowchart for explaining switching between the execution mode and the learning mode in the fifth embodiment.

FIG. 19 is a diagram for explaining classification numbers in the sixth embodiment, and FIG. 19B is a diagram showing the results of classification by selecting two out of five types of targets.

FIG. 20 is a diagram showing a configuration of one of two divided image classification devices as a sixth embodiment.

FIG. 21 is a diagram showing the other half of the configuration of the image classification device as the sixth embodiment.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Image input device, 2 ... Filing device, 3 ... Normalization part, 4 ... Feature extraction part, 5 ... Classification determination part, 6 ... Display part, 7 ... Learning control part, 8 ... Preprocessing part, 9 ... Multivariate analysis Department.

Claims (5)

[Claims] 1. Information input means for inputting information to be classified, feature extraction means for dividing the effective area of the information input by the information input means and other areas, and extracting characteristic information. An image classification apparatus, comprising: at least a classification determination unit that determines the type of information based on the characteristic information extracted by the characteristic extraction unit and classifies the information for each type. 2. Information input means for inputting information to be classified, and at least one of a power spectrum, a difference statistic, a co-occurrence matrix, and a run length matrix is used from the information input by the information input means. A feature extraction means for extracting feature information, and a classification determination means for determining the type of information based on at least the feature information extracted by the feature extraction means and classifying the information for each type. Characteristic image classification device. 3. An information input unit for inputting information to be classified, a characteristic extraction unit for extracting characteristic information from the information input by the information input unit, and characteristic information extracted by the characteristic extraction unit. An image classification apparatus comprising: a classification determination unit that classifies a specific region a plurality of times and finally classifies the classification based on the classification result. 4. Information input means for inputting information to be classified, feature extraction means for extracting feature information from the information input by the information input means, and at least based on the feature information extracted by the feature extraction means. A classification determining unit that determines the type of information and classifies the information into different types; and a learning unit that reconstructs the feature extracting unit and / or the classification determining unit by using the classification result of the classification determining unit. An image classification device comprising: 5. An information input unit for inputting information to be classified, a characteristic extraction unit for extracting characteristic information from the information input by the information input unit, and at least based on the characteristic information extracted by the characteristic extraction unit. An image classification apparatus comprising: a classification determination unit that determines a type of information and repeats feedback of a plurality of types to classify the information for each type.