JPH1131214A - Picture processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent the generation of blur or an after-image due to movement in a processed result, and to reduce noise at the time of especially using a moving image as an object with respect to a picture processor for obtaining picture information by irradiating or transmitting an information carrier such as an electromagnetic wave or an ultrasonic wave to an object, and measuring the change of this information carrier. SOLUTION: At the time of using a moving image represented by an ultrasonic fault image or the like as an object to be processed, the time change components of picture data are calculated from time sequential data in a prescribed area including at least picture elements under consideration inputted from an original picture supplying part 1 are calculated by a time change component calculating part 7, and the calculated result and the time sequential data in the prescribed area including the picture elements under consideration from the original picture supplying part 1 are outputted to a picture conversion processing part 2, and the movement of an objective site is learned by the learning operation of the picture conversion processing part 2 constituted of a neural network. Thus, it is possible to prevent the generation of blur or an after-image in the processed result in the picture conversion processing part 2, and to reduce noise.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for irradiating or penetrating an object with an information carrier such as an electromagnetic wave or an ultrasonic wave including light or X-rays, and measuring a change in the information carrier to obtain image information. For image processing devices such as medical image diagnostic devices or broadcast video processing devices or industrial image processing devices or consumer image capturing and displaying devices, blurs and residual images due to motion are particularly likely to occur when processing moving images. The present invention relates to an image processing device capable of preventing occurrence of noise and reducing noise.

[0002]

2. Description of the Related Art A conventional image processing apparatus of this type is shown in FIG.
As shown in FIG. 1, an original image supply unit 1 for outputting image data to be processed and an artificial neural element are connected to form an input layer, an intermediate layer, and an output layer so as to form a layered structure, thereby forming a network. An image conversion processing unit 2 composed of a neural network for inputting time-series data of a predetermined area including at least the pixel of interest from the supply unit 1 and performing image conversion processing and outputting data corresponding to the pixel of interest; And a display unit 3 for inputting and displaying the image information output after the image conversion processing by the conversion processing unit 2. Then, for example, time-series data of an X-ray image is output from the original image supply unit 1, and time-series data of the X-ray image output from the original image supply unit 1 is input by the image conversion processing unit 2, and the X-ray image is input. Perform image conversion processing to recover blur of line image,
X output from the image conversion processing unit 2 by the display unit 3
An X-ray image whose blur has been restored is displayed by inputting a line image.

Here, the input image data in the image conversion processing unit 2 shown in FIG. 11 is, for example, as shown in FIG.
A target image 5 of a predetermined area including at least the pixel 4 to be processed
Is a plurality of time series (T-4, T-3, T-2, T-1,
T) consisted of the time-series data 6 arranged in line.
Then, each target image 5 of the time-series data 6 was sequentially input to the image conversion processing unit 2 as time passed.

[0004]

However, in such a conventional image processing apparatus, the input image data in the image conversion processing unit 2 is the target image 5 of the time-series data 6.
Were only entered sequentially over time,
When a moving image typified by an ultrasonic tomographic image or the like is to be processed, there is little information on the movement of the target portion provided from the input time-series data 6, and the image conversion processing configured by the neural network is performed. In the learning operation of the part 2, the movement of the target part could not be sufficiently learned. Therefore, the processing result in the image conversion processing unit 2 may cause a motion blur or an afterimage. For this reason, noise cannot be reduced sufficiently, and the image quality of the obtained image may deteriorate.

Accordingly, the present invention has been made to address the above-described problems, and has been disclosed in an image processing apparatus capable of preventing a blur or an afterimage due to motion from occurring in a processing result when a moving image is to be processed, and reducing noise. It is intended to provide a device.

[0006]

In order to achieve the above object, an image processing apparatus according to the present invention comprises an original image supply unit for outputting image data to be processed, and an image data input from the original image supply unit. Then, a time-varying component calculating unit that calculates a time-varying component of the image data, and the image data from the original image supply unit and the time-varying component calculated by the time-varying component calculating unit are input to perform image conversion processing. It has an image conversion processing section and a display section for inputting and displaying the image information output by the image conversion processing by the image conversion processing section.

The time-varying component calculating unit calculates a time-varying component of the image data from the time-series data of a predetermined area including at least the pixel of interest input from the original image supplying unit, and compares the calculation result with the original image. This is output to the image conversion processing unit together with the time-series data of the predetermined area including the target pixel from the supply unit.

Further, the image conversion processing unit forms a network by connecting artificial neural elements so as to form a layer structure of an input layer, an intermediate layer, and an output layer, and at least focuses a pixel of interest from the original image supply unit. The neural network is configured to input time series data of a predetermined area including the input data, input a calculation result from the time change component calculation unit, perform image conversion processing, and output data corresponding to the pixel of interest.

Furthermore, a teacher image supply unit for inputting a teacher signal for causing the neural network to perform a learning operation may be connected to the image conversion processing unit comprising the neural network.

[0010]

Embodiments of the present invention will be described below in detail with reference to the accompanying drawings. FIG. 1 is a block diagram showing an embodiment of an image processing apparatus according to the present invention. This image processing apparatus irradiates or penetrates an object with an information carrier such as electromagnetic waves or ultrasonic waves including light or X-rays,
This is a device for obtaining image information by measuring the change of the information carrier, such as various medical image diagnostic devices, broadcast video processing devices, industrial image processing devices, or consumer image capturing and displaying devices, as shown in FIG. Thus, the original image supply unit 1
It comprises a time-varying component calculation unit 7, an image conversion processing unit 2, and a display unit 3.

The original image supply unit 1 outputs image data to be processed. For example, the original image supply unit 1 outputs medical data such as an X-ray photographing apparatus, an ultrasonic tomography apparatus, an X-ray CT apparatus, a nuclear medicine imaging apparatus, or a magnetic resonance imaging apparatus. A measuring unit that measures or captures an object in an image diagnostic device, a broadcast video processing device, an industrial image processing device, a consumer image capturing and displaying device, or the like, or stores image data measured for the object and reads out the data as necessary. It is a storage unit or the like. The time-varying component calculation unit 7 is added in the present invention, and inputs the image data from the original image supply unit 1 and calculates a time-varying component of the image data. The image conversion processing unit 2 performs image conversion processing by inputting the image data from the original image supply unit 1 and the time change component calculated by the time change component calculation unit 7, and includes, for example, a filter unit. . Further, the display unit 3 is provided with the image conversion processing unit 2.
And displays the image information that has been subjected to the image conversion processing and is, for example, a television monitor.

Here, in the present invention, the time change component calculation section 7 calculates a time change component of the image data from the time series data of a predetermined area including at least the target pixel input from the original image supply section 1. The calculation result is output to the image conversion processing unit 2 together with the time-series data of the predetermined area including the target pixel from the original image supply unit 1. As shown in FIG. 2, the time-varying component calculation unit 7 generates a plurality of target images 5 in a predetermined area including the processing target pixels 4 in a time series (T-4, T-
3, (T-2, T-1, T), the image data composed of the time-series data 6 is input, and a time-varying component of the image data is calculated. That is, the processing target pixel 4 existing in the target image 5 at the time T in FIG.
In the past (T-1, T-2, T-3, T-4) frame images, the pixels 4a, 4b, 4c, and 4d existing at the same position as the pixel 4 to be processed are: The operation is performed so as to obtain the time-varying components 8a, 8b, 8c, 8d of the time-series data 6 by calculating the differences between the density values.

In the present invention, the image conversion processing unit 2
A network is formed by connecting artificial neural elements so as to form a layer structure of an input layer, an intermediate layer, and an output layer, and time-series data of a predetermined area including at least a pixel of interest is input from the original image supply unit 1 and It is configured by a neural network that inputs a calculation result from the time change component calculation unit 7, performs image conversion processing, and outputs data corresponding to the pixel of interest.

That is, the image conversion processing unit 2
The neural network 9 shown in FIG. The neural network 9 inputs the time-series data of a predetermined area including at least the pixel of interest from the original image supply unit 1 and the calculation result from the time change component calculation unit 7 to perform image conversion processing. It outputs data corresponding to pixels. The artificial neural element 10 is connected to form a layer structure of an input layer, an intermediate layer, and an output layer to form a network. Is configured using a linear function.

FIG. 4 is an explanatory diagram showing an artificial neural element 10 constituting the neural network 9. As shown in the figure, an artificial neural element (hereinafter referred to as a “neuron model”)
) 10 that is an element of the multi-input first output that simulates the function of biological neural elements (neurons), the output Oj is determined by the product sum of the input Ii (I ₁ -In) and the coupling coefficient Wji (Wj ₁ ~Wjn) Is done. That is, it is determined as follows using the input / output function f (x). Here, θj is the offset of the input / output function corresponding to the threshold, and n is the number of inputs.

FIG. 3 is an explanatory diagram showing a hierarchical structure of the neural network 9 which is connected to form a layer structure and is formed into a network. As shown in the figure, the neural network 9 uses a large number of the neuron models 10 and forms a network by connecting the input layer, the intermediate layer, and the output layer so as to form a layer structure.
It is configured to realize the functions of signal processing and information processing. In FIG. 3, reference numeral 11 denotes a branch connecting the input layer and the intermediate layer.
Are respectively connected to all the neuron models 10 in the hidden layer. Reference numeral 12 denotes a branch connecting the intermediate layer and the output layer, and each neuron model 10 in the intermediate layer is connected to every neuron model 10 in the output layer. The neural network 9 converts the input information 13 supplied to the input layer and outputs the converted information as output information 14 from the output layer.

With this configuration, the neural network 9 is an information processing mechanism having a learning function and a self-organizing function. Then, the neural network 9 performs self-organization by giving a teacher signal 15 and performing learning so that desired output information 14 is output from the output layer with respect to the input information 13 supplied to the input layer. . Here, in the input layer, an identity function as shown in the following equation is used as an input / output function. fi (x) = x As a result, the input is output as it is. The input information 13 may be modulated using another function instead of the identity function.

FIG. 5 is a graph showing a sigmoid function used as an input / output function of the intermediate layer of the neural network 9. In the intermediate layer, the output fs is changed from “0” to “1” as an input / output function as shown in the figure.
, A monotonically increasing sigmoid function is used. This sigmoid function is expressed by the following equation. Here, U ₀ is a parameter for controlling the inclination. And differentiating this gives: It has the characteristic that it can be expressed by the original sigmoid function. As described above, by using a nonlinear function such as a sigmoid function for the intermediate layer, the neural network 9 can handle a complicated nonlinear conversion process.

FIG. 6 is a graph showing a linear function used as an input / output function of the output layer of the neural network 9. In the output layer, a linear function whose output increases and decreases linearly with respect to the input as shown in the figure is used as the input / output function instead of the sigmoid function. Now, assuming that the inclination of the straight line is a, for example, it is represented by the following equation. Differentiating this gives: Becomes As a result, the output from the output layer becomes an analog value, and an analog value such as image information can be handled.

In the neural network 9, learning is performed on the input information 13 so that desired output information 14 is obtained. That is, the coupling coefficient of the branch 11 connecting the input layer and the intermediate layer and the coupling coefficient of the branch 12 connecting the intermediate layer and the output layer are changed so that the error between the teacher signal 15 serving as a teacher and the output information 14 is reduced. After learning, self-organization is performed so that a desired output is obtained. FIG. 7 shows a learning algorithm of the neural network 9 for analog output.
Shown in The meanings of the symbols in FIG. 7 are as follows. I: input information Ok; output Wk of the k-th layer neuron model Wk; coupling coefficient θk between the k-th layer and the (k + 1) th layer; offset T of the k-th layer neuron model T; teacher signal ηw; learning constant for W ηθ; learning constant for .theta.k; error amount fi for correcting the (k-1) th layer fi; identity function fs; sigmoid function fl; linear function Note that input information I and output O of the neuron model on the kth layer
k, the offset θk of the neuron model of the k-th layer, the teacher signal T, and the error amount δk for correcting the (k−1) -th layer are represented by a one-dimensional vector, and the k-th layer and the (k + 1) -th layer Is represented by a two-dimensional vector. Also,
Since the neural network shown in the example has three layers, the first layer is an input layer, the second layer is an intermediate layer, and the third layer is an output layer.

The learning algorithm will be described below. Mean square error E between the teacher signal T and the output O _{3 of the} output layer
Is It is. In order to reduce this, the coupling coefficient Wk between the k-th layer and the (k + 1) -th layer is changed by the following correction amount. Modify by

Here, In other words, δ ₃ that determines the amount of correction between the hidden layer and the output layer
Is It is. Therefore, the amount of correction is Becomes

Thus, as described above, the output from the output layer has an analog value. Δ ₂ that determines the amount of correction between the input layer and the output layer is And the correction amount is Becomes The algorithm shown in FIG. 7 is obtained by similarly calculating the offset.

The learning of the neural network 9 is performed by such an algorithm based on the teacher signal 15 given by the operator. Here, an example of the three-layer neural network 9 has been described. However, a neural network 9 having three or more layers can be configured by increasing the number of intermediate layers. In this case, the learning algorithm reverses the error amount δ. Propagation and the amount of correction may be asymptotically determined, and it is possible to deal with image information including more complicated conversion. The image data converted by the neural network 9 is converted to an appropriate density scale (not shown), and is input to the display unit 3 shown in FIG. 1 and displayed.

FIG. 8 shows the quasi-linear region 16 of the sigmoid function. This is because the neural network 9 shown in FIG. 3 uses a function other than the linear function, for example, a sigmoid function shown in FIG. 8 as an input / output function of an output layer at the last stage of the network connected in a layered structure.
The case where the quasi-linear region 16 is used limited to the range of the output image information is shown. Also in this case, the image information can be output as an analog value as described above.

FIG. 9 is a block diagram showing another embodiment of the present invention. In this embodiment, a teacher image supply unit 17 for inputting a teacher signal 15 (see FIG. 3) for causing the neural network 9 to perform a learning operation with respect to the neural network 9 as the image conversion processing unit 2 shown in FIG. Connected. In the embodiment shown in FIG. 1, an average, a standard, or a specific teacher signal 15 is given at the time of factory shipment or installation at a use facility to perform necessary learning in advance. In some cases, the situation does not match the usage situation of each user at the site, and therefore, an attempt is made to improve the situation. That is,
9, a teacher image supply unit 17 is a measuring device for measuring or photographing an object or a storage device for storing image data measured for an object and reading out the image data as necessary. 9 is connected to an output terminal from which output information 14 is output from an output layer 9 (see FIG. 3), and individual users and the like are connected to image data output from the original image supply unit 1 by using the teacher image supply unit 17. On the other hand, an ideal teacher image is supplied as a teacher signal 15 as needed, and the neural network 9 learns.

Next, an operation in which a teacher image is supplied from the teacher image supply unit 17 as the teacher signal 15 and the neural network 9 learns will be described. According to the present invention,
When removing the inherent noise and habit of the device system, the original image supply unit 1 supplies the image data including the noise and habit obtained by the target device system to the neural network 9 as the image conversion processing unit 2. As well as
Image data free from noise and habits obtained by the ideal apparatus system from the teacher image supply unit 17 is supplied to the neural network 9, and learning is performed by the algorithm shown in FIG. In the neural network 9 after the learning, a conversion model from a target device system including noise and habit to an ideal device system including no noise and habit is automatically constructed. The habit can be removed.

As an example, when an ultrasonic diagnostic apparatus is used as a target system, the image data from the original image supply unit 1 and the time change component calculated by the time change component calculation unit 7 are sent to the image conversion processing unit 2. The process of inputting and processing the target image for the purpose of improving the image quality of the ultrasonic tomographic image will be described with reference to FIG. First, the density value of the processing target pixel 4 at the time T of the time-series data 6 of the ultrasonic tomographic image supplied from the original image supply unit 1 and the time one frame past by the time change component calculation unit 7 shown in FIG. The difference with the density value of the pixel 4a located at the same position at T-1 is calculated, and the time change component 8a is calculated. At this time, the pixels at the same position at the past times T-2, T-3, and T-4 are referred to as 4b, 4c, and 4d, respectively.

In FIG. 10, in each frame from time T to time T-4, each target pixel 4, 4
Kernel (local area) 1 for a, 4b, 4c, 4d
8 is kept spatially symmetric, and the kernel 18 is made smaller as going back to the past frame, and the input pixel 19 is determined by these. FIG.
In the example of (9), at the time T, 9 frames are sequentially arranged from the frame where the pixel 4 to be processed exists to the past frame.
× 5 = 45 pixels, 7 × 3 = 21 pixels, 5 × 1 = 5 pixels,
A total of 75 input pixels 19 of 3 × 1 = 3 pixels and 1 × 1 = 1 pixel are selected. The input pixel 19 and the time-varying component 8a calculated by the time-varying component calculating unit 7 calculate the density value of a total of 76 pixels as the input pixel density value normalizing unit 20.
In, for example, the values are normalized to values of −1 to 1, respectively.

Thereafter, the input pixel density value normalizing section 20
The input information 13 of the input layer of the neural network 9 (see FIG. 3) as the image conversion processing unit 2
The information is sequentially propagated from the intermediate layer to the output layer, and the output information 14 of the output layer is obtained. The output information 14 is inversely normalized to the pixel density value in the output pixel density value normalizing section 21 in the reverse manner to the normalization performed by the input pixel density value normalizing section 20, and the output pixel density value 22 is obtained. obtain. This output pixel density value 22 is the image conversion processing result of the processing target pixel 4. Then, the output image 23 is obtained by performing the above processing on the entire target image 5.

Next, a learning operation for enabling the neural network 9 to obtain desired output information 14 for the pixel 4 to be processed will be described. First, FIG.
At 0, a teacher image 24 is created. At this time, if the purpose is to improve the image quality of the ultrasonic tomographic image, the teacher image 2
4 is a high-quality ultrasonic tomographic image. A high-quality ultrasonic tomographic image is created by selecting a plurality of images from a time-series original ultrasonic tomographic image and averaging them. In addition, in the case of an ultrasonic tomographic image having a moving part, an image synchronized with the movement is selected from the time-series image in order to prevent the moving part from being blurred by the addition, and the averaging is performed. Create an image. The method of obtaining these high-quality images is an example, and other filter processing or the like is used, or a plurality of filter processings different for each region in the image so as to be an image desired by the operator, for example, a high-frequency emphasis filter, etc. May be created.

The learning operation performed using the teacher image 24 as described above is executed as follows. First, a value 27 obtained by normalizing the target pixel 26 in the attention area 25 of the created teacher image 24 to −1 to 1 by the output pixel density value normalizing unit 21 and the output information 14 from the neural network 9 are described. An error is calculated, and the coupling coefficient of the neural network 9 is changed so that the error is reduced. This operation is repeated in the attention area 25,
Learning is performed until the repetition reaches a specified number of times or the error becomes equal to or less than an allowable error. By this learning, when the original image is input, the neural network 9 is automatically constructed so that the processed image desired by the operator is output, and the neural network 9 acquires a desired image filter input / output characteristic.

[0033]

The present invention has been configured as described above.
When a moving image typified by an ultrasonic tomographic image or the like is to be processed, a time change component of the image data is obtained from the time series data of a predetermined area including at least the pixel of interest input from the original image supply unit by the time change component calculation unit. The component is calculated, and the calculation result is output to the image conversion processing unit together with the time-series data of the predetermined area including the pixel of interest from the original image supply unit, and is used as a target in the learning operation of the image conversion processing unit configured by the neural network. You can learn the movement of the part. Therefore, it is possible to prevent the processing result in the image conversion processing unit from being blurred due to motion or from having an afterimage. Thus, when a moving image is targeted, it is possible to prevent the processing result from being blurred or an afterimage due to motion, to reduce noise, and to improve the image quality of the obtained image.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment of an image processing apparatus according to the present invention.

FIG. 2 is an explanatory diagram illustrating a calculation state of a time-varying component of image data by a time-varying component calculating unit according to the present invention.

FIG. 3 is an explanatory diagram showing a hierarchical structure of a neural network which is connected to form a layer structure and is formed into a network.

FIG. 4 is an explanatory diagram showing an artificial neural element constituting a neural network.

FIG. 5 is a graph showing a sigmoid function used as an input / output function of an intermediate layer of the neural network.

FIG. 6 is a graph showing a linear function used as an input / output function of an output layer of the neural network.

FIG. 7 is an explanatory diagram showing a learning algorithm of a neural network for analog output.

FIG. 8 is a graph showing a quasi-linear region of a sigmoid function.

FIG. 9 is a block diagram showing another embodiment of the present invention.

FIG. 10 shows an example in which image data from an original image supply unit and a time-varying component calculated by a time-varying component calculating unit are input to an image conversion processing unit. It is explanatory drawing shown about the process which performs a process.

FIG. 11 is a block diagram showing a conventional image processing apparatus of this type.

FIG. 12 is an explanatory diagram showing input information in the conventional image processing apparatus.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Original image supply part 2 ... Image conversion processing part 3 ... Display part 4 ... Processing target pixel 5 ... Target image 6 ... Time series data 7 ... Time change component calculation part 8a-8d ... Time change component 9 ... Neural network 10 ... Artificial neural elements 11, 12 Branch 13 Input information 14 Output information 15 Teacher signal 16 Semi-linear region of sigmoid function 17 Teacher image supply unit

Continued on the front page (51) Int.Cl. ⁶ Identification code FI // A61B 5/055 A61B 5/05 380

Claims (4)

[Claims] An original image supply unit for outputting image data to be processed; a time change component calculation unit for inputting image data from the original image supply unit and calculating a time change component of the image data; An image conversion processing unit that inputs image data from the original image supply unit and the time change component calculated by the time change component calculation unit to perform an image conversion process, and an image that is subjected to image conversion processing and output by the image conversion processing unit A display unit for inputting and displaying information. 2. The time-varying component calculating unit calculates a time-varying component of the image data from time-series data of a predetermined area including at least a pixel of interest input from the original image supplying unit, and calculates a result of the calculation based on the original image. 2. The image processing apparatus according to claim 1, wherein the image processing apparatus outputs the image data to the image conversion processing unit together with the time-series data of the predetermined area including the target pixel from the supply unit. 3. The image conversion processing unit configures a network by connecting artificial neural elements to form a layer structure of an input layer, an intermediate layer, and an output layer, and forms at least a target pixel from the original image supply unit. The neural network is configured to input time-series data of a predetermined area including the input data, input a calculation result from a time-varying component calculation unit, perform image conversion processing, and output data corresponding to the pixel of interest. The image processing device according to claim 1. 4. The image conversion processing section comprising the neural network is connected to a teacher image supply section for inputting a teacher signal for causing the neural network to perform a learning operation. The image processing device according to claim 1.