JP2859377B2 - Image processing method and image processing apparatus using neural network - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method suitable for restoration processing from a binary image to a multivalued image or image area separation processing, and more particularly, to an image processing method using a neural network. The present invention relates to an image processing method using a converted table. The present invention also relates to an image processing device incorporating the conversion table.

[Prior art] The use of a table for image processing requires high speed,
It is generally better because of the small size of the device.

In image processing for restoring a multi-value halftone from a binary image, for example, a conversion table equivalent to applying a smoothing filter to a binary image is conventionally created and used in place of the smoothing filter. That is being done. Further, in determining a table to be used for image processing as a region separation process from a multi-valued image, a table that performs the function of an edge detection filter has been used. On the other hand, at present, there is no proposal for segmentation from a binary image.

[Problems to be Solved by the Invention] However, in the image processing using a table in these conventional techniques, when a data stored in the table is determined, a filter expected by a human is used, so that an actual output is used. Is not the best one.

Means for Solving the Problems and Action Thereof The present invention has been conceived in order to solve the drawbacks of the related art, and has been developed for restoration processing from a binary image to a multi-valued image and image area separation processing. To provide an image processing method for determining a conversion table for use by using neural nets and an image processing apparatus having the table.

In order to achieve the above object, an image processing method according to the present invention for inputting binary image data to a conversion table and restoring the binary image data into multivalued image data includes preparing binary image data for learning and an ideal output. Inputting the binary image data for learning and the ideal output to a neural network, and learning a coupling coefficient suitable for image restoration between neurons in the network; and Inputting binary image data and obtaining an output thereof, thereby determining a conversion table having the input binary image data as an input and an output of the network as an output; And a step of replacing the network.

Similarly, the image processing method of the present invention for achieving the above-mentioned object and performing image area separation of input image data using a conversion table includes the following steps: input image data for learning and an ideal output representing a corresponding image area separation result. Preparing; and inputting the learning image data and the ideal output to a neural network, and learning a coupling coefficient suitable for image area separation between neurons of the network. A step of inputting the input image data and obtaining an output thereof, thereby determining a conversion table for inputting the input image data and outputting the output of the network, and a step of replacing the neural network with the conversion table. A neural network is used.

The above-described image processing method of the present invention uses the learning of the neural network to determine the configuration of the conversion table. In the learning of neural networks, in order to gain the experience of "learning" by solving various patterns and comparing with the answer (ideal output), the value when the error from the answer converges is determined value. The optimum value is obtained for every pattern. Then, by determining it as a table, a good processing result can be obtained at a high speed.

Thus, the image processing method according to the first aspect is effective for restoration processing from a binary image to a multi-valued image, and the image processing method according to the second aspect is effective for image area separation. One of the learning methods is, for example, a back propagation method.

Further, an image processing apparatus according to the present invention for achieving the above object includes a table created by applying the above method.

Embodiment An embodiment according to the present invention will be described below in detail with reference to the accompanying drawings.

In this embodiment, as an example, the back propagation method of the neural net is used to determine a table for estimating multi-valued image data from binary image data. As a window indicating a reference area when restoring multi-valued image data from binary image data, for example, 3 ×
Adopt a size of 3.

FIG. 1 is a block diagram showing the configuration of an image processing apparatus according to an embodiment in which this 3 × 3 is used as a reference area. In the figure, image data read from a document by an image input device (not shown) is input to a three-line buffer 101. The three-line buffer 101 stores data of three lines continuous in the vertical direction of the image. Of the three lines,
Data from the left end to the right end of the same horizontal position is sequentially output to the delay element (latch) group 102 in synchronization with the clock. Then, data in a 3 × 3 small area is latched in the latch group 102 in the horizontal and vertical directions as shown in FIG. 2A. The latch group 102 includes nine latches,
The output of each latch becomes the address input of the ROM 103. ROM103
Is a table for storing data for binary to multi-value conversion in advance. In the binary-to-multivalue conversion using this table, image data as pixels at the position (shown in FIG. 2B) corresponding to the central pixel of this small block is read from the ROM 103 from the nine binary pixels in FIG. 2A. Is output. In this embodiment, 8 bits (256 gradations) are adopted as the multi-valued image data, for example.

The neural network is used to determine the table data stored in the ROM 103 of this embodiment. That is, learning of the data stored in the ROM 103 is performed in the neural network. The idea of the neural net itself is, for example, "using the neural net for pattern recognition, signal processing, and knowledge processing" (Nikkei Electronics; 1987
August 10).

FIG. 3 is a block diagram showing the configuration of one embodiment of the neural net for this learning. The neural network shown in the figure is generally written so as to correspond to a reference block having a size of M × N. 301 is the input layer i
, 302 indicates an intermediate layer j, and 303 indicates an output layer k. Therefore, when the reference block having the size shown in FIG. 2A is applied to the apparatus of this embodiment, nine neurons are used as the input layer i, nine neurons are used as the intermediate layer, and one neuron is used as the output layer. Neurons are needed. Each neuron includes a plurality of adders, multipliers, and dividers (not shown). Third
In the figure, a solid line connecting each neuron indicates a data input / output direction. For example, the first neuron in the hidden layer is connected to all neurons in the input layer. The coupling strength between the input layer i and the intermediate layer j is represented by _Wji , and the coupling strength between the intermediate layer j and the output layer k is represented by _Wkj .

A procedure for learning table data will be briefly described with reference to FIG. First, multi-valued image data and data obtained by converting the multi-valued image data into binary data by a predetermined binarization process are prepared as learning data. The original multi-valued image data is used as an ideal output as a teacher signal (teach signal).
Used to generate That is, the bond strengths W _ji and W _kj
Is appropriately determined initially, and then the binary image data is input as learning data to a network connected based on the initial values, and output multi-valued image data (kout) is obtained. This output is compared with the original multi-valued image data as the ideal output, and a teacher signal (teach) is generated from the comparison. This process is shown as a process in FIG. This teacher signal, the back propagation method, the W _ji while learning, will modify the W _kj. This process is shown in FIG. By repeating this learning, the connection strengths W _ji and W _kj are modified as appropriate for the binary-to-multi-valued conversion (restoration).

In this way, once the bond strength is determined and fixed,
The neural network of FIG. 3 functions as a system suitable for binary-to-multilevel conversion (restoration).

Next, the ROM 103 is written in the system as shown in FIG. That is, in the ROM writing system of FIG. 4, predetermined binary data of 3 × 3 = 9 bits are sequentially generated, input to the network of FIG. 3, and the output thereof is output.
The data is written to the ROM 103 as ROM write data.
The predetermined binary data is pseudo binary image data.
Then, the ROM written in this manner is stored in the ROM 10 of FIG.
Use as 3.

FIG. 5 is a flowchart for this learning. In step S402, the initial value is given to the weight coefficient W _ji indicating the strength of the connection between the input layer and the intermediate layer and the weight coefficient W _kj indicating the strength of the connection between the intermediate layer and the output layer. As the initial value, a value in the range of -0.5 to +0.5 is selected in consideration of the convergence of the learning process. In step S404, a pixel at an arbitrary position is selected from the learning input data as a pixel of interest (* in FIG. 2A).
(The position indicated by). Steps S404 to S406 are procedures for setting input / output data to the neural network. That is, in step S406, the image data iout in the 3 × 3 area centered on the target pixel is set.
(I) (where i = 1 to 9) is set in the input layer. In step S408, multivalued image data (ideal_out) is prepared as an ideal output corresponding to the target pixel. Step S410 shows a procedure for calculating the output kout (k) based on the given transform coefficient.

That is, the data iout (i) from the input layer is added to the coefficient of the intermediate layer.
Multiplied by the W _ji, and the sum Sum _Fj, , And then the output jout (j) of the j-th hidden layer
Using the sigmoid function to normalize Calculate by Here, F of Sum _Fj is positive (Forwar
means d). Similarly, the output value kout from the intermediate layer to the output layer is obtained as follows. First, using the coefficient W _kj of the output layer, the sum of the products is obtained from the output value jout (j) from the intermediate layer to obtain the value Sum _Fk .

Next, using the sigmoid function to normalize to 0/1, To obtain the output kout (k) of the output layer k.

Thus, FORWARD for a set of sample data
End the calculation of the direction. The following is the procedure for correcting the coupling strength by calculating the BACKWARD, that is, learning from the sample data consisting of the pair of the input and the ideal output.

Therefore, at step S412, compares W _ji of the initial value (set in step S402), the output value kout calculated from W _kj, and an ideal output ideal_out (k) that is prepared in advance. That is, by this comparison, the teacher signal teach_k (k) of the output layer is obtained.
Is calculated as: teach_k (k) = {ideal_out (k) −kout (k)} * kout (k) * {1−kout (k)} (5) Kout (k) * ｛1-ko in the equation (5)
ut (k)｝ has a significance of a differential value of the sigmoid function. Next, in step S414, the width of change of the coupling degree of the output layer △ W
_{Find kj} (k, j). That is, ΔW _kj (k, j) = β * jout (j) * teach_k (k) + α * △ W _kj (k, j) (6) Note that α and β are coefficients of fixed values, and are set to 0.5 in this embodiment. In step S415, this variation width △ W _kj
Based on (k, j), the degree of coupling W _kj (k,
j) is updated. That is, it learns.

W _kj (k, j) = △ W _kj (k, j) + W _kj (k, j) (7) Next, in step S416, the teacher signal teach_j of the intermediate layer
Calculate (j). That is, first, , Calculate the contribution in the BACKWARD direction from the output layer in each neural of the hidden layer. Then, this contribution is normalized by using the differential value of the sigmoid function according to the following equation to calculate the teacher signal teach_j (j) of the intermediate layer.

teach_j (j) = jout (j) * {1-jout (j)} * S
um _Bj (9) Note that teach_j (j) in equation (9) has significance as an ideal signal in the intermediate layer.

Next, in step S418, the teacher signal teach_j (j)
_{Is used} to calculate the change width △ W _ji (j, i) of the coupling degree of the intermediate layer. That is, ΔW _ji (j, i) = β * iout (i) * teach_j (j) + α * △ W _ji (j, i) (10) Then, in step S420, the coupling degree W _ji (j, i) ) To update. That is, W _ji (j, i) = W _ji (j, i) + W _ji (j, i) (11)

Thus, the ideal multivalued image data idea of one pixel of interest
From l_out and the corresponding binary image data, that is, the binary image data of 9 pixels around the corresponding position, the connection degree W
_ji and W _kj are learned, and it is checked in step S422 whether such an operation has been performed for all the target pixels (all sets of sampling data), and until steps S404 to S420 have been executed for all sets of sampling data. Repeat the procedure.

It is considered that the accuracy is low if the learning for all sets of sampling data is performed only once.
Until it is determined in S424 that the accuracy has increased, the processing of steps S404 to S422 is repeated.

It is preferable that the target pixel specified in step S404 be specified randomly instead of sequentially.

In this way, the coupling constant in the neural network in FIG. 4 is determined so as to be optimal for the binary / multivalue conversion.

In the ROM data determination device shown in FIG. 4, since the size of the reference area is set to 3 × 3 in the conversion device shown in FIG. 1, 2 ⁹ = 512 input patterns can be considered. . Therefore, such 512 patterns are sequentially input to the apparatus shown in FIG. 4, and the output is used as ROM data.
However, the top 8 of the output of the neural network in FIG.
Bits are assumed to be ROM data. The reason why the number of bits is set to 8 is that the multi-valued image data to be restored is set to 256 (= 2 ⁸ ) gradations.

The method of determining table ROM data described above can be applied not only to binary-to-multilevel conversion but also to area separation. Especially 2
Region separation from a value image is effective because there is no conventional example.

An example of the application to the area separation will be described below with reference to FIGS. 6 and 7.

FIG. 6A shows an example in which an image includes an image portion in which the original image is a character region and an image portion in which the original image is a photograph region (halftone image region). In FIG. 6A, a restoring process suitable for each image quality is performed while separating a photographic area from a shaded area. FIG. 6B shows that, in the binary image, the binary image portion in which the original image was a character region and the binary image portion in which the original image was a photographic region (halftone image region) are distinguished from both a character and a photograph. When a binary image portion that is an area that cannot be included is included, the character area (the area indicated by hatching in FIG. 6B) and the photographic area, and an image having an intermediate characteristic between these two types of image quality ( (Represented by cross lines in FIG. 6B) while performing restoration processing suitable for each image quality. FIG. 7 is a block diagram of the binary to multi-value conversion processing device. However, when targeting input image data as in FIG. 6A, the ROM 202 will not be required.

In FIG. 7, ROM 201 for character images and ROM 2 for intermediate images
02 and the photographic image ROM 203 have been described with reference to FIGS.
It has ROM data created by the method of creating ROM data. However, it goes without saying that when creating each ROM data, the image data of the character image, the photographic image, and the intermediate image prepared in advance are used as the “ideal output”.

In order to create ROM data for area separation,
A system as shown in the figure is used. That is, at the time of learning, as in the ROM creation system shown in FIG. 8, a binary image as shown in FIG. 6A or 6B is used as a binary input to the neural network. Also, as a logical value of the separation signal, for example, a character area is assigned a value of “2”, an intermediate image area is assigned a value of “1”, and a photographic image is assigned a value of zero. A map of the separation signal corresponding to the position is input to the neural network as an ideal output. A map corresponding to the image example of FIG. 6A is, for example, as shown in FIG. Through such learning, the data in the ROM 204 is determined. The output of the ROM 204 is an optimal separation signal (one of 0, 1, and 2) according to the value of the input binary image data. Selector 205
Receives the separation signal and selects one of the outputs of the ROM 201 to the ROM 203.

In the above example of application to the area separation, the ideal output for the area separation signal uses a logical value determined in advance by a human. This is because learning becomes more accurate. As a method other than the manual method for obtaining the ideal output, there is a method of filtering the multivalued original image, detecting edges, and setting the edge portion to a character area. According to this modification method, learning is performed in the same manner as in the above embodiment, an output is obtained from an input in an M × N reference area, and a weight coefficient (coupling coefficient) is updated so as to approach an ideal output. A table for 2 ^(MXN) input patterns can be obtained from the weight coefficient values obtained.

[Effects of the Invention] As described above, according to the image processing method of the present invention according to the first aspect, the image restoration from binary to multi-valued based on the learning using the learning function of the neural network. It is superior to the conventional table determination because the data of the conversion table is determined.

Further, according to the image processing method according to the second aspect, since the data of the conversion table for image area separation is determined based on the learning using the learning function of the neural network, it is superior to the conventional table determination. I have.

According to the image processing method of the third aspect, the binary image data for learning has a size of m × n, and the ideal output is an original multivalued image of the binary image. When the multi-valued image is binarized, the information of the original image is diffused into pixel blocks of a predetermined size, so that the input data for learning of the neural network at the time of learning is expressed as this pixel block. By using the binary image data, the restoration of the multi-valued image from the binary image becomes accurate.

According to the image processing method of the fourth aspect, the learning of the neural network is based on a back propagation method. Learning by the back propagation method is fast.

According to the image processing method of the fifth aspect, the learning image data has a size of m × n, and the ideal output is at least an area signal determined before learning. As a result, the learning of the neural network is used at the time of generating the data of the table for the image processing for the region separation, and the accuracy of the region separation is improved.

Further, according to the image processing device of the sixth aspect, since the image processing device includes the table created by applying the image processing method, the size of the image processing device itself is reduced.

[Brief description of the drawings]

FIG. 1 is a block diagram showing the configuration of an apparatus according to an embodiment in which the present invention is applied to a 3 × 3 window, and FIGS. 2A and 2B are diagrams each showing input binary image data and ideal output as multiple outputs. FIG. 3 is a block diagram showing a configuration of a neural network used in determining data of a table ROM according to the present embodiment. FIG. 5 is a block diagram showing the configuration of an apparatus for creating ROM data after the coupling constant of the network is determined. FIGS. 5A and 5B are control procedures performed in the network for learning in FIG. FIGS. 6A and 6B show image processing RO in the case where the present invention is applied to image processing for images having different image qualities.
FIG. 7 is a view for explaining input image data input to a neural network to determine M data and area separation ROM data. FIG. 7 shows image processing when the present invention is applied to image processing for images having different image qualities. FIG. 8 is a block diagram showing the structure of the device, FIG. 8 is a block diagram showing the structure of a device for creating ROM for area separation, and FIG. 9 is an ideal output when creating a region separation ROM. FIG. 3 is a diagram showing an input signal as a map. In the figure, 101 ... 3 line buffer, 102 ... latch group, 103 ...
ROM for binary to multi-level conversion, 200 ... Input terminal, 201 ... ROM for character image processing, 202 ... ROM for intermediate image, 203 ... ROM for photo image processing, 204 ... ROM for area separation, 205 ... …selector,
301 ... input layer, 302 ... intermediate layer, 303 ... output layer.

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-62-107573 (JP, A) JP-A-63-184880 (JP, A) JP-A-2-72491 (JP, A) JP-A-2- 215276 (JP, A) JP-A-3-273761 (JP, A) (58) Fields investigated (Int. Cl. ⁶ , DB name) G06T 1/00-1/40 H04N 1/40-1/409

Claims (6)

(57) [Claims] 1. An image processing method for inputting binary image data into a conversion table and restoring the binary image data into multi-valued image data, comprising the steps of: preparing binary image data for learning and an ideal output; Inputting the binary image data and the ideal output to a neural network and learning a coupling coefficient suitable for image restoration between neurons of the network; and inputting the binary image data to the neural network after the learning. A step of determining a conversion table that inputs the binary image data of the input and obtains an output thereof, and that outputs the output of the network as an input; and replacing the neural network with the conversion table. An image processing method using a neural network. 2. An image processing method for image area separation of input image data using a conversion table, comprising the steps of: preparing input image data for learning and an ideal output representing a corresponding image area separation result; Inputting the image data and the ideal output to a neural network and learning a coupling coefficient suitable for image area separation between neurons of the network; and inputting the input image data to the neural network after the learning. Obtaining the output thereof, and determining a conversion table that receives the input image data as an input and outputs the output of the network as an output; and replacing the neural network with the conversion table. Image processing method using 3. The learning binary image data according to claim 1, wherein said learning binary image data has a size of m × n, and said ideal output is an original multi-valued image of said learning binary image. Image processing method using a neural network. 4. The image processing method using a neural network according to claim 1, wherein the learning of the neural network is performed by a back propagation method. 5. The neural network according to claim 2, wherein the learning input image data has a size of m × n, and the ideal output is at least a region signal determined before learning. Image processing method using 6. An image processing apparatus comprising a table created by applying the image processing method according to claim 1 or 2.