JPH08161521A - Image processor - Google Patents

Abstract

PURPOSE: To add adequate expression at a proper place corresponding to the color and pattern of an image. CONSTITUTION: An image direction detecting means 39 detects the image direction of respective pixels of an input image from input image data 1 and extracts image direction components d1. An area dividing means 40 divides the input image into areas by using coordinate values (x) and (y) of respective pixels in the input image and the image direction components d1. An area integrating means 50 integrates areas specified among the respective divided areas. An expression adding means 60 adds indicated expression to image parts of the areas specified among the respective divided areas after the integration.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus for adding an expression to a part of an image, such as changing the brightness and color of a part which becomes a motif of an image such as a photograph.

[0002]

2. Description of the Related Art Desktop publishing has become popular in small offices and individuals, and image processing in desktop publishing has become easier than before due to the development of image editing devices and image retouching systems. ing. As one of the image processes in this desktop publishing, there is a process of adding an expression to a part of an image, such as changing the brightness and color of a part that becomes a motif of an image such as a photograph.

In order to add an expression to a part of an image in this way, conventionally, a certain pixel in the image is specified, and the specified pixel is surrounded by a fixed figure such as a circle or a square to express the expression. A method to specify the area to be added and add the expression to the image part of the specified area, or to specify a pixel in the image, and specify a pixel close to the specified pixel and A pixel is specified by specifying the area to which the expression is added by expanding the area so that it is included in the same area as the specified pixel, and adding the expression to the image portion of the specified area. There is.

[0004]

However, in the former method, the area to which the expression is added is specified by enclosing the specified pixels in the image with a fixed figure. There is a drawback that it is difficult to add an appropriate expression to an appropriate place because it cannot be an appropriate one according to a picture or a texture.

Also in the latter method, since the specified pixel in the image and the pixel whose position and color are close to the specified pixel are the areas to which the expression is added, the area to which the expression is added does not necessarily correspond to the pattern of the image. There is a drawback in that it cannot be made appropriate and it is difficult to add an appropriate expression to an appropriate place.

Therefore, the present invention is suitable for an image processing apparatus that adds expression to a part of an image, such as changing the brightness and color of a part that becomes a motif of an image such as a photograph, in accordance with the color and pattern of the image. This is so that an appropriate expression can be added to the place.

[0007]

In the present invention, when the reference numerals of the embodiment shown in FIG. 1 and described later are made to correspond to each other, the image input means 10 and the area into which the input image from the image input means 10 is divided. Dividing means 40 and its area dividing means 4
Display means 80 for displaying the division result by 0, area specifying means 92 for specifying a part or all of the plurality of divided areas displayed on the display means 80, and area specified by the area specifying means 92 Expression instructing means 93 for instructing an expression to be added to the image part of the
And an expression adding means 60 for adding an expression to the image portion of the area designated by the area designating means 92 based on the instruction by.

[0008]

In the image processing apparatus of the present invention configured as described above, the area dividing means 40 divides the input image into areas according to the color and the pattern, and the operator is displayed on the display means 80. The area to which the expression is added in the input image is specified by designating a part of the area by the area designating unit 92 by looking at the division result. Therefore, the location to which the expression is added depends on the color or pattern of the input image. It is possible to reliably and easily specify an appropriate place, and to appropriately and easily add an appropriate expression to an appropriate place.

[0009]

1 shows an example of an image processing apparatus of the present invention. The image processing apparatus of this example includes an image input unit 10, an image direction detecting unit 39, a region dividing unit 40, and a region integrating unit 5.
0, expression adding means 60, display processing means 70, display means 8
0, and a system control unit 90.

In this example, the image input means 10 has an image input section 11 and image buffers 21, 22, 23. The image input unit 11 is an image scanner unit for reading an image on an original in the system, a unit for generating an image by computer processing, or a unit for capturing an image generated outside the system into the system. In this example, From the image input unit 11, a full-color image expressed in the L ^* a ^* b ^* color space recommended by CIE (International Commission on Illumination) is obtained as an input image.

However, for example, 8-bit image data of each of the lightness component and the two chromaticity components is expressed as l ^* , a ^* , b ^{* in the} drawings, and * is omitted in the specification. Notated as l, a, and b.

The image buffers 21, 22, and 23 are for writing input image data l, a, and b from the image input section 11, respectively.

The image direction detecting means 39 has an image direction detecting section 30 and an image buffer 35 in this example. The image direction detection unit 30 detects the image direction of each pixel of the input image from the input image data 1 read from the image buffer 21 and extracts the image direction component d1. A specific example thereof will be described later. To do. The image buffer 35 is for writing the image direction component d1 from the image direction detection unit 30.

The area dividing means 40, in this example, is an area dividing section 41, a divided area buffer 42 and an image buffer 4.
3 The area dividing unit 41 uses the image buffers 21 and 2.
Input image data 1, a, b,
The coordinate values x and y of each pixel on the input image obtained from the system control unit 90 as read addresses for the image buffers 21 to 23, and the image direction component d1 read from the image buffer 35 will be described later. The input image is divided into regions as follows.

With respect to the area dividing unit 41, the dividing condition specifying means 91 of the system control unit 90 can specify a dividing condition such as the number of divided areas or the number of pixels of each divided area.

The divided area buffer 42 includes an area dividing unit 41.
The area division process in step 1 and the divided area table formed as a result are written in the image buffer 4.
Reference numeral 3 is for writing the correspondence between each pixel of the input image and each divided area, which is obtained as a result of the area dividing process performed by the area dividing unit 41.

The area integration means 50 has an area integration section 51 and an area integration map 52 in this example. The area integration unit 51 integrates the areas designated by the area designation unit 92 of the system control unit 90 among the respective divided regions of the division result by the region division unit 40. In the area integration map 52, the process and result of area integration in the area integration unit 51 are written.

In this example, the expression adding means 60 includes the input image data l, a and b read from the image buffers 21, 22 and 23, and each pixel and each pixel of the input image read from the image buffer 43. The image data I for outputting the input image as it is from the correspondence with the divided areas and the divided area table read from the divided area buffer 42.
1. Image data I for outputting the division result of the input image
2 and the expression designated by the expression designating means 93 of the system control unit 90 with respect to the image portion of the area designated by the area designating unit 92 of the system control unit 90 among the divided regions of the input image division result. The image data I3 for outputting the added image is generated and output.

The display processing means 70 converts the image data I1, I2 and I3 from the expression adding means 60 into display image data expressed in an RGB color space or the like, and the converted image data is Each is converted into an analog image signal.

The display means 80 are, in this example, C
It has three displays 81 to 83 such as an RT display and a liquid crystal display, the display 81 displays the input image as it is, the display 82 displays the division result of the input image, and the display 83 displays the designated area. The image in which the expression is added to the image part of is displayed.

The system control unit 90 includes an operation unit, and although not shown, a CPU, a ROM in which a system control program to be executed by the CPU is written,
And RAM used as a work area for the CPU
Etc. and controls each part of the system. The system control unit 90 uses the division condition designating means 91 and the area designating means 9 described above.
2 and expression instruction means 93.

FIG. 2 shows an example of the image orientation detecting section 30 when the method described in Research Report No. 835, page 80 of the Institute of Electronics, Technology is used.

That is, the image buffer 21 shown in FIG.
The input image data 1 read out from the block circuit 3
3 from the blocking circuit 31 centered on the pixel of interest Pe as shown in FIG. 3A of the input image.
As shown in FIG. 3B, nine pieces of image data la to 3 for a pixel group Gp including pixels Pa to Pi that are three pixels
An image data group Gd consisting of li is taken out and supplied to the convolution operation circuits 32 and 33, and convolution operation is carried out by the coefficient groups Kh and Kv each having a coefficient of a value as shown for the pixels Pa to Pi of the pixel group Gp. Then, from the convolution operation circuits 32 and 33, respectively, CC = la · (−1) + lb · (−1) + …… + li · 1 (1) LL = la · (−1) + lb · 0 + An output represented by + li · 1 (2) is obtained.

The image direction of the input image is the direction of the contour line when the density of the input image is displayed in contour lines, and in this example, the value of the input image data 1 is the density of the input image. The coefficient group Kh is for detecting the horizontal image direction, and the coefficient group Kv is for detecting the vertical image direction.

Further, in the image direction detecting section 30,
Outputs CC and LL of convolution operation circuits 32 and 33
Is supplied to the arithmetic circuit 34, and as shown in FIG. 4, when α = arctan (LL / CC) (3), da = 16 (α + π / 2) / π = 16 {arctan (LL / CC) + π / 2} / π (4) db = log (CC · CC + LL·LL + 1) (5) The 4-bit component represented by the following is calculated.

The component da is the direction of the image direction component extracted from the input image data l, that is, the direction of the contour line when the density of the input image is displayed in contour lines, from da = 0 to da.
Since there is no need to consider the direction of the contour line, the direction of the contour line, that is, the image direction of the input image is shown as a range from 0 to π as shown in FIG. It is enough if done.

The component db is the size of the image-direction component extracted from the input image data 1, that is, the density difference between the contour line of the pixel of interest when the density of the input image is displayed by contour lines and the contour line adjacent thereto. , Db = 0 to db =
It is shown in 16 steps up to 15, and by taking the logarithmic value of the sum of the square of the output CC and the square of the output LL, one is added so that the value will be maintained even when the output CC and LL are small. It is the one.

Specifically, when the image data group Gd shown in FIG. 3B has a value as shown in FIG. 6A or 6B, α = −π / 2 or α = π / 2. And da
= 0, and it is detected that the image direction in the pixel of interest is the vertical direction. The image data group Gd is shown in FIG.
Alternatively, when the value is as shown in (D), α = −π
/ 4 and da = 4, and it is detected that the image direction in the pixel of interest is the direction of π / 4 from the upper left to the lower right.

When the image data group Gd has a value as shown in FIG. 6 (E) or (F), α = 0, and
Since da = 8, it is detected that the image direction in the target pixel is the horizontal direction. The image data group Gd is shown in FIG.
When the value is as shown in (G) or (H), α
= Π / 4 and da = 12, and it is detected that the image direction in the target pixel is 3π / 4 from the upper right to the lower left.

From the arithmetic circuit 34 shown in FIG. 2, that is, from the image direction detecting section 30, the components da and db are output as an 8-bit image direction component d1 as a whole.

The input image is divided into areas by the image processing apparatus of the example shown in FIG. 1, and the divided areas resulting from the division are integrated into areas.
Further, when adding an expression to the image portion of the designated area of each of the divided areas after the integration, the image input unit 11
Input image data l, a, b from the image buffer 21,
First, the operator displays the input image in the system control unit 90 in the state where the input image is displayed on the display unit 22 and 23.
The input image data l, a, b are read from the image buffers 21, 22, and 23 by the operation of displaying them on the screen or automatically by the system control program in the system control unit 90, and the read out. The input image data 1, a, and b are image data I from the expression adding means 60.
1, the image data I1 is converted into a display analog image signal by the display processing means 70, the image signal is supplied to the display 81, and the input image is displayed on the display 81 as it is.

Next, by looking at the display, the operator instructs the system control unit 90 to perform direction detection and area division, or automatically by the system control program in the system control unit 90, as described above. In the section 30, the image direction for each pixel of the input image is detected from the input image data 1 from the image buffer 21 over the entire input image, the image direction component d1 is extracted, and the image direction component d1 is written in the image buffer 35. At the same time, the area dividing unit 41 divides the entire input image into areas by, for example, a K-means algorithm so that the shape of the area can be changed according to the color and the image direction.

The K-means algorithm includes initial region segmentation which sets K initial region centers for the input image,
The divided areas are kept constant by repeating the area division for several orders until the divided areas converge to a certain state, and calculating the area center of each divided area after each area division after the initial area division. It is determined whether or not the state has converged. As the area center, generally, the center of gravity of each divided area and the representative color value (representative pixel value) are set or calculated, but in the example of FIG. 1 of the present invention,
In addition to this, the representative image direction component of each divided area is set or calculated.

In the following, the repetition order of each area division including the initial area division is i (i = o in the meaning of zero in the initial area division, and i = 1, 1 in the subsequent area divisions).
2)) and the number of the area divided by each area division is j (j = 1, 2, ... K), and the area center of each divided area by each area division including the initial area division is Cij. , The center of gravity is Xij, Yij, and the representative color values are Lij, Aij, Bi
j and the representative image direction component are indicated by Dij, respectively. However, the center of gravity Xij, Yij, the representative color values Lij, Aij, B
Details of ij and the representative image direction component Dij will be described later.

The specific procedure of the area division in the area division unit 41 by the K-means algorithm considering such an image direction will be described below with reference to FIG. However, FIG. 7 shows the overall procedure of the area division executed by the area division unit 41,
The following example is a case where the area division number K is 100.

First, in step S1, 100 initial area centers are set for the input image. That is,
From the input image data l, a, b read from the image buffers 21, 22, 23, the image direction component d1 read from the image buffer 35, and the coordinate values x, y obtained from the system control unit 90, For example, as shown in FIG. 8, the input image 1 has a total of 1 of 10 pixels each having the same number of pixels.
The area center Co of each of the divided areas is divided into 00 areas.
j, that is, the center of gravity Xoj, Yoj of each divided region,
As the representative color values Loj, Aoj, Boj and the representative image direction component Doj, the coordinates xo of the central pixel of each divided area
input image data lo for j, yoj, and the central pixel
The image direction component d1oj for j, aoj, boj and the center pixel is obtained.

However, with respect to the representative image direction component Doj, this is taken as 5-bit data, and it is dboj for the central pixel of the 4-bit component db in the image direction component d1.
Is greater than or equal to 1, the central pixel is assumed to have the image direction, the most significant bit of the 5 bits is set to 0, and the lower 4 bits of the central pixel of the 4-bit component da in the image direction component d1 If the daj is taken in and the dbj for the central pixel of the component db is 0, it is assumed that the central pixel has no image direction, the most significant bit among the 5 bits is set to 1, and the lower 4 bits are all set. The value is set to 0 and 16 is set as a whole.

Next, in step S2, the area center Coj of each divided area obtained as described above, that is, the center of gravity Xoj, Yoj (xoj, yoj) of each divided area, and the representative color values Loj, Aoj, Boj (loj, aoj, bo
j) and the representative image direction component Doj are written in the divided area buffer 42 as an initial divided area table.

FIG. 9 shows a divided area table including the initial divided area table, which is created after each subsequent area division, and is expressed as i = o in the initial divided area table.

Next, in step S3, the initial divided area table, that is, the initial area center Coj, is read from the divided area buffer 42, and the image buffer 21,
Input image data l, a, read from 22, 23,
b, the image direction component d read from the image buffer 35
1, and the coordinate value x obtained from the system control unit 90,
From y, all pixels of the input image are sequentially focused on the target pixel P.
n is the pixel of interest Pn and all initial region centers C
The distance Hojn from oj is calculated as follows. n
Is the pixel number of the input image.

However, this distance calculation and the following area determination of each pixel of the input image by this calculation are performed after each next area division until the divided areas converge to a constant state, as described later. It is something that is repeated. That is, after each subsequent region division, all pixels of the input image are sequentially
As the attention pixel Pn, the distance Hijn between the attention pixel Pn and all new area centers Cij by the area division of the rank is calculated.

Therefore, hereinafter, the calculation of the distance Hijn and the determination of the area of each pixel of the input image by this will be described as a common procedure after each subsequent area division including the initial area division. However, the center of gravity Xij, Yij of each divided area by each subsequent area division after the initial area division is the average value of the coordinate values x, y of all pixels in each divided area, and the representative color value Li
j, Aij, and Bij are average values of the input image data l, a, and b for all pixels in each divided area, and the representative image direction component Dij indicates an image direction in which each divided area is remarkable, as described later. The value is set to a value of 0 to 15 or a value of 16 depending on whether or not it has.

In order to calculate the distance Hijn, first, Δxijn = xn-Xij (11) Δyijn = yn-Yij (12) Δlijn = ln-Lij (13) Δaijn = an-Aij (14) Δbijn = Bn-Bij (15) As shown in (15), the centroids Xij, Yij and the representative color values Lij, Aij, Bij of each divided area, and the target pixel Pn
The differences between the coordinates xn, yn of the input image data and the input image data ln, an, bn for the target pixel Pn are individually calculated.

As for the image direction component, as shown in FIG. 5, since the image direction is expressed in 16 steps as an angle in the range from 0 to π, the image direction component is divided into the following three cases, and Representative image direction component Dij and image direction component d1
That of the target pixel Pn of the 4-bit component da in
The difference Δan from an is obtained.

First, the representative image direction component Dij is 16
And when the dbn of the 4-bit component db of the image direction component d1 for the pixel of interest Pn is 0, neither the region center Cij nor the pixel of interest Pn has an image direction, so there is no difference Δdijn, and Δdijn = 0 ... (16)

Secondly, when the representative image direction component Dij is 15 or less and the component dbn is 1 or more, Δdijn = | dan-Dij | ... (17)

However, the difference Δdijn is the representative image direction component D at the area center Cij as shown in FIG.
image direction indicated by ij and the component dan in the target pixel Pn
Corresponds to the angle difference β from the image direction, and the angle difference β is smaller than π / 2 as shown in FIG.
Since jn should be 7 or less, Δdijn = | d
When an-Dij | ≧ 8, Δdijn = 15− | dan-Dij | (18).

Thirdly, the representative image direction component Dij is 16
, And the component dbn is 1 or more, or the representative image direction component Dij is 15 or less and the component dbn is 0, that is, the region center Cij and the target pixel P
When only one of n has an image direction and the other has no image direction, the area center Cij and the target pixel Pn
Difference .DELTA.dijn is larger than that when the image directions are the same, that is, the difference .DELTA.dijn is not 0, and the area center Ci
It is considered that the difference Δdijn is smaller than when the image direction of j and the pixel of interest Pn is deviated by π / 2, that is, the difference Δdijn is smaller than 8, and the difference Δdijn is set between 0 and 7, and Δdijn = 3.5 (19) ).

The above differences Δxijn, Δyijn, Δli
In addition to calculating jn, Δaijn, Δbijn, and Δdijn, a function F (Δxijn, Δ) that controls the size of the area
yijn, Dij) is defined as follows.

That is, as shown in FIG. 11, γ = arctan (Δyijn / Δxijn) (21), and dcn = 16 (γ + π / 2) / π = 16 {arctan (Δyijn / Δxijn) + π / 2} / π ... (22), when | Dij-dcn | ≦ 7, F (Δxijn, Δyijn, Dij) = 1 + ke (| Dij-dcn | /3.5-1) (23) and | Dij When -dcn | ≧ 8, F (Δxijn, Δyijn, Dij) = 1 + ke {(15- | Dij-dcn |) /3.5-1} (24)

However, as is apparent from FIG. 11, the dcn is the direction of the pixel Pn of interest with respect to the area center Cij (center of gravity Xij, Yij), and the area center Cij (center of gravity Xij, Yij).
The angle (γ +) of the xy coordinates centering on Yij) with respect to the y axis
π / 2), 1 from dcn = 0 to dcn = 15
The angle (γ + π / 2) is set in the range of 0 to π by not showing the direction only by the direction. ke is a coefficient as described later.

Since | Dij-dcn | of the equation (23) and (15- | Dij-dcn |) of the equation (24) are respectively 0 or more and 7 or less, the function F (Δxijn, Δ
yijn, Dij) has a spread of ± ke around 1 and can always be 1 when ke = 0 and 0 when ke = 1. Therefore, the coefficient ke is set to 0 <ke <1.

The function F (Δxijn, Δyijn, Dij) for controlling the size of the area by thus defining the coefficient ke
In step S3 of FIG. 7, all pixels of the input image are sequentially set as the target pixel Pn, and the distance Hijn between the target pixel Pn and all the area centers Cij is defined.
Is generally expressed as “(z) ** 2”, the square of z is Hijn = kx · F (Δxijn, Δyijn, Dij) × {(Δxijn) ** 2+ (Δyijn) ** 2} + Kl · (Δlijn) ** 2 + ka · (Δaijn) ** 2 + kb · (Δbijn) ** 2 + kd · (Δdijn) ** 2 (25). However, kx, kl, ka, k
b and kd are arbitrary coefficients.

Next, in step S4, the divided area having the smallest distance Hijn among the area centers Cij of the respective divided areas immediately before the pixel of interest Pn of the input image is divided into the divided areas. The input image is newly segmented so as to include the pixel of interest Pn.

That is, if the distance Hijn to the center Cij of the region included in the first divided region as a result of the immediately preceding region division is the smallest, for example, in the second divided region. , The pixel is included in the second divided area, or is included in the second divided area, for example, from a certain pixel, the distance H from the area center Cij
If it is assumed that the smallest ijn is in the first divided area, the area division is corrected so that the pixel is included in the first divided area.

FIG. 12 shows the relationship between the relative position between the pixel of interest Pn of the input image and the area center Cij of each divided area immediately before, and the representative image direction component Dij of the area center Cij. .

As is apparent from the figure, the function F (Δxijn, Δyij) represented by the equation (23) or (24).
n, Dij) is the area center Cij (center of gravity Xij, Yi)
j) and the direction of the straight line connecting the pixel of interest Pn, that is, the expression (2
The direction indicated by the component dcn represented by 2) is the target pixel Pn.
Is at the point Pn (0) and the component dcn is represented by dcn (0), and when the image direction coincides with the image direction indicated by the representative image direction component Dij, it becomes the minimum, and the direction of the above straight line,
That is, the direction indicated by the component dcn is such that the pixel of interest Pn is at the point Pn (π / 2) and the component dcn is dcn (π / 2).
When it is shifted by π / 2 with respect to the image direction indicated by the representative image direction component Dij, as in the case of

That is, in step S4, the pixels in the direction indicated by the representative image direction component Dij with respect to the region center Cij obtained by the immediately preceding region division are located in the region center Cij even if the pixels are distant from the region center Cij. Pixels that are more likely to be included in the divided area of Cij but are in a direction shifted by π / 2 with respect to the direction indicated by the representative image direction component Dij include those close to the area center Cij and the area center Cij.
The input image is divided into regions so that it is less likely to be included in the divided region of ij.

Therefore, in step S4, the shape of each divided area by the immediately preceding area division is changed to a shape along the direction indicated by the representative image direction component Dij. FIG.
Shows the state in which the area division is corrected in this way and the shape of each divided area is changed.

Next, in step S5, the correspondence between the pixel of interest Pn of the input image and each of the divided regions by the new region division, that is, the divided region to which the pixel of interest Pn of the input image belongs by the new region division belongs. The number is written in the image buffer 43.

Next, in step S6, a new area center is calculated. That is, the image buffers 21, 22,
23, the input image data l, a, b read from 23, the image direction component d1 read from the image buffer 35, the divided area number read from the image buffer 43, and the coordinates obtained from the system control unit 90. From the values x and y, the centroids Xij and Yij, the representative color values Lij, Aij and Bij, and the representative image direction component D are newly divided for each area.
The area center Cij composed of ij is recalculated.

Here, as described above, the center of gravity Xij, Y
ij is the average value of the coordinate values x and y of all the pixels in each divided area, and the representative color values Lij, Aij, and Bij are the averages of the input image data l, a, and b for all the pixels in each divided area. The value. The representative image direction component Dij is obtained according to the calculation procedure shown in FIG.

That is, some pixels of the input image may have a plurality of image directions, but at most two directions can be clearly observed, and if there is a second direction that can be clearly observed, it is the first direction. About π with respect to
Since it is considered that there is a shift of / 2, first, in step S11, dbn is determined for each newly divided region.
Of the pixels satisfying the condition of ≧ 1, the number of pixels N0, N1, for which the component dan has a value of 0, 1, 2, ... 15, respectively.
N2 ... N15 are calculated, and the number N16 of pixels for which dbn = 0 is calculated.

Next, in step S12, the component value Dmax having the largest number of pixels and the largest pixel number Nma.
Calculate x. However, when N16 is the largest,
Dmax = 16. Further, in step S13, it is determined whether Dmax = 16, and Dmax = 16.
If it is determined to be 16, the region does not have an image direction, and in step S14, the representative image direction component Dij of the region is set to 16.

If it is determined in step S13 that Dmax ≠ 16, then in step S15, as shown in FIG.
5 (A) or (B) shows a direction shifted by π / 2 with respect to the direction indicated by the component value Dmax having the largest number of pixels, that is, a component value shifted by +8 or -8 with respect to the component value Dmax. Dmid and the number of pixels Nmid that becomes the component value Dmid are calculated, and the process proceeds to step S16.
It is determined whether the number of pixels Nmax is more than twice the number of pixels Nmid.

If it is determined in step S16 that the number of pixels Nmax is less than or equal to twice the number of pixels Nmid,
Assuming that the area does not have a prominent image direction, in step S14, the representative image direction component Dij of the area is set to 16 as in the case where it is determined that Dmax = 16 in step S13.

When it is determined in step S16 that the number of pixels Nmax is more than twice the number of pixels Nmid, the area has a remarkable image direction, but 2 in the close direction.
It is possible that the directional components are spread and exist such that the two directional components are overlapped with each other. Then, the process proceeds to step S17, and the weighted average value Dave is calculated for the region as follows and the weighted average Dave is calculated. The value Dave,
The representative image direction component Dij of the area is set.

That is, as shown in FIG. 16, ± 1, ± 2, ± 3, ± with respect to the component value Dmax having the largest number of pixels.
4 shifted component values Dp1, Dm1 ... Dp4, Dm4 and the number of pixels Np1, Nm1 ... Np4, Nm4 which become the respective component values Dp1, Dm1 ... Dp4, Dm4 are calculated,
As a weighted average value Dave, Dave = (Nm4.Dm4 + Nm3.Dm3 + Nm2.Dm2 + Nm1.Dm1 + Nmax.Dmax + Np1.Dp1 + Np2.Dp2 + Np3.Np3 + Np3 + Np3 + Np4 + Np3 + Np4 + Np4 + Np4 + Np4 + Np4 + Np4 + Np4 + Np4 + Np4 + Np4 + Np4 + Np4 + Np4 + Np4 + Np4 + Np4 + Np4 + Np4 + Np4 + Np4 + Np4 + Np4 + Np4 + Np4 + Np4 + Np4 + Np4 + Np4;

In step S14, the representative image direction component Dij is set to 16, or in step S17, the weighted average value Dave of the equation (26) is set to the representative image direction component Di.
If j, the process proceeds to step S18, and it is determined whether or not the representative image direction component Dij has been obtained for all the areas. If it is determined that there is an area for which the representative image direction component Dij has not yet been obtained, Returning to S11 and thereafter, the representative image direction component Dij is obtained for the next area as described above.

As described above, in step S6 of FIG. 7, the centers of gravity Xij and Yi are calculated for each newly divided area.
The area center Cij composed of j, the representative color values Lij, Aij, Bij, and the representative image direction component Dij is recalculated.

Next, in step S7, the area center Cij of each divided area thus obtained, that is, the center of gravity Xi
j, Yij, the representative color values Lij, Aij, Bij, and the representative image direction component Dij are used as a divided area table as shown in FIG. 9 while the divided area table for the immediately previous area division is left. Write to the area buffer 42.

Next, in step S8, this new divided area table, that is, the new area center and the previous divided area table, that is, the previous area center, are read from the divided area buffer 42 and divided. Area number j
The deviation amounts of the new area center with respect to the previous area center are calculated as follows between the areas having the same value.

However, here, the new region center is set to Ci.
j, its centroid is Xij, Yij, and representative color values are Lij, A
ij and Bij, the representative image direction component is indicated by Dij, and s = i−1, and the center of the preceding region is Csj, its center of gravity is Xsj, Ysj, and the representative color value is Ls.
j, Asj, Bsj, the representative image direction component are indicated by Dsj, and the shift amount of the new region center Cij from the previous region center Csj is indicated by Hsij.

To calculate the shift amount Hsij, first, ΔXsij = Xij-Xsj (31) ΔYsij = Yij-Ysj (32) ΔLsij = Lij-Lsj (33) ΔAsij = Aij-Asj (34) ΔBsij = Bij−Bsj (35) As shown by (35), the center of gravity Xi of the new area center Cij
j, Yij and representative color values Lij, Aij, Bij,
Differences between the barycenters Xsj, Ysj and the representative color values Lsj, Asj, Bsj of the area center Csj before that are individually obtained.

As to the representative image direction component, as described above, the representative image direction component Di of the area centers Cij and Csj.
Since j and Dsj can take values from 0 to 16,
The difference ΔDsij between the representative image direction component Dij and the representative image direction component Dsj is calculated in the following three cases.

First, the representative image direction components Dij and D
When both sj are 16, ΔDsij = 0 (36).

Second, the representative image direction components Dij and D
When both sj are 15 or less, ΔDsij = | Dij−Dsi | (37).

However, the difference ΔDsij corresponds to the angle difference between the image direction indicated by the representative image direction component Dij and the image direction indicated by the representative image direction component Dsj. The angle difference is smaller than π / 2, and the difference ΔDsij is 7 or less. Should be
When ΔDsij = | Dij−Dsj | ≧ 8, ΔDsij = 15− | Dij−Dsj | ... (38).

Third, the representative image direction components Dij and D
When only one of sj is 16, the difference ΔDsij is larger than when both the representative image direction components Dij and Dsj are 16, that is, the difference ΔDsij is not 0, and the image indicated by the representative image direction component Dij is The difference ΔDsij is smaller than that when the angle difference between the image direction and the image direction indicated by the representative image direction component Dsj is π / 2, that is, the difference ΔDsij is considered to be smaller than 8, and the difference ΔDsij is set between 0 and 7 to obtain ΔDsij. = 3.5 (39)

The above differences ΔXsij, ΔYsij, ΔLs
ij, ΔAsij, ΔBsij, ΔDsij are calculated, and in step S8 of FIG.
Between the areas having the same
The amount of shift Hsi of ij with respect to the previous region center Csj
If j is generally represented by the square of Z as “(Z) ** 2”, then Hsij = kx {(ΔXsij) ** 2+ (ΔYsij) ** 2} + kl · (ΔLsij) ** 2 + ka.・ (ΔAsij) ** 2 + kb ・ (ΔBsij) ** 2 + kd ・ (ΔDsij) ** 2 (40) However, kx, kl, ka, k
b and kd are arbitrary coefficients.

Next, in step S9, it is determined whether or not the shift amount Hsij for all the divided areas is smaller than 1, and the shift amount Hsi for any of the divided areas is determined.
When it is determined that j is 1 or more, it is determined that each divided area has not converged to a constant state yet, and step S3 is performed.
Return to below.

That is, in step S3, all the pixels of the input image are sequentially set as the target pixel Pn, and the distance H between the target pixel Pn and all new area centers Cij is set.
ijn is calculated according to equation (25), then step S4
Proceeding to step 1, the input image is further divided into regions so that the pixel of interest Pn is included in the divided region having the smallest distance Hijn among all the new region centers Cij viewed from the pixel of interest Pn. Next, in step S5, the correspondence relationship between the pixel of interest Pn of the input image and each divided area by new area division is written in the image buffer 43, and then the process proceeds to step S6 to calculate a new area center Cij. Then, in step S7, the obtained new area center Cij is written in the divided area buffer 42 as the divided area table, and then the process proceeds to step S8, in which the new area center Cij is added to each divided area. , A calculation of the shift amount Hsij with respect to the previous region center Csj according to the equation (40) is repeated. To run.

When it is determined in step S9 that the shift amount Hsij for all the divided areas is smaller than 1, it is determined that each divided area has converged to a certain state, and the area division is terminated.

As described above, the area dividing section 41 divides the entire input image into areas so that the shape of the area can be changed according to the color and the image direction. Therefore, a pattern portion having a certain direction (flow), such as a human hair portion or a petal portion in the input image, is divided into, for example, elongated regions along the direction, and the result of the region division is the input image. It is close to what a person perceives.

When the area division is completed, the image buffer 43 stores in the image buffer 43 the correspondence between each pixel of the input image and each final divided area, that is, the divided area to which each pixel of the input image finally belongs. In addition to the number being written, the final area center of each divided area is written in the divided area buffer 42.

After the area division is completed in this way, the operator operates the system control unit 90 to display the division result on the display 82, or automatically by the system control program in the system control unit 90. , The correspondence relationship between each pixel of the input image and each divided area from the image buffer 43, the representative color value of each divided area from the divided area buffer 42, respectively, and the read correspondence relationship and representative color value. From the expression adding means 60, image data I2 is displayed in which each divided area of the input image is filled in with a representative color and is displayed. The image data I2 is converted into a display analog image signal by the display processing means 70, and the image is displayed. The signal is supplied to the display 82, and the division result of the input image is displayed on the display 82. It is.

At this time, at the same time, the input image is displayed on the display 81 as it is, and the operator can compare the input images on the displays 81 and 82 and the division result.

When the number of divided areas of the divided result is large,
If the division result is not suitable for adding the expression as it is to the image portion of the divided area, the operator then instructs the area integration in the system control unit 90 so that the area integration unit In 51, the areas divided by the area dividing means 40 are integrated.

In this case, when the area designation means 92 of the system control unit 90 designates "entire input image",
As shown below, the divided areas of the division result are integrated according to a certain procedure, and when a plurality of areas of the division result are specified, the plurality of areas are integrated as one divided area as described later. It

FIG. 17 shows an area integration procedure executed by the area integration unit 51 when area integration is performed on each of the divided areas of the entire input image.

First, in step S21, the centroids of all the divided areas of the division result are read from the divided area buffer 42 through the area dividing unit 41, and one of them is set as the centroid of the divided area of interest, and this and all other divided areas. Calculate the distance from the center of gravity of.

Next, in step S22, eight divided areas out of all the other divided areas in the order of decreasing distance are listed as belonging to the group of the divided area of interest, and the area integrated map 52 is displayed. Write in.

Next, in step S23, it is determined whether or not other divided areas that should belong to the respective groups are listed for all the divided areas of the division result.
If it is determined that all the divided areas have not been listed yet, the process returns to steps S21 and S22 to sequentially set all the divided areas as the focused divided area, and the above distance calculation and listing are repeated.

FIG. 18A shows a state in which, in the area integrated map 52, other divided areas which should belong to the respective groups are listed in the area integrated map 52 in this way. Other divided areas that should belong to are divided areas 2, 4, 3, 17, 5, 16,
15 are listed.

If it is determined in step S23 that all the divided areas of the division result have been listed, then in step S24, the divided area buffer 4 is generated.
The representative color value of one target divided area as a result of the division is read from 2 and the numbers of all other divided areas listed as belonging to the group of the target divided area are read from the area integrated map 52, and the divided area is also read. The representative color values of all the other divided areas listed are read out from the buffer 42, and the difference between the representative color value of the target divided area and the representative color values of all the other divided areas listed is calculated.

Next, in step S25, it is determined whether or not the difference is smaller than a predetermined value, and other divided areas smaller than the predetermined value are left as listed divided areas, but other divided areas equal to or larger than the predetermined value. Rewrites the area integrated map 52 so as to exclude it from the listed divided areas.

Next, in step S26, it is determined whether or not the list of other divided areas that should belong to the respective groups has been updated for all the divided areas of the divided result, and all the divided areas still remain. When it is determined that the list-up update is not performed, the process returns to steps S24 and S25 so that all the divided areas are sequentially set as the target divided area, and the difference calculation and the list-up update described above are repeated.

FIG. 18B shows a state in which the area integrated map 52 is updated for all the divided areas in this way. For example, as the other divided areas that should belong to the group of the divided area 1, the divided area 2 , 4, 17, 16, and 15 are left, and the divided areas 3 and 5 are excluded.

If it is determined in step S26 that the list-up update has been performed for all the divided areas of the division result, then in step S27, the duplicated data is deleted from the area integrated map 52.

That is, in the state of FIG. 18B, for example, the divided areas 2 and 1 are listed up redundantly as belonging to the other group, and the divided areas 4, 17 and 16 are respectively divided areas. 1 and 2 are listed redundantly as belonging to the group 1, divided area 6 is listed as belonging to the group of divided area 2, and divided area 14 is listed as belonging to the group of divided area 4. Up, divided area 1
5 is listed up redundantly as belonging to the group of divided areas 1 and 4, and divided area 8 is divided areas 3 and 5
18C, the divided areas 1, 2, 4, 6, 14, 17, 16, 15 are divided into the divided areas 1 as shown in FIG. 18C. The divided areas 5, 8 and
It is assumed that 7 belongs to the group of the divided areas 3 and none belongs to the group of the divided areas 2, 4 and 5.

Next, in step S28, the number of the divided area to which each pixel of the input image belongs is read from the image buffer 43, and the area integrated map 52 is referred to by that number to determine the divided area to which each pixel of the input image belongs. The group to which the pixel belongs is checked, and the number of the group is used as the number of the divided area after the integration, and the number of the divided area after the integration to which each pixel of the input image should belong, that is, each pixel of the input image and each divided area after the integration. The correspondence relation of is written in the image buffer 43.

Next, in step S29, each divided area is integrated based on the area integration map 52 and the above correspondence in the image buffer 43, that is, the area integration map 52 is listed as belonging to one group. Using the plurality of segmented areas that have been updated as one segmented area after integration, the center of gravity, representative color value, and representative image direction component of each segmented area after integration are recalculated, and the area center of each segmented area after integration is calculated. The divided area table after the integration is written in the divided area buffer 42, and the area integration over the entire input image is completed.

When a plurality of areas of the division result displayed on the display 82 are designated by the mouse operation or the like in the system control unit 90, the areas are integrated so that the plurality of areas become one divided area. The image buffer 43 is rewritten, and the divided area table of the divided area buffer 42 is rewritten.

After the area integration is completed in this way, the operator operates the system controller 90 to display the integrated division result on the display 82 or the system control program in the system controller 90. The correspondence between each pixel of the input image and each divided area after integration is automatically read out from the image buffer 43, and the representative color value of each integrated area after integration is read out from the divided area buffer 42. Based on the correspondence relationship and the representative color value, the expression adding unit 60 obtains image data I2 in which each divided area of the input image after integration is filled in with a representative color and displayed, and the image data I2 is displayed by the display processing unit 70. Is converted into an analog image signal for display by the image signal, and the image signal is supplied to the display 82 to be displayed. 82 division result after the integration of the input image is displayed on.

At this time, at the same time, the input image is displayed as it is on the display 81, and the operator can compare the input images on the displays 81 and 82 and the division result after the integration.

Looking at the input images on the displays 81 and 82 and the division result after the integration, the operator then designates the division area after the integration by the area designating means 92 of the system control unit 90, and designates it. The expression adding unit 60 instructs the expression adding unit 60 to specify the type of expression to be added to the image portion of the divided area and various parameters accompanying the expression, so that the expression adding unit 60 specifies the specified divided area after integration. The following various expressions are added to the image portion of.

First, there is an expression that can be called dramatic filter processing. When this expression is instructed, in the expression adding means 60, the representative color values Lr, A of the specified divided area after integration are set.
A value obtained by multiplying the difference Lr-ln between the lightness component Lr in r and Br and the lightness component ln in the pixel values ln, an and bn of each pixel in the divided area by a coefficient k1 is added to the lightness component ln, lr = ln + k1 (Lr−ln) = (1−kl) ln + kl·Lr (51)

Image data I3 in which the lightness component ln is replaced by the lightness component lr represented by the equation (51) for each pixel in the divided area thus designated is the display processing means 7.
0 is converted into a display analog image signal, and the image signal is supplied to the display 83,
The image after the expression is added is displayed on the screen 3.

When the coefficient k1 is set to 1, the lightness component ln is replaced by the lightness component Lr of the average value of the divided area for each pixel in the specified divided area, and the coefficient k1
By adjusting between 0 and 1, the brightness distribution of the specified divided area can be expanded or contracted around the average brightness of the divided area, giving different impression to the image. You can In particular, when this expression is added only to the image portion serving as the motif of the input image, the image portion serving as the motif can be dramatically enhanced.

When the coefficient k1 is set to a value close to 1, a streak like a Mach band can be seen in the output image, but unlike the Mach band that occurs when the number of colors is simply reduced, the appearance is noticeable. There is no such thing.

Second, there is an expression that can be called cutout filter processing. When this expression is instructed, in the expression adding means 60, the representative color values Lr, Ar, Br of the specified divided area after integration are set.
And pixel values ln, a for each pixel in the divided area
The pixel values ln, an and b are obtained by multiplying the differences Lr-ln, Ar-an and Br-bn from n and bn by a coefficient k2.
It is added to n, respectively. The processing represented by (Br-bn) = (1-k2) bn + k2 · Br (54) is performed.

Here, when the coefficient k2 is set to 1, each pixel in the designated divided area is filled with the representative color of the divided area, and an effect like a cutout is obtained, and the coefficient k2 is changed from 0 to 1. By adjusting in between, the color distribution of the specified divided area can be expanded or contracted around the representative color of the divided area, and a different impression can be given to the image. In particular, when this expression is added only to the image portion that becomes the motif of the input image, the effect is great.

Third, there is an expression that can be called dodging filter processing. If this expression is given,
In the expression adding unit 60, for the specified divided area after integration, for example, the divided area C1 shown in FIG.
For 0, C14, C15, and C19, the brightness component In for each pixel in the divided area is shown in FIG.
(B) As shown in (C), a lightness component lp having a constant value k3 is added, and for the divided areas in contact with the designated divided area, for example, divided areas C8, C9, C shown in FIG.
As for 13 and C16, as shown in FIGS. 19B and 19C, with respect to the lightness component ln for each pixel of the divided area, a constant value k3 is set for the pixel at a position in contact with the specified divided area. , 0 for pixels located on the opposite side, and k3 for pixels located between them according to the distance from the pixel at the position in contact with the specified divided area.
A process of adding the lightness component lq having a value from 0 to 0 is performed.

In a photographic image, the image may be partially crushed,
However, when performing the above dodging filter processing in the present invention, the area designating unit 92 determines the area where dodging is required. The value can be specified reliably and easily in units, and since the brightness changes gently from the specified area to the surrounding area, the image after dodging does not appear unnatural.

Fourth, there is an expression that can be called wet-on-wet filter processing. When this expression is instructed, in the expression adding means 60, for the pixels existing near the boundary of the specified plurality of divided areas after integration, the representative color of the plurality of divided areas is set to the position of the pixel. The color obtained by mixing with the mixing ratio according to is treated as the color of the pixel, and the effect that wet paints are applied next to each other on the image is obtained.

Although not shown, by converting the image data I3 from the expression adding means 60 into an image signal for print output and supplying the image signal to the printer, it is possible to print the image to which the various expressions described above are added. it can.

Although the above-mentioned example is the case where the input image is divided into regions by the K-means algorithm, the K-means algorithm generally calculates the distances between all the pixels of the input image and all the new center of regions. Therefore, when the input image size is large, a large work area or divided area buffer is required, and the area division processing requires a long time.

Therefore, the inventor can generally divide a large-sized input image into regions at high speed by using a small-sized work area or divided region buffer.
A method called a divided K-means algorithm was invented and proposed by Japanese Patent Application No. 6-96328.

This divided K-means algorithm divides the input image into a plurality of middle regions and divides each middle region into regions by the K-means algorithm. However, the input image is simply divided into a plurality of middle regions. , If each middle area is divided into regions, the connection of the finally divided regions will be poor at the boundaries of each middle region. In the area division of an area, all boundary areas that are in contact with the middle area are area-divided together.

Also in the example shown in FIG. 1 of the present invention, the input K-algorithm can divide the input image into regions. However, it is desirable that the divided K-means algorithm uses the image direction component d1 extracted from the input image data 1 as in the above-mentioned K-means algorithm.

The pixel size of the image direction detecting section 30 does not need to be limited to 3 × 3 pixels as shown in FIGS. Also, not only from the input image data l,
For example, input image data 1, a, b are added, and the image direction of the input image is detected from the input image data of the addition result,
The image direction component d1 can also be extracted. Further, according to the present invention, the input image may be, for example, RGB other than the Lab color space.
The invention can be applied not only to a full-color image expressed in a color space or the like, but also to a monochrome image such as a monochrome image.

[0122]

As described above, according to the present invention, it is possible to add an appropriate expression to an appropriate place according to the color and pattern of an image.

[Brief description of drawings]

FIG. 1 is a functional block diagram showing an example of an image processing apparatus of the present invention.

FIG. 2 is a functional block diagram showing an example of an image direction detecting section.

FIG. 3 is a diagram for explaining detection of an image direction.

FIG. 4 is a diagram for explaining detection of an image direction.

FIG. 5 is a diagram showing a relationship between a value of an image direction component and an image direction.

FIG. 6 is a diagram showing a relationship between a value of input image data and an image direction.

FIG. 7 is a flowchart showing an overall procedure of area division in an area division unit.

FIG. 8 is a diagram showing an example of a mode of initial area division.

FIG. 9 is a diagram showing an example of a region division table.

FIG. 10 is a diagram for explaining a distance calculation in area division.

FIG. 11 is a diagram for explaining distance calculation in area division.

FIG. 12 is a diagram for explaining new area division.

FIG. 13 is a diagram showing a state in which the shape of each divided area is changed by new area division.

FIG. 14 is a flowchart showing a procedure of calculating a representative image direction component of a new area center.

FIG. 15 is a diagram for explaining calculation of a representative image direction component.

FIG. 16 is a diagram for explaining calculation of a representative image direction component.

FIG. 17 is a flowchart showing a procedure of area integration in an area integration unit.

FIG. 18 is a diagram showing how the area integrated map is sequentially updated.

FIG. 19 is a diagram for explaining the dodging filter process.

[Explanation of symbols]

 10 image input means 39 image direction detecting means 40 area dividing means 50 area integrating means 60 expression adding means 80 display means 90 system control section 92 area designating means 93 expression instructing means

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification number Office reference number FI Technical display location H04N 1/40 H04N 1/40 Z

Claims (1)

[Claims] 1. An image inputting means, an area dividing means for dividing an input image from the image inputting means into areas, a display means for displaying a division result by the area dividing means, and a plurality of display means displayed on the displaying means. Based on an area designating means for designating a part or all of the divided areas, an expression designating means for designating an expression to be added to an image portion of the area designated by the area designating means, and an instruction by the expression designating means. An image processing apparatus comprising: an expression adding unit that adds an expression to an image portion of an area designated by the area designating unit.