JPH07113969B2 - Image processing method - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method, and more particularly to an image processing method capable of cutting out a specific image from an image by interactive processing with an image processing apparatus.

[Prior Art] For example, a process of cutting out a specific image of only a person (hereinafter, referred to as a subject image) from a photographic image and removing the remaining image (hereinafter, referred to as a background image) generally requires skill and is a process. Takes a lot of time and labor. Therefore, various image processing apparatuses have been proposed to facilitate this type of image processing.

One is to connect an image display device to the electronic computer and display the target image there, and the operator uses a coordinate input device such as a joystick to display the cursor over the image of the display in the image. It is moved along the boundary between the subject image and the background image, and the cutout coordinates are input to the computer. The computer cuts out the subject image or erases the background image using this coordinate data. According to this device, the subject image can be enlarged and cut out, and the correction is easy, but the operator still needs to accurately trace the contour of the subject image, which requires considerable time. Requires skill.

The other one is to perform spatial differentiation of pixel intensities for the entire image and detect pixels whose absolute value of the differential coefficient is larger than a certain threshold as boundary candidates of the subject image. In general,
There is a marked change in pixel intensity at the boundary between the subject image and the background image. Therefore, the apparatus according to such a method is useful for enabling automatic extraction of a subject image. However, with this method, discontinuous boundary candidates are detected not only inside the subject image but also inside the background image, and therefore it is necessary to remove unnecessary boundary candidates. Moreover, conventionally, this process has been manually and skillfully referred to.

Still another one focuses on the density difference and the color difference between the subject image and the background image, and determines whether or not the subject image is inside by detecting the pixel intensity or the color identity. . However, as a practical matter, the target boundary is not accurately detected because the density and color are rarely uniform in the subject image.

As described above, in the conventional device, considerable skill and labor were required to extract the subject image.

[Problems to be Solved by the Invention] The present invention has been made in view of the above-mentioned drawbacks of the related art, and an object of the present invention is to determine the commonality of the subject images by the determination factor An object of the present invention is to provide an image processing method capable of quickly and easily extracting a subject image without requiring skill by discriminating images and minimizing intervention of operator operation.

[Means for Solving Problems] The present invention is an image processing method for extracting a specific image portion of input image data, wherein an image corresponding to the input image data is displayed on a display, and the displayed image is displayed. Referring to a plurality of sample data included in the edge portion of the specific image portion to be extracted specified by the operator above, based on the referred to the plurality of sample data, determine the edge detection parameter of the specific image portion, the edge It is characterized in that the specific image portion is extracted using a detection parameter.

Embodiments Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. FIG. 1 is a block diagram of the image processing apparatus of the embodiment. In the figure, reference numeral 1 is an image input device such as a CCD camera or a drum scanner.
The primary color digital image data is stored in the RGB image image area 61 of the image memory 6 via the central processing unit (CPU) 2. This is the original image. The original image 61 is based on the CPU2, the hue H (Hue), the brightness V
(Value) and saturation S (Saturation) are calculated, and the calculation result is stored in the HSV image area 62. These are a hue image H, a lightness image V, and a saturation image S. Further, the CPU 2 obtains the edge degrees dH / ds and dV / ds for the hue image H and the lightness image V by differential filters, respectively, and calculates the calculation results as dH / ds.
The image area 63 and the dV / ds image area 64 are stored. These are the hue edge image dH / ds and the lightness edge image dV / ds. Next, an operator operation intervenes to determine which of the hue edge image dH / ds and the lightness edge image dV / ds discriminates the subject image well, and preferably one edge image is selected and selected. In the edge image, several points on the edge that surround the subject image are designated. Next, the CPU 2 determines a predetermined threshold width based on the edge pixel information of the designated point,
The binarization processing is performed so that the edge pixels included in the threshold width and the other edge pixels are separated. At the same time, unnecessary isolated points are removed. Reference numeral 5 is a coordinate input device for inputting coordinates for designating points on the edge image, which is, for example, a digitizer. A color monitor 4 displays various images in the image memory 6 in color as required. An image output device 7 such as a printer or a film recorder is used to output a cut-out subject image, a background-filled image, or a mask image formed to cut out the subject image. The series of processing and control functions described above are realized by the central processing unit (CPU) 2. For this purpose, for example, the processing programs shown in FIGS. 2 and 3 are RAM, It is stored in the main memory 3 composed of a ROM. 8
Is a common bus of CPU2 connecting between blocks.

Hereinafter, a case where a subject image is cut out using this device will be described.

FIG. 3 is a flow chart showing the calculation process of hue H and lightness V. In step S31, the maximum data Max (RGB) is extracted from each R, G, B value of one pixel data and stored in the register M. Conversely, in step S32, the minimum data Min (RGB) is extracted and stored in the register m. In step S33, the contents of the registers M and m are added and divided by 2, and the result is stored in the brightness register Br. That is, 3 of the original image
The hue H is removed from the primary color RGB data to abstract only the lightness V, and the result is stored in the lightness image image V of the HSV image 62. This is one of the judgment factors that well show the commonality within the subject area. At step S34, it is determined whether or not the contents of the register M and the register m are equal. If they are equal, there is no hue, so the flow advances to step S41, and that fact is defined by "0". That is, in the image, the pixels that have no hue are “0”.
To be This is because only the hue component is abstracted. If not equal in step S34, register M in step S35.
And the contents of the register R are equal to each other. If they are equal to each other, a large amount of red component is included, so the flow advances to step S37 to store the calculation result of {2+ (GB) / (Mm)} in the hue register Hr. The constant "2" on the right side of the formula means a multiple of the angle of 60 degrees to be multiplied later by the process of step S40, and means that the hue including a large amount of red component is encoded around 120 degrees as an angle. To do. When they are not equal in step S35, it is determined in step S36 whether the contents of the register M and the register G are equal. If they are equal to each other, a large amount of green component is included, so the flow advances to step S39 to store the calculation result of {4+ (B−R) / (M−m)} in the hue register Hr. Similarly, the constant "4" means that the hue containing a lot of green component is encoded around 240 degrees in angle. If it is not equal in step S36, a large amount of blue component is contained, so the flow proceeds to step S38, and the hue register Hr is set to {(RG).
The calculation result of / (M-m)} is stored. It means that a hue containing a lot of green component is coded around 0 degree in angle. In step S40, the content of the hue register Hr is multiplied by an angle of 60 degrees and encoded. In this case, the 360 base system is used. As described above, only the hue H is abstracted from the three primary color RGB data, and the result is stored in the hue image image H of the HSV image image 62. This is another way to judge the commonality within the subject area.

The saturation S is not used in this embodiment.

FIG. 2 is a flow chart showing the clipping process of the subject image. In step S1, the original RGB image data is input and stored in the area 61 of the image memory 6. Step S2
Then, the RGB image data is converted into the hue image H, the saturation image S, and the lightness image V by the processing of FIG. 3 described above, and stored in the area 62 of the image memory 6. For color images,
It is advantageous to discriminate the subject images by treating only the hue H and the lightness V in abstraction rather than the RGB image data themselves. This is because the RGB data itself is affected by the lightness even if the hue is the same, and the hue is affected even if the lightness is the same. In the embodiment, each 8 bit data of RGB is set to H.
The data is converted into 8-bit SV data, and the hue image H and the brightness image V are displayed on the color monitor 4 in step S3.

In step S4, the hue image data and the lightness image data are respectively subjected to differential filters to extract edge images of hue and lightness, and each edge image is displayed on the color monitor 4.

FIG. 4 is a diagram showing a pixel array of a hue image or a lightness image in relation to the differential operator of the embodiment. In FIG. 4, each pixel A to I of the hue image or the brightness image corresponds to an example of a 3 × 3 differential filter, and the edge component E is calculated by the following equation, for example.

E = {(A + B + C-G-H-I) 2 + (A + D + G-
In C-F-I) ² } ^1/2 step S5, the operator looks at the monitor 4 and the edge portion of the subject image is well represented in the hue edge image or the lightness edge image, or both. It is determined whether or not both of the edge images appear well or dispersedly, and the edge image to be used for the subsequent extraction processing is determined.

In step S6, for the selected edge image, several points on the edge line that surround the subject image are designated. This designating method is performed, for example, by pointing a few points on the edge line with a light pen or the like on the display (while observing on any image display in the embodiment of the present invention, the edge portion of the specific image portion to be extracted on the image is displayed. Specify multiple sample data included).

In step S7, the pixel data of several points on the specified edge line are referred to, and the binarization processing of the edge image is performed. This 2
The width of the initial threshold value used for the binarization is set to the range between the minimum value and the maximum value of the pixel data of several points on the specified edge line, and the edge pixel data included in this range becomes a logical "1". It is binarized, and the other edge pixel data is binarized to the logic "0".

FIGS. 9 (a) and 9 (b) are histogram diagrams in which the edge intensities of the respective pixels are aggregated for several designated points. For example, if this histogram is distributed as shown in FIG. 9 (a) or (b), the corresponding initial threshold width is CPU2
Automatically set to ITH. However, when the final result of the subject image extraction performed with this initial threshold width ITH is not satisfactory, the threshold value width is widened to WTH as shown in FIG. Alternatively, the threshold width is set to NTH as shown in FIG. 9 (b). In this way, the optimum edge line binarization process for subject image extraction is achieved. Also step S1
It is also possible to respecify the points by the judgment of the operator of 1. When a few points on the edge are specified in step S6, the binarized edge line width may be narrowed depending on the dark or thin portion of the edge, which may cause breakage. Therefore, the operator may specify the dark portion and the light portion on the edge line. That is, it is sufficient to mechanically specify at regular intervals so as to surround the subject image.

In step S8, isolated point removal processing is performed. The image binarized by the processing of step S7 contains a lot of noise, and it is necessary to remove isolated points to remove it.

FIG. 5 is a view of an isolated point removal window of N × M pixels provided for the purpose of automating isolated point removal. The figure is N × N
Shows the case. For example, when the 4 × (N−1) pixel data corresponding to the outer circumference 22 of the rectangular window as shown in the figure is viewed and all the values are added together, the sum value becomes “0”. In this case, it is determined that the pixel clusters in the {(N-2) × (N-2)} images on the inside surrounded by all the pixels on the outer periphery 22 are isolated, and the pixel clusters are removed. .

6 (a) to 6 (e) are diagrams for explaining the process of determining the size of the isolated point removal window. FIG. 6 (a) assumes an edge line 32 that finally surrounds the subject image 31. Before that, the CPU2 sets the designated points shown in Fig. 6 (b).
A line segment that connects between 33 with a straight line is assumed, and the line segment A having the shortest length is detected. As a result, CPU2
As shown in FIG. 7C, the line segment A is the outer periphery of the isolated point removal window.
Initialize the size so that it has a radius of approximately 22. The CPU 2 scans the entire screen in this isolated point removal window and removes isolated points. However, by removing the isolated points in this way, some edge lines of the subject image may be lost. Also in this case, the operator operation intervenes at step S12, and the size of the outer periphery 22 of the isolated point removal window is narrowed as shown in FIG. 6 (d), or FIG. 6 (e).
In this way, the size of the outer periphery 22 of the isolated point removal window can be widened and adjusted.

In step S9, edge line tracking around the subject image is performed,
Binarized edge lines are connected so as to surround the subject image.

FIG. 7 is a diagram for explaining the process of connecting the binarized edge lines with the n × m pixel tracking window 30. The CPU 2 uses the tracking window 30 and, for example, the first designated edge line.
Assuming a vector 34 starting from the point 33 ₁ on the point 32 and pointing to the second designated point 32 ₂ on the edge 32, using the direction of the vector 34 as a guide, the actual binarized edge line Track 32. If the size of the tracking window 30 is appropriate, both sides sandwiching the edge line 32 can include images that are not always the edge line 32, and the tracking window 30 is configured so that the non-edge pixel amounts on both sides are equal. Proceed on the binary edge line 32. Thus the center of the tracking window 30 moves toward the point 33 ₂ can be formed as edge line surrounding the object image with the moving locus of the center point.
Further, the fine fluctuation of the center point can be smoothed by adding the direction of the vector 34, for example.

In step S10, the operator judges whether the edge extraction is good or bad. For example, the original image and the edge line image obtained in step S9 are overlay-displayed and evaluated. If you like, this subject image is cut out along the edge line and then the shooting is performed. If not satisfied, the process proceeds to step S11 to ask the operator whether to re-designate the edge points or change the parameters. To re-specify the edge points, return to step S6. If not, the process proceeds to step S12, and if necessary, the parameters such as the threshold width of the binarization process, the size of the isolated point removal window outer periphery 22 and the size of the tracking window 30 are changed to perform the step.
Return to S7.

FIGS. 8 (a) to 8 (c) are views showing one mode of parameter change in step S12. As a result of the binarization process and the isolated point removal process, the binarized edge line becomes discontinuous on the way as shown in FIG. 8 (b), and therefore the edge line of the final extracted image must be smoothly connected to the operator. If the judgment is made, the operator gives an instruction to thicken the binarized edge. As a result, the CPU 2 widens the threshold width of the binarization process and thickens the edge line. At the same time, the CPU 2 reduces the size of the isolated point removal window to reduce the size of the isolated point to be removed. As a result, as shown in FIG.
It is expected that the digitized edge lines will continue. Conversely, when the binarized edge line is too thick as shown in FIG. 8 (c), the operator gives an instruction to thin the edge line. This makes CP
U2 increases the threshold width to increase the size of the isolated point removal window. By the intervention of the relatively simple operation of the operator as described above, a satisfactory extraction of the binarized edge line can be obtained.

The above-described subject image cutting technique can also be applied to the field of image edge extraction, which is a prerequisite for the technique.

The edge tracking processing described above can also be applied to tracking rivers and roads in remote sensing.

In other words, it can be applied in the fields of edge extraction and edge tracking in image processing.

EFFECTS OF THE INVENTION As described above, according to the present invention, a parameter for edge detection of a specific image portion is determined according to data of a plurality of user-specified sampling points included in the edge portion of the specific image portion. , Since the specific image part is extracted using the edge detection parameter, it is not necessary to specify all the image parts to be extracted on the display, and the operability is good,
In addition, there is an effect that the specific image can be extracted with high accuracy as compared with the case where the user specifies one point as the sampling point.

[Brief description of drawings]

FIG. 1 is a block configuration diagram of an image processing apparatus according to an embodiment, FIG. 2 is a flow chart showing a cutting process of a subject image, FIG. 3 is a flow chart showing a calculation process of hue and lightness, and FIG. 4 is a differential operator. FIG. 5 is a diagram showing a pixel array of a hue image or a lightness image in relation to FIG. 5, FIG. 5 is a diagram of an n × m pixel isolated point removal window provided for the purpose of automating isolated point removal, and FIGS. e) is a diagram for explaining the process for determining the size of the isolated point removal window, FIG. 7 is a diagram for explaining the process for connecting the edge lines with a tracking window of n × m pixels, and FIGS. 8 (a) to 8 (c) are FIG. 9 (a) and FIG. 9 (b) are diagrams showing one mode of parameter change, and are histograms in which the edge intensities of the respective pixels are aggregated for several designated points. In the figure, 1 ... Image input device, 2 ... Central processing unit (CPU), 3 ... Main memory, 4 ... Color monitor, 5 ... Coordinate input device, 6 ... Image memory, 7 ... Image output device, 8 ... Common It's a bus.

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical display location 7459-5L G06F 15/70 335 Z (72) Inventor Kazunori Urushihara 770 Shimonoge, Takatsu-ku, Kawasaki-shi, Kanagawa Address Canon Inc., Tamagawa Plant (72) Inventor Susumu Matsumura 770 Shimonoge, Takatsu-ku, Kawasaki City, Kanagawa Prefecture Canon Inc., Tamagawa Plant (72) Hiroshi Omura 770 Shimonoge, Takatsu-ku, Kawasaki City, Kanagawa Canon Inc. Company Tamagawa Plant (56) References JP-A-60-169977 (JP, A)

Claims (1)

[Claims] 1. An image processing method for extracting a specific image portion of input image data, wherein an image corresponding to the input image data is displayed on a display, and an extraction specified by an operator is performed on the displayed image. Referring to a plurality of sample data included in the edge portion of the specific image portion to be, based on the referred to the plurality of sample data, determine the edge detection parameter of the specific image portion, the specific image portion using the edge detection parameter An image processing method comprising: