JP6930099B2 - Image processing device - Google Patents

Description

The present invention relates to an image processing apparatus, an image processing method, an image processing system, and a program.

The process of cutting out a specific area from an image is one of the indispensable fields of image editing and processing, such as analyzing the characteristics of a specific area or designing a specific area. Various approaches can be considered as a method for cutting out a specific region, and for example, the GraphCut method is a known representative method.

In Patent Document 1, a user instruction receiving unit that acquires position information representing a representative position of a designated area designated by the user as a specific image area from the image and pixels constituting the image are regarded as vertices and adjacent to each other. A graph setting unit that sets a graph consisting of weights determined according to the side connecting pixels and the closeness of chromaticity between adjacent pixels, and reduction that reduces the vertices of the set graph according to a predetermined rule. An image processing apparatus including a reduction unit and a region detection unit that detects a designated region based on the position of a pixel corresponding to a contracted vertex is disclosed.
Further, in Patent Document 2, one pixel included in a designated area in an image is selected as a reference pixel, and a specific range around the reference pixel is set as a first range, and only one selected reference pixel is targeted. After determining whether or not each of the target pixels included in the first range belongs to the designated area, a new reference pixel is selected, the first range is set and determined again, and the designation is made. A region determination device that detects a region is disclosed.

Japanese Unexamined Patent Publication No. 2016-4309 Japanese Unexamined Patent Publication No. 2016-6647

When the user performs image processing, it is necessary to cut out a designated area designated as an area for image processing by the user. At this time, a seed may be input as a representative position of the designated area for the designated area, and the designated area may be cut out by an area expansion method for expanding the area based on the seed.
However, when expanding the area, even if the area is outside the designated area, the area having a similar color may be regarded as the designated area.
An object of the present invention is to provide an image processing apparatus or the like capable of cutting out a designated area with higher accuracy when cutting out a designated area.

The invention according to claim 1 includes a position information acquisition unit that acquires position information representing a representative position of a designated area in an image, and an area detection unit that detects a designated area from the position information. Detects the designated area based on the similarity between the texture around the pixel belonging to the designated area and the texture around the pixel to be determined whether or not it is included in the designated area, and determines the representative position. An image division unit that divides the image in which is set for each block, and a local area setting unit that sets a first local area and a second local area as a range for calculating a texture in the block as a unit. The image processing apparatus includes a similarity calculation unit for calculating the similarity between the texture in the first local region and the texture in the second local region.
According to the second aspect of the present invention, when it is detected that the pixels in the second local region are included in the designated region by the similarity, any pixel in the second local region is included in the designated region. The image processing apparatus according to claim 1 , further comprising a pixel setting unit for setting whether or not the image is included in the above.
In the invention according to claim 3 , the similarity calculation unit further calculates the strength to which the second local region belongs to the designated region based on the similarity, and uses the strength to obtain the first. The image processing apparatus according to claim 1 or 2 , wherein the designated area is detected by determining whether or not the pixels in the local area of 2 belong to the designated area.
In the invention according to claim 4, after the area detection unit detects a designated area having the block as a unit, the area detecting unit cancels the setting of the designated area in the boundary area of the designated area having the block as a unit, and designates the designated area. The image processing apparatus according to claim 1 , wherein the designated area is detected again in the boundary area of the designated area based on the reference pixel selected from the pixels for which the area setting has not been released. ..

According to the invention of claim 1 , the processing speed is further increased.
According to the invention of claim 2 , pixels for expanding the designated area can be set.
According to the invention of claim 3 , the processing speed is further increased.
According to the invention of claim 4 , it is possible to cut out a designated area with higher accuracy .

It is a figure which shows the configuration example of the image processing system in this embodiment. It is a block diagram which shows the functional structure example of the image processing apparatus in this embodiment. It is a figure which showed the example of the method of performing the work of specifying a designated area interactively by a user. (A) to (c) show how a designated area is cut out from the image shown in FIG. 3 by the area expansion method. (A) to (b) show an example of a screen displayed on the display screen of the display device when the user selects a designated area. An example of a screen displayed on the display screen of the display device when performing image processing is shown. (A) to (c) are views explaining the conventional area expansion method. (A) to (e) are diagrams showing how an image is divided into two designated regions when two seeds are given by using a conventional region expansion method. It is a figure explaining the 1st range. (A)-(b) are diagrams showing a method of determining influence. It is a figure which showed the result of having made the judgment by the method based on the strength about the target pixel in the 1st range shown in FIG. 8-2. (A) to (h) are diagrams showing an example of a process of sequentially labeling by the region expansion method described with reference to FIGS. 8-3 to 8-4. It is a block diagram which shows the functional structure example of the area detection part in 1st Embodiment. It is a figure explaining the 1st local area and the 2nd local area. It is a figure which showed the case where the texture of the 1st local area and the texture of a 2nd local area are similar (the degree of similarity is small), and a peripheral pixel is determined to belong to the same designated area as a reference pixel. It is a figure which showed the case where the texture of the 1st local area and the texture of a 2nd local area are similar (the degree of similarity is small), and a peripheral pixel is determined to belong to the same designated area as a reference pixel. It is a flowchart explaining the operation of the area detection part in 1st Embodiment. It is a block diagram which shows the functional structure example of the area detection part in 2nd Embodiment. Represents a seeded image. (A) shows an example of dividing an image in which a seed is set. (B) indicates that the seeded image is divided into blocks consisting of 3 pixels × 3 pixels. (A) to (b) are diagrams for explaining the operation of the seed reflecting unit. (A) shows an example in which the first local region and the second local region are set in the seed reflection image. (B) shows that the first local region is composed of 3 pixels × 3 pixels = 9 pixels. (C) shows that the second local region is composed of 3 pixels × 3 pixels = 9 pixels. (A) is a diagram showing the influence of the reference pixel and its periphery in the conventional method. Further, (b) is a diagram showing the relationship of influence between the reference block and the surrounding blocks. In the seed reflection image of FIG. 18A, an example is shown in which the pixels in the second local region are included in the designated region. The pixel setting unit shows the case where one pixel in the center of the pixels is included in the designated area in the image. It is a flowchart explaining the operation of the area detection part in 2nd Embodiment. (A) is a figure which showed the case where the boundary of a region became rough when the region separation was performed. (B) is a figure which showed the case where the label near the boundary area and the unknown area which reset the strength were set. FIG. (C) is a diagram showing a case where a region separation result in which the boundary of the region becomes smooth is obtained. It shows the case where the seed 1 and the seed 2 are set in the image of the photograph including the floor reflected as the foreground and the background other than the floor. It is a figure which showed the clipping result with excessive detection and underdetection. It is a figure which showed the clipping result which was able to cut out a designated area more accurately. It is a figure which showed the hardware configuration of an image processing apparatus.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

<Explanation of the entire image processing system>
FIG. 1 is a diagram showing a configuration example of the image processing system 1 according to the present embodiment.
As shown in the figure, in the image processing system 1 of the present embodiment, the image processing device 10 that performs image processing on the image information of the image displayed on the display device 20 and the image information created by the image processing device 10 are input. A display device 20 that displays an image based on the image information, and an input device 30 for the user to input various information to the image processing device 10 are provided.

The image processing device 10 is, for example, a so-called general-purpose personal computer (PC). Then, the image processing device 10 is designed to create image information and the like by operating various application software under the control of the OS (Operating System).

The display device 20 displays an image on the display screen 21. The display device 20 is composed of, for example, a liquid crystal display for a PC, a liquid crystal television, a projector, or the like, which has a function of displaying an image by additive color mixing. Therefore, the display method in the display device 20 is not limited to the liquid crystal system. In the example shown in FIG. 1, the display screen 21 is provided in the display device 20, but when a projector is used as the display device 20, the display screen 21 is a screen or the like provided outside the display device 20. It becomes.

The input device 30 is composed of a keyboard, a mouse, and the like. The input device 30 starts and ends the application software for performing image processing, and as will be described in detail later, when performing image processing, the user inputs an instruction for performing image processing to the image processing device 10. Used for.

The image processing device 10 and the display device 20 are connected via a DVI (Digital Visual Interface). Instead of DVI, connection may be made via HDMI (registered trademark) (High-Definition Multimedia Interface), DisplayPort, or the like.
Further, the image processing device 10 and the input device 30 are connected via, for example, USB (Universal Serial Bus). Instead of USB, it may be connected via IEEE1394, RS-232C, or the like.

In such an image processing system 1, the display device 20 first displays an original image which is an image before image processing is performed. Then, when the user inputs an instruction for performing image processing to the image processing device 10 using the input device 30, the image processing device 10 performs image processing on the image information of the original image. The result of this image processing is reflected in the image displayed on the display device 20, and the image after the image processing is redrawn and displayed on the display device 20. In this case, the user can interactively perform the image processing while looking at the display device 20, and the image processing work can be performed more intuitively and more easily.

The image processing system 1 in the present embodiment is not limited to the embodiment shown in FIG. For example, a tablet terminal can be exemplified as the image processing system 1. In this case, the tablet terminal is provided with a touch panel, and the touch panel is used to display an image and input a user's instruction. That is, the touch panel functions as the display device 20 and the input device 30. Similarly, a touch monitor can be used as a device that integrates the display device 20 and the input device 30. This uses a touch panel as the display screen 21 of the display device 20. In this case, the image processing device 10 creates image information, and the image is displayed on the touch monitor based on the image information. Then, the user inputs an instruction for performing image processing by touching the touch monitor or the like.

<Explanation of image processing device>
FIG. 2 is a block diagram showing a functional configuration example of the image processing device 10 according to the present embodiment. Note that FIG. 2 shows a selection of various functions of the image processing apparatus 10 that are related to the present embodiment.
As shown in the figure, the image processing device 10 of the present embodiment includes an image information acquisition unit 11, a user instruction reception unit 12, an area detection unit 13, an area switching unit 14, an image processing unit 15, and an image information output. A unit 16 is provided.

The image information acquisition unit 11 acquires the image information of the image to be image-processed. That is, the image information acquisition unit 11 acquires the image information of the original image before the image processing is performed. This image information is, for example, RGB (Red, Green, Blue) video data (RGB data) for display on the display device 20.

The user instruction receiving unit 12 is an example of a position information acquisition unit, and receives an instruction by a user regarding image processing input by the input device 30.
Specifically, the user instruction receiving unit 12 receives an instruction to specify a designated area designated as a specific image area by the user from the images displayed on the display device 20 as user instruction information. This specific image area is, in this case, an image area in which the user performs image processing. Actually, in the present embodiment, the user instruction receiving unit 12 acquires the position information representing the representative position of the designated area input by the user as the user instruction information.
Further, as will be described in detail later, the user instruction receiving unit 12 receives as user instruction information an instruction for the user to select an image to be actually processed from the designated area. Further, the user instruction receiving unit 12 receives instructions regarding processing items, processing amounts, and the like for which the user performs image processing to the selected designated area as user instruction information. A more detailed explanation of these contents will be described later.

In the present embodiment, a method of performing the work of designating the designated area in a user-interactive manner as described below is adopted.
FIG. 3 is a diagram showing an example of a method of performing the work of designating a designated area user-interactively.
FIG. 3 shows a case where the image displayed on the display screen 21 of the display device 20 is an image G of a photograph including a person and a background reflected behind the person. Then, the case where the user selects the face part of the person as the foreground and the part other than the face as the background as the designated area is shown. That is, in this case, there are two designated areas. Hereinafter, the designated area of the face portion may be referred to as a "first designated area", and the designated area of the portion other than the face may be referred to as a "second designated area".

Then, the user gives a representative locus to each of the designated areas. This locus can be input by the input device 30. Specifically, when the input device 30 is a mouse, the mouse is operated to drag the image G displayed on the display screen 21 of the display device 20 to draw a locus. When the input device 30 is a touch panel, the trajectory is similarly drawn by tracing and swiping the image G with a user's finger, a touch pen, or the like. It may be given as a point instead of a locus. That is, the user may provide information indicating a representative position for each designated area such as a hair portion. This can be rephrased as the user inputting the position information representing the representative position of the designated area. Hereinafter, this trajectory, point, etc. may be referred to as a "seed".

In the example of FIG. 3, seeds are drawn on the face portion and the non-face portion, respectively. Hereinafter, the seed drawn on the face portion in the foreground may be referred to as "seed 1", and the seed drawn on the portion other than the face portion in the background may be referred to as "seed 2".

The area detection unit 13 detects a designated area from the image displayed on the display device 20 based on the seed position information. As will be described in detail later, the area detection unit 13 has a designated area based on the similarity between the texture around the pixel belonging to the designated area and the texture around the pixel to be determined whether or not it is included in the designated area. Is detected.

In order for the area detection unit 13 to cut out a designated area based on the seed position information, first, a label is added to the pixel at the place where the seed is drawn. In the example of FIG. 3, "label 1" is added to the pixel corresponding to the seed 1 drawn on the face portion, and "label 2" is added to the pixel corresponding to the seed 2 drawn on the portion other than the face. In the present embodiment, assigning a label in this way is referred to as "labeling".

4 (a) to 4 (c) show how a designated area is cut out from the image G shown in FIG. 3 by the area expansion method.
Of these, FIG. 4A is a diagram similar to FIG. 3, showing a state in which seed 1 and seed 2 are drawn as seeds in the image G.
Then, as shown in FIG. 4 (b), the area expands into the designated area from the place where the seed is drawn, and finally, as shown in FIG. 4 (c), there are two designated areas as designated areas. The designated area S1 of 1 and the second designated area S2 are cut out.

By adopting the above method, even if the designated area has a complicated shape, the user can cut out the designated area more intuitively and more easily.

The area switching unit 14 switches a plurality of designated areas. That is, when a plurality of designated areas exist, the user selects the designated area for which image adjustment is desired, and the area switching unit 14 switches the designated area accordingly.

5 (a) to 5 (b) show an example of a screen displayed on the display screen 21 of the display device 20 when the user selects a designated area.
In the examples shown in FIGS. 5A to 5B, the image G in a state where the designated area is selected is displayed on the left side of the display screen 21, and either the area 1 or the area 2 is displayed on the right side of the display screen 21. The radio buttons 212a and 212b to be selected are displayed. In this case, the area 1 corresponds to the first designated area S1, and the area 2 corresponds to the second designated area S2. Then, when the user selects the radio buttons 212a and 212b using the input device 30, the designated area is switched.

FIG. 5A shows a state in which the radio button 212a is selected, and the first designated area S1 which is an image area of the face portion is selected as the designated area. Then, when the user selects the radio button 212b, the designated area is switched to the second designated area S2, which is an image area other than the face portion, as shown in FIG. 5 (b).

Actually, the result of the operation described in FIG. 5 is acquired by the user instruction receiving unit 12 as the user instruction information, and the designated area is switched by the area switching unit 14.

The image processing unit 15 actually performs image processing on the selected designated area.
FIG. 6 shows an example of a screen displayed on the display screen 21 of the display device 20 when performing image processing.
Here, an example of adjusting the hue, saturation, and brightness with respect to the selected designated area is shown. In this example, the image G with the designated area selected is displayed on the upper left side of the display screen 21, and the radio buttons 212a and 212b for selecting either the area 1 or the area 2 are displayed on the upper right side of the display screen 21. Will be done. Here, 212a of the radio buttons is selected, and the first designated area S1, which is an image area of the face portion, is selected as the designated area. It should be noted that the designated area can be switched by operating the radio buttons 212a and 212b, as in the case of FIG.

Further, on the lower side of the display screen 21, a slide bar 213a for adjusting hue, saturation, and brightness, and a slider 213b are displayed. The slider 213b can be slid by moving left and right in the drawing on the slide bar 213a by operating the input device 30. The slider 213b is initially located at the center of the slide bar 213a, and represents the state before adjustment of hue, saturation, and brightness at this position.

Then, when the user slides the hue, saturation, or brightness slider 213b on the slide bar 213a to the left or right in the figure using the input device 30, image processing is performed on the selected designated area. The image G displayed on the display screen 21 also changes accordingly. In this case, when the slider 213b is slid to the right in the figure, image processing is performed to increase any of the corresponding hue, saturation, and brightness. On the other hand, when the slider 213b is slid to the left in the drawing, image processing is performed to reduce any of the corresponding hue, saturation, and brightness.

Returning to FIG. 2, the image information output unit 16 outputs the image information after the image processing is performed as described above. The image information after the image processing is performed is sent to the display device 20. Then, an image is displayed on the display device 20 based on this image information.

<Explanation of area detection unit>
Next, a method in which the area detection unit 13 cuts out a designated area by the area expansion method will be described in more detail.

Here, first, the conventional area expansion method will be described.
7 (a) to 7 (c) are diagrams illustrating a conventional region expansion method.
Of these, FIG. 7A is an original image, which is composed of a region of 3 × 3 = 9 pixels of 3 vertical pixels and 3 horizontal pixels. This original image is composed of two image areas. In FIG. 7A, these two image regions are represented by the difference in the color density of each pixel. It is assumed that the pixel values included in each image area show values close to each other.

Then, as shown in FIG. 7B, the seed 1 is given to the pixels located in the 2 rows and 1 column, and the seed 2 is given to the pixels located in the 1 row and 3 columns.

At this time, consider a case where it is determined whether or not the pixel located in the 2 rows and 2 columns, which is the central pixel, belongs to the designated area including the seed 1 or the designated area including the seed 2. Here, for the central pixel, the pixel value of the central pixel and the pixel value of the seed existing in the peripheral eight pixels in contact with the central pixel are compared. Then, if the pixel values are close, it is determined that the center pixel is included in the designated area including the seed. In this case, the peripheral 8 pixels include two seeds, a seed 1 and a seed 2, but the pixel value of the central pixel is closer to the pixel value of the seed 2 than the pixel value of the seed 1, so that the seed is used. It is determined that it belongs to the designated area including 1.

Then, as shown in FIG. 7 (c), the central pixel belongs to the region of seed 1. And the center pixel is now treated as a new seed. Then, in this case, the central pixel is labeled with the same "label 1" as the seed 1.

In the conventional area expansion method, a pixel in contact with the seed pixel is selected as a target pixel to be determined whether or not it is included in the specified area (the center pixel in the above example), and the pixel value of this target pixel is selected. And the pixel value of the seed included in the pixels of the eight pixels around the target pixel are compared. Then, the target pixel is labeled because it is considered to belong to the region included in the seed whose pixel value is close. The basic idea is to expand the area by repeating this.

8-1 (a) to 8-1 (e) are diagrams showing how an image is divided into two designated regions when two seeds are given by using a conventional region expansion method.
Here, as shown in FIG. 8-1 (b), two seeds, seed 1 and seed 2, are given to the original image of FIG. 8-1 (a). Then, the area is expanded based on each seed. In this case, as described above, the area can be expanded according to the proximity to the pixel values of the peripheral pixels in the original image. At this time, if there is a conflict between the regions as shown in FIG. 8-1 (c), the pixel is the target pixel for re-judgment, and which region is the relationship between the pixel value of the target pixel for re-judgment and the nearby pixel value. You just have to decide if it belongs to. At this time, the method described in the following document can be used.

V.Vezhnevets and V.Konouchine: "Grow-Cut" -Interactive Multi-label N-D Image Segmentation ", Proc.Graphicon.pp 150-156 (2005)

In the example of FIG. 8-1 (d), the target pixel for re-judgment is finally determined as the region of seed 2, and as shown in FIG. 8-1 (e), two seeds are used as the basis for the two seeds. It is divided into regions and converges. In this example, the case of dividing into two regions is shown, but it is also possible to give three or more seeds and divide into three or more regions.

As described above, in the conventional area expansion method, the target pixel is focused on, and the pixel value of the seed pixel in the peripheral eight pixels is compared with the pixel value of the target pixel to determine the designated area to which the target pixel belongs.

However, in the conventional area expansion method, it is necessary to select pixels one by one as target pixels and label them, which has a problem that the processing speed tends to be slow. In addition, the accuracy at the time of separation tends to be low in places where the area is complicated. In order to solve this problem, for example, there is also an area expansion method as described below.

In this area expansion method, a predetermined first range is set around the reference pixel which is a pixel of the seed 1 and the seed 2. Then, within this first range, the pixels other than the reference pixel are set as the target pixels, and the target pixels are set to the designated area including the seed 1 and the designated area including the seed 2 by using the “strength” of the pixels. Judge that it belongs.

FIG. 8-2 is a diagram illustrating the first range.
As shown in the figure, seed 1 and seed 2 which are reference pixels are selected for each of the two image regions. Further, the seed 1 and the seed 2 are positioned at the center, and the range of 5 pixels in the vertical direction and 5 pixels in the horizontal direction is set as the first range. In the figure, this range is displayed as the range within the thick line frame.

The "strength" is the strength belonging to the designated area corresponding to the label, and represents the magnitude of the possibility that a certain pixel belongs to the designated area corresponding to the label. The higher the strength, the more likely the pixel belongs to the designated area corresponding to the label, and the lower the strength, the less likely the pixel belongs to the designated area corresponding to the label. The strength is determined as follows.
First, the strength of the pixels included in the representative position first specified by the user is 1 as an initial value. That is, the pixels of the seed 1 and the seed 2 before expanding the area have a strength of 1. For pixels that have not yet been labeled, the strength is 0.

Then, consider the influence of the pixel given the strength on the surrounding pixels.
8-3 (a)-(b) are diagrams showing a method of determining influence. In FIG. 8-3 (a) ~ (b) , the horizontal axis represents the Euclidean distance _{d i,} and the vertical axis represents the influence.
The Euclidean distance d _i is the Euclidean distance d _i of the pixel value determined between the pixels located around the pixel and the pixel given strength. If the pixel value is represented by RGB values, the Euclidean distance d _i can be expressed as the following equation (1). In equation (1), the pixel value of the pixel given the strength _{(R O, G O, B} O) and then, the pixel values of pixels positioned around case of _{(R i, G i, B} i) and shows the Euclidean distances d _i.

The example defines a monotonically decreasing function of the nonlinear as shown in FIGS. 8-3 (a), with respect to the Euclidean distance d _i, a value determined by the monotonically decreasing function and influence.
That enough Euclidean distance d _i is small, influence becomes larger, as the Euclidean distance d _i is large, the influence becomes smaller.
The monotonous decreasing function is not limited to the one having the shape shown in FIG. 8-3 (a), and is not particularly limited as long as it is a monotonous decreasing function. Therefore, it may be a linear monotonous decrease function as shown in FIG. 8-3 (b). Also a linearly specific range of the Euclidean distance d _i, may be a monotonically decreasing function of the piecewise linear such that the non-linear in other ranges.

The strength of the pixel determined to belong to the designated region is obtained by multiplying the strength of the reference pixel by the influence. For example, if the strength of the reference pixel is 1 and the influence on the target pixel adjacent to the left side is 0.9, it is given when it is determined that the target pixel adjacent to the left side belongs to the specified area. The strength is 1 × 0.9 = 0.9. Further, for example, when the strength of the reference pixel is 1 and the influence on the target pixels adjacent to the left side of the two is 0.8, the strength given when it is determined that the target pixels belong to the designated area. The value is 1 × 0.8 = 0.8.

Using the above calculation method, it is also possible to make a judgment based on the strength given to the target pixel within the first range. At this time, if the target pixel does not have a label, it is determined that it is included in the designated area to which the reference pixel belongs, and if the target pixel already has a label for another designated area, the designated area having the larger strength is determined. Judged to be included in. In the former case, the same labeling as the reference pixel is performed. In the latter case, the stronger of the characteristics is labeled.

FIG. 8-4 shows the result of determining the target pixel in the first range shown in FIG. 8-2 by a method based on the strength.
The first range shown in FIG. 8-2 partially overlaps the seed 1 and the seed 2. And where the first range does not overlap, that is, where the seed 1 and the seed 2 do not conflict, in this case they are unlabeled and are all labeled the same as the reference pixel seed 1 or seed 2. To do. On the other hand, where the first range overlaps the seed 1 and the seed 2, that is, where they compete with each other, the stronger one is labeled. As a result, labeling is performed as shown in FIG. 8-4.

8-5 (a) to 8-5 (h) are diagrams showing an example of a process of sequentially labeling by the region expansion method described with reference to FIGS. 8-3 to 8-4.
FIG. 8-5 (a) shows a first range set for the seed pixels at this time. That is, the seed 1 and the seed 2 which are the reference pixels are selected. Further, the seed 1 and the seed 2 are positioned at the center, and the range of 3 vertical pixels × 3 horizontal pixels is set as the first range. In the figure, this first range is displayed as a range within the frame of the thick line.
In the present embodiment, starting from the seed 2 set at the position of 2 rows and 2 columns as shown in FIG. 8-5 (b), first, which designated area the target pixel in the first range belongs to belongs to. Judge whether or not. Then, as shown in FIGS. 8-5 (c) to 8-5, while moving the reference pixel one pixel at a time to the right side in the figure, it is determined which designated area the target pixel in the first range belongs to. I will do it. This judgment is made by the strength described above.
Then, after determining up to the right end in the figure as the target pixel, the reference pixel is then moved to the third column, and similarly, the reference pixel is moved to the right side in the figure one by one, and the target pixel within the first range is moved. It is determined which designated area it belongs to. Then, after determining the right end of the figure as the target pixel, the process moves to the next column. This is repeated as shown in FIGS. 8-5 (e) to 8-5 (g) until the reference pixel moves to the lower right end in the figure. That is, the judgment is made while moving the reference pixel so as to scan each pixel.

After the reference pixel reaches the lower right end and the pixel cannot be moved any more, the reference pixel is moved in the opposite direction to the above case, and the same processing is performed until the reference pixel moves to the upper left end. .. This means that the reference pixel has moved back and forth once. After that, the reciprocating movement of the reference pixel is repeated until it converges.

Finally, as shown in FIG. 8-5 (h), the first designated area and the second designated area are separated.

By expanding the area by the above method, it is possible to improve the speed and the accuracy of the clipping boundary. However, this method also expands the area based on the color distance between pixels. Therefore, even if the area is outside the designated area, the areas having similar colors may be incorrectly labeled and may be regarded as the same designated area.

Therefore, in the present embodiment, the area detection unit 13 has the following configuration, and even between pixels having similar colors, it is possible to suppress erroneous labeling and improve the accuracy of the clipping boundary.

[First Embodiment]
First, the first embodiment of the region detection unit 13 will be described.
FIG. 9 is a block diagram showing a functional configuration example of the area detection unit 13 according to the first embodiment.
As shown in the figure, the area detection unit 13 of the present embodiment includes the pixel selection unit 131, the local area setting unit 132, the similarity calculation unit 133, the determination unit 134, the characteristic change unit 135, and the convergence determination unit 136. And.

In the first embodiment, the pixel selection unit 131 selects a reference pixel from the pixels belonging to the designated area. Here, the "pixel belonging to the designated area" is, for example, a pixel included in a representative position designated by the user, that is, a pixel of the seed described above. It also includes pixels newly labeled by region expansion. Here, the pixel selection unit 131 selects one pixel from the pixels belonging to the designated area as the reference pixel.
Further, the pixel selection unit 131 selects peripheral pixels that are set with respect to the reference pixels and are targets for determining whether or not they are included in the designated area. Here, the pixel selection unit 131 selects one pixel from the pixels around the reference pixel as peripheral pixels. Peripheral pixels are selected from pixels that have not been seeded.

The local area setting unit 132 sets the first local area as a range for calculating the texture around the reference pixel. Further, the local area setting unit 132 sets a second local area as a range for calculating the texture around the peripheral pixels.

FIG. 10 is a diagram illustrating a first local region and a second local region.
In the illustrated example, the reference pixel is P ₁ and the peripheral pixels are P ₂ . And so as to be positioned around the reference pixel P _1, and a vertical 5 pixels × 5 horizontal pixel range of the first local region R _1. Similarly, the peripheral pixel P ₂ is located at the center, and the range of 5 vertical pixels × 5 horizontal pixels is defined as the second local region R ₂ . In the figure, these ranges are displayed as the ranges within the thick line frame.

The similarity calculation unit 133 calculates the similarity between the texture in the _first local region R _{1 around the reference pixel P 1} and the texture in the second local region R _{2 around the peripheral pixel P 2.} do. Here, the similarity of textures means that the impressions produced by the entire local region are similar. For example, if there is an image that includes a chair made of wood as well as a floor made of wood, the color is the same brown. However, the texture of these surfaces is different, and as a result, the impression that a person receives is different. In the present embodiment, the similarity of this impression is calculated as the similarity of the texture. More specifically, the similarity of textures can be extracted as the similarity of the features of all pixels in each local region. This feature amount is, for example, a pixel value of pixels included in a local region, an average value of pixel values, a dispersion of pixel values, an image frequency spectrum, a repeating pattern, and the like. As described above, in the local region R ₁ and in the local region R _2, the height difference between the brightness, the frequency of the luminance, rules of the luminance occurs, if more like similar pattern is seen in the chromaticity or the like, the texture similar Assuming that the degree is large, here, for example, the degree of similarity is calculated as follows.

Assuming that the similarity between the texture of the first local region R _{1 and} the texture of the second local region R ₂ is s, the similarity s can be calculated by, for example, the following equation (2). _{In Equation 2} , the coordinates of the pixel positions in the first local region R ₁ and the second local region R 2 are represented by (x, y). Then, the pixel value at the pixel position (x, y) in the first local region R _{1 is set to I 1} (x, y), and the _{pixel value at the pixel position (x, y) in the second local region R 2} is set. The _{degree of similarity s when I 2} (x, y) is obtained. Here, only the luminance component I is used as the pixel value.

In Equation 2, the similarity s is calculated by using the square of the difference in pixel values, but the similarity s is calculated by using the absolute value of the difference in pixel values as in Equation 3 below. You may. In addition, any index may be used as long as it represents the similarity of textures.

That is, the local region is considered as a multidimensional vector having the same dimension as the number of pixels (in this case, 5 × 5 = 25 dimensions), and the similarity s is the pixels of _{the first local region R 1} and the second local region R _2. It can also be thought of as a quantity based on the distance between two multidimensional vectors obtained from the values. Similarity s in Equation 2 and Equation 3 Equation also has a first local region R ₁ textures and the second local region R ₂ texture smaller value means that are similar. Further, the larger the value, the more the _{texture of the first local region R 1 and} the texture of the second local region R ₂ are not similar.

Here, when the color data of each pixel is RGB data, it is ^{converted into a luminance color difference space such as L *} a ^* b ^* data or YCbCr data, and the luminance components L ^* and Y are _{converted into I 1} (x, y). ) Or I ₂ (x, y). Further, the brightness component may be V of HSV data or the like. Further, the value is not limited to the luminance component, and may be a value representing chromaticity or saturation. For example, if it is HSV data, S or H may be used. Further, in the L ^* a ^* b ^* data, the value calculated by the following equation 4 may be used, and in the YCbCr data, the value calculated by the following equation 5 may be used. That is, any reference may be used as long as it represents the degree of brightness, chromaticity, and saturation, and this _{may be a pixel value of I 1} (x, y) or I ₂ (x, y).

_{The determination unit 134 determines whether or not the peripheral pixel P 2} is included in the designated region based on the similarity of the texture calculated by the similarity calculation unit 133.
For example, the determination unit 134 provides a threshold value θ, and when the similarity s is smaller than the threshold value θ as shown in Equation 6, determines that the peripheral pixel P ₂ belongs to the same designated area as the reference pixel P _1. When the similarity s is equal to or greater than the threshold value θ, it is determined that the peripheral pixel P ₂ does not belong to the same designated area as the reference pixel P _1.

Further, the threshold value θ is not limited to a fixed value, and may be changed according to the image. Further, the threshold value θ may be changed according to the positions of _{the reference pixel P 1} and the peripheral pixels P _{2 in the image.}

Alternatively, the determination unit 134 may make a determination by a method based on the above-mentioned strength. For example, since the reference pixel P ₁ is a pixel labeled by seeding or updating as described above, the strength is set to 1, and _{the texture of the first} local region R 1 and the second local region R _{2 are set.} from the similarity s between texture, calculates the influence in the manner described in Figure 8-3 (However, in this case, the horizontal axis represents not the Euclidean distance d _i, a similarity s). Then, the strength may be propagated to _{the peripheral pixels P 2} depending on the strength and influence of the reference pixel P _1. Assuming that the strength of the reference pixel P _{1 is u 1} and the influence of the reference pixel P ₁ on the peripheral pixel P _{2 is w 12} , the u ₁ w ₁₂ is the strength propagated to the peripheral pixel P _2. ..

Whether or not to label may be determined depending on whether or not the following equation 7 is satisfied. In the formula 6, the smaller the similarity, the more it belongs to the specified region, but in the case of the formula 7, since the judgment is made based on the strength, the strength u ₁ w ₁₂ exceeds the threshold value θ _w. This serves as a reference for the determination unit 134 to determine that the peripheral pixel P ₂ belongs to (labels) the same designated area as the reference pixel P _1. When the strength u ₁ w ₁₂ is equal to or less than the threshold value θ _w , the determination unit 134 determines that the peripheral pixel P ₂ does not belong to the same designated area as the reference pixel P _{1, and does not label the peripheral pixel P 2.} Further, the threshold value θ _w is not limited to a fixed value as described above, and may be changed according to the positions of _{the reference pixel P 1} and the peripheral pixels P _{2 in the image.}

In the case of FIG. 10, although it depends on the value of the threshold value θ, _{the texture of the first local region R 1 and} the texture of the second local region R ₂ are not similar (the degree of similarity s is large and the texture is propagated). The strength is small), and it is determined that the peripheral pixel P ₂ does not belong to the same designated area as the reference pixel P _1.

11 and 12 are similar to the texture of the first local region R _{1 and} the texture of the second local region R ₂ (similarity s is small), and the peripheral pixels P ₂ are the same as the reference pixels P _1. It is a figure which showed the case which is judged to belong to a designated area.
For example, in FIG. 12, the pixel values (in this case, the brightness values) of _{the reference pixel P 1} and the peripheral pixels P _{2 are significantly different, but the texture of the first local region R 1 and} the texture of the second local region R ₂ Is similar, so _{it is determined that the peripheral pixel P 2} belongs to the same designated area as the reference pixel P _1.

By operating the determination unit 134 as described above, there is an effect of automatically diffusing the seed with respect to the given seed.

The characteristic changing unit 135 changes the characteristic given to the _{peripheral pixel P 2.}
Here, the characteristic refers to the label and strength given to the pixel, and as described above, _{the strength of the reference pixel P 1} as a seed is 1, and the strength of the peripheral pixel P ₂ is. It becomes the propagated u ₁ w ₁₂ (here, for example, this is referred to as u ₂ ).
As described above, the label indicates which designated area the pixel belongs to. The pixel belonging to the designated area 1 is given the label 1, and the pixel belonging to the designated area 2 is given the label 2. Here, the label of the seed 1 is the label 1, and the label of the seed 2 is the label 2. Therefore, when the determination unit 134 determines that the pixel belongs to the designated area 1, the peripheral pixel P ₂ is labeled on the label 1. NS. If the determination unit 134 determines that the pixel belongs to the designated area 2, the peripheral pixel P ₂ is labeled on the label 2. Here, in the case of the equation 7 _{, the label was determined by the threshold value θ w} , but if the peripheral pixel P ₂ is not labeled, the label is used as it is as the reference pixel P ₁ and u ₁ w ₁₂ is used as it is. The strength u ₂ of the peripheral pixel P ₂ may be set. After that, if the strength propagated from others _{is greater than u 2} , the strength may be changed to the stronger strength and the label, and this may be repeated to converge.

The peripheral pixel P ₂ is sequentially selected for all pixels other than the _{reference pixel P 1} in the image, _{and it is determined whether or not the selected peripheral pixel P 2} belongs to the same designated area as the reference pixel P _1. Further, the reference pixel P ₁ is also sequentially selected for the pixels to which the seed is given. In this process, the pixel selection unit 131 sequentially selects the reference pixel P ₁ and the peripheral pixels P _2.

And this series of processing is repeated until it converges. That is, new the pixels which are determined to belong to the designated region 1 and designated region 2, the newly selected as the reference pixel P _1, the peripheral pixels P ₂ is further selected peripheral pixels P _2, the designated region 1 and designated It will be determined whether or not it belongs to the region 2. By repeating and updating this process, the area where the labeling characteristics are changed is sequentially expanded, and the designated area 1 and the designated area 2 can be cut out. According to this method, as described above, a pixel once labeled on a certain label may be changed to another label.

The convergence test unit 136 determines whether or not the above series of processes have converged.
Convergence determination unit 136 determines that convergence has occurred, for example, when there are no pixels whose labels are changed or when the strength does not change. It is also possible to set the maximum number of updates in advance and consider that the items have converged when the maximum number of updates is reached.

Next, the operation of the region detection unit 13 in the first embodiment will be described.
FIG. 13 is a flowchart illustrating the operation of the area detection unit 13 in the first embodiment.
First, the pixel selection unit 131 selects the reference pixel P ₁ from the pixels belonging to the designated area (step 101). Further, the pixel selection unit 131 selects _{the peripheral pixel P 2 which is set with respect to the reference pixel P 1} and is a target for determining whether or not it is included in the designated area (step 102).

Next, the local region setting unit 132 sets _{the first local region R 1} as a range for calculating the texture around the _{reference pixel P 1} (step 103). Further, the local area setting unit 132 sets _{the second local area R 2} as a range for calculating the texture around the _{peripheral pixel P 2} (step 104).

Then, the similarity calculation unit 133 determines the similarity between the texture in the _first local region R _{1 around the reference pixel P 1} and the texture in the second local region R _{2 around the peripheral pixel P 2.} Calculate (step 105). This can be calculated, for example, by the above-mentioned equations 1 and 2. When the determination unit 134 makes a judgment based on the strength, the strength propagated to the _{peripheral pixel P 2} is further calculated based on the strength and influence of the _{reference pixel P 1.}

_{Next, the determination unit 134 determines whether or not the peripheral pixel P 2} is included in the designated region based on the similarity or strength of the texture calculated by the similarity calculation unit 133 (step 106).
As a result, when it is determined that the peripheral pixel P ₂ is included in the designated area (Yes in step 106), the characteristic changing unit 135 changes the label as the characteristic given to the _{peripheral pixel P 2 (step 107).} .. When the determination unit 134 makes a determination based on the strength, the characteristic change unit 135 further changes the strength as a characteristic given to the _{peripheral pixel P 2.}

After step 107, and _{when it is determined that the peripheral pixel P 2} is not included in the designated area (No in step 106), the convergence test unit 136 determines whether or not the above series of processes have converged (step 106). 108).
As a result, when the processing converges (Yes in step 108), all the processing ends.
On the other hand, if the processing has not converged (No in step 108), the process returns to step 101, a new reference pixel P ₁ or peripheral pixel P ₂ is selected, and _{whether the peripheral pixel P 2} is included in the designated area until it converges. Whether or not it is judged.

[Second Embodiment]
Next, a second embodiment of the region detection unit 13 will be described.
FIG. 14 is a block diagram showing a functional configuration example of the area detection unit 13 in the second embodiment.
As shown in the figure, the area detection unit 13 of the present embodiment includes an image segmentation unit 137, a seed reflection unit 138, a local area setting unit 132, a similarity calculation unit 133, a determination unit 134, and a pixel setting unit 139. , A characteristic changing unit 135, and a convergence determination unit 136.
The area detection unit 13 shown in FIG. 14 does not have the pixel selection unit 131 as compared with the area detection unit 13 shown in FIG. On the other hand, an image segmentation unit 137, a seed reflection unit 138, and a pixel setting unit 139 are added.

The image segmentation unit 137 divides the seeded image into blocks.
FIG. 15 represents the seeded image G. Here, seed 1, seed 2, and seed 3 are set in the image G.
Further, FIG. 16A shows an example in which the seeded image G is divided.
Here, the seeded image G is divided into blocks B composed of 3 pixels × 3 pixels shown in FIG. 16 (b). Here, the block B composed of 3 pixels × 3 pixels = 9 pixels is set, but the present invention is not limited to this. For example, it may be n pixels × n pixels, n pixels × m pixels (n and m are integers of 1 or more, but at least one of n and m is 2 or more).

The seed reflection unit 138 is an example of the representative position reflection unit, and reflects the seed on the image in which the block B is one pixel.
17 (a) to 17 (b) are views explaining the operation of the seed reflecting unit 138.
Here, FIG. 17A is the same as FIG. 16A, and shows an example in which the seeded image G is divided into blocks B. Further, FIG. 17B shows _{a seed reflection image G s} in which the seed is reflected by assigning a seed to the block B including the pixel in which the block B is set as one pixel and the seed is set in the image G. ing. In this case, the case where the seed 1, the seed 2, and the seed 3 are reflected in the _{image G s is shown.} As a result, this seed reflected image _{G s,} FIG. 17 (a) similarly to seed 1, seed 2, seed 3 is set. In this case, the seed reflection image G _s shown in FIG. 17 (b) can be seen as a reduced version of the image G shown in FIG. 17 (a).

Local region setting unit 132 sets the first local region R ₁ as a range for calculating the texture in units of block B. This selects one pixel from among pixels seed seed reflected image G _s is reflected, this is the first local region R _1. Since one pixel of the seed reflection image G _s is a block B composed of 3 pixels × 3 pixels = 9 pixels in the image G, this can be regarded as a local region in the image G. Similarly, the local area setting unit 132 sets the second local area R ₂ as a range for calculating the texture with the block B as a unit. That is, further selects one pixel around the first one pixel selected as the local region R ₁ of the seed reflected image G _s. It is selected from the pixels seed has not been reflected in the seed reflected image G _s. Then, this is designated as the second local region R ₂ . Although one block B is selected here, a plurality of blocks B may be selected as the first local region R ₁ or the second local region R ₂ .

FIG. 18 (a) shows the seed reflected image G _s, an example of setting the first and local region R ₁ and the second local region R _2. The first local region R ₁ has one pixel in the seed reflection image G _s , but the first local region R ₁ has 3 pixels × 3 pixels in the image G as shown in FIG. 18 (b). It consists of 9 pixels. Similarly, the second local region R ₂ is composed of 3 pixels × 3 pixels = 9 pixels in the image G as shown in FIG. 18 (c).

_{The similarity calculation unit 133 calculates the similarity between the texture of the first local region R 1 and} the texture of the second local region R ₂ by using the pixel value of the image G.

_{The determination unit 134 determines whether or not the pixels in the second local region R 2} are included in the designated region based on the similarity of the texture calculated by the similarity calculation unit 133. If it is included in the designated area, the label of the designated area represented by the seed included in the designated area is given. This determination may be based on the same idea as in the case of the first embodiment, and can be made based on the similarity s. Further, the strength may be calculated from the similarity s and judged based on this strength, or labeling can be performed in the same manner as in the first embodiment.

In the second embodiment, the point is to block the pixels and use the relationship in block B units, that is, to use the relationship (closeness) between the reference block B and its peripheral blocks B. ..

FIG. 19A shows the influence of the reference pixel P ₁ and its peripheral pixels P ₂ in the conventional method, and the thicker the line, the closer the pixel color is and the greater the influence. This influence is described in FIG. 8-3 above. Then, in the same manner as in FIG. 19A, in the present embodiment, this can be expanded in the case of block B, for example, if one pixel in FIG. 19A is replaced with one block of 3 × 3. The relationship of influence between the reference block B ₁ and the surrounding block B ₂ is shown in FIG. 19 (b). Here, the thicker the line, the greater the influence between _{block B 1} and block B _2.

In the first embodiment, it is not necessary that the blocks B are connected to each other, but in the second embodiment, as shown in FIG. 19B, one block B is regarded as one pixel, and the influence is measured. This is an example of introduction. As can be seen from FIG. 19B, the relationship between _{the luminance generation feature of the central reference block B 1} and the peripheral block B ₂ is located in the right column of the _{central reference block B 1.} (equivalent to influential) and three blocks B _2, with one block B ₂ immediately below, since the texture is close to that shown by the bold line of. In addition, the other blocks B ₂ are shown by thin lines because they are not similar (corresponding to a small influence). Further, although an example in which the first local region R ₁ and the second local region R ₂ are composed of one block B has been described here, the first local region R ₁ and the second local region R ₂ may be described. When it is composed of a plurality of blocks B, the relationship between the plurality of blocks B is established. That is, there is a relationship between the first local region R ₁ and the second local region R _2.

Assuming that the strength of the block B when the block B includes the pixels of the seed is 1, the method of spreading the seed and the method of updating the strength can be performed in the same manner as in the case of the first embodiment. Further, if the block B is regarded as a pixel, labeling and clipping of a specific area can be performed with the same idea as in FIG. 8-5. In this case, since the strength is propagated based not only on the color closeness of one pixel but also on the influence, it is possible to perform labeling close to that of a person judging an area.

If the influence shown in FIG. 19B is calculated first for all blocks B after blocking as shown in FIG. 17, this influence does not change, so that the entire processing is performed. It can also be done at high speed.

Figure 20 is the seed reflected image G _s in FIG. 18 (a), shows an example in which a pixel of the second in local regions R ₂ are included in the designated area. In this case, the second local region R ₂ indicates that the seed 2 is now included in the designated region to which the seed 2 belongs.

When the pixel setting unit 139 _{detects that the pixels in the second} local area R 2 are included in the designated area due to the similarity, which pixel in the second local area R ₂ is included in the designated area. To set.
That is, when the determination unit 134 determines that _{the pixels in the second local region R 2} are included in the designated region, the pixels in the second local region R ₂ are 3 pixels × 3 pixels = 9 pixels. Therefore, which of these pixels should be included in the designated area becomes a problem. In this case, any one of the nine pixels may be selected, or a plurality of pixels may be selected. For example, it may be one pixel at the center of the nine pixels, or one or more pixels may be randomly selected from the nine pixels. Further, all 9 pixels may be included in the designated area.

FIG. 21 shows a case where the pixel setting unit 139 assumes that one pixel in the center of _{the second local region R 2 is included in the designated region in the image G.} Further, in this case, it is shown that three pixels are included in the seed 2.

The subsequent operations of the characteristic changing unit 135 and the convergence determination unit 136 are the same as those in the first embodiment. Similar to the first embodiment, if it can be diffused as a seed, or if it is regarded as an image in which one block B is one pixel as shown in FIG. 17 (b), it is a conventional image as shown in FIG. 8-5. If it is carried out by the propagation method, more accurate clipping and region separation can be performed.

Next, the operation of the area detection unit 13 in the second embodiment will be described.
FIG. 22 is a flowchart illustrating the operation of the area detection unit 13 in the second embodiment.
First, the image segmentation unit 137 divides the seeded image into each block B (step 201).

Next, the seed reflecting unit 138 reflects the seed on the image in which the block B is one pixel (step 202). This can be done by the method described in FIG.

The local region setting unit 132, a local region R ₁ range as the first to calculate the texture in units of block B, and set among the pixels seed seed reflected image G _s is reflected (step 203 ). Furthermore the local region setting unit 132 sets among the pixels of the second local region R ₂ seed seed reflected image G _s as a range for calculating the texture in units of block B was not reflected (step 204 ).

Next, the similarity calculation unit 133 calculates the similarity _{between the texture of the first local region R 1 and} the texture of the second local region R ₂ using the pixel values of the image G (step 205). When the judgment unit 134 makes a judgment based on the strength, the strength propagated to the _{surrounding block B 2} is further calculated based on the strength and influence of the _{reference block B 1.}

_{Further, the determination unit 134 determines whether or not the pixels in the second local region R 2} are included in the designated region based on the similarity or strength of the texture calculated by the similarity calculation unit 133 (step 206).

As a result, when it is determined that the pixels in the second local area R ₂ are included in the designated area (Yes in step 206), the pixel setting unit 139 determines which pixel in the second local area R ₂ is included. It is set whether it is included in the designated area (step 207). Then, the characteristic changing unit 135 changes the label as a characteristic given to the pixel set by the pixel setting unit 139 (step 208). When the determination unit 134 makes a determination based on the strength, the characteristic change unit 135 further changes the strength as a characteristic given to the pixel set by the pixel setting unit 139.

After step 208, and if the second pixel in the local region R ₂ is determined not to be included in the specified area (No in step 206), the convergence determination unit 136, whether the series of processing has converged not (Step 209).
As a result, when the processing converges (Yes in step 209), all the processing ends.
When the process for has not converged (No in step 209), returns to step 202, new first set local regions R ₁ and the second local region R _2, the second local region to converge It is determined whether or not the pixels in _{R 2 are included in the designated area.}

There are, for example, two ways of using the second embodiment. The first is to use it for the purpose of diffusing the seed as shown in FIG. 21, and the diffused state is regarded as convergence and is temporarily terminated. After that, the block B is released, and this time, it is considered as an image G having the original pixels, and the regions are separated by propagation of the intensity as shown in FIGS. 8-2 to 8-5.
Secondly, after blocking as shown in FIG. 17B, block B is regarded as a pixel, and in this case, strength comparison is performed rather than seed diffusion that depends only on judgment based on the threshold value. This is a method in which propagation is performed to converge according to the update rule, and the converged result is used as the result of region separation. This may be used when the image G is large.

[Third Embodiment]
In the second embodiment, when not only the seed diffusion but also the region separation is performed by the convergence method based on the update of the strength, depending on the size of the image G and the size of the block B, FIG. 23 (a) shows. As shown, the boundaries of the area can be rough.

In this case, as shown in FIG. 23 (b), an unknown region M in which the label and strength near the boundary region are reset is set. Then, the block B is released, the structure is returned to the pixel unit structure, and the pixels labeled other than the unknown region M are used as seeds. Further, when the strength is propagated by the region expansion method shown in FIG. 8-5 as in the past by using the influence of each pixel shown in FIG. 19 (a), it is as shown in FIG. 23 (c). It is possible to obtain the result of region separation in which the boundaries of various regions are smoothed.
This is because the area detection unit 13 cancels the setting of the designated area in the boundary area of the designated area with the block B as the unit after detecting the designated area with the block B as the unit, and the setting of the designated area is not canceled. It can also be said that the designated area is detected again in the boundary area of the designated area based on the reference pixel selected from the pixels.

The advantage of using such a method is that it is equivalent to processing the image G using a reduced image, and thus is faster. Also, by comparing the similarity of textures compared to simply comparing the closeness of colors, the amount of information when calculating the influence increases, so it is possible to perform region separation with higher accuracy than before. ..

The designated area can be cut out according to the first to third embodiments described above.
FIG. 24 shows a case where the seed 1 and the seed 2 are set in the image G of the photograph including the floor reflected as the foreground and the background other than the floor.
In the present embodiment, the clipping result (result with excessive detection or insufficient detection) as shown in FIG. 25 is unlikely to occur, and the first designated area S1 and the second designated area S2 are further formed as shown in FIG. It can be cut out with high accuracy.

The seed may be diffused only in the foreground or only in the background. It may also be done in both the foreground and the background. In the case of performing in both the foreground and the background, if the similarity s is both equal to or less than the threshold value θ, a judgment such as selecting the smaller one may be made.
Further, in the above-described example _{, the number of pixels included in the first local region R 1} and the second local region R ₂ is the same, but may be different.

The processes performed by the area detection unit 13 described above include a position information acquisition step of acquiring position information representing a representative position (seed) of the designated area in the image, and an area detection step of detecting the designated area from the position information. , And the area detection step detects the designated area based on the similarity between the texture around the pixel belonging to the designated area and the texture around the pixel to be determined whether or not it is included in the designated area. It can also be regarded as an image processing method characterized by this.

<Hardware configuration example of image processing device>
Next, the hardware configuration of the image processing device 10 will be described.
FIG. 27 is a diagram showing a hardware configuration of the image processing device 10.
The image processing device 10 is realized by a personal computer or the like as described above. As shown in the figure, the image processing device 10 includes a CPU (Central Processing Unit) 91 as a calculation means, a main memory 92 as a storage means, and an HDD (Hard Disk Drive) 93. Here, the CPU 91 executes various programs such as an OS (Operating System) and application software. The main memory 92 is a storage area for storing various programs and data used for executing the various programs, and the HDD 93 is a storage area for storing input data for various programs and output data from various programs.
Further, the image processing device 10 includes a communication interface (hereinafter, referred to as “communication I / F”) 94 for communicating with the outside.

<Program description>
The process performed by the image processing device 10 in the present embodiment described above is prepared as, for example, a program such as application software.

Therefore, in the present embodiment, the processing performed by the image processing apparatus 10 is a position information acquisition function for acquiring the position information representing the representative position (seed) of the designated area in the image, and detecting the designated area from the position information. The area detection function is realized, and the area detection function is based on the similarity between the texture around the pixel belonging to the specified area and the texture around the pixel to be determined whether or not it is included in the specified area. It can also be regarded as a program that detects a specified area.

The program that realizes this embodiment can be provided not only by communication means but also by storing it in a recording medium such as a CD-ROM.

Although the present embodiment has been described above, the technical scope of the present invention is not limited to the scope described in the above embodiment. It is clear from the description of the claims that the above-described embodiment with various modifications or improvements is also included in the technical scope of the present invention.

1 ... Image processing system, 10 ... Image processing device, 11 ... Image information acquisition unit, 12 ... User instruction reception unit, 13 ... Area detection unit, 14 ... Area switching unit, 15 ... Image processing unit, 16 ... Image information output unit , 20 ... Display device, 30 ... Input device, 131 ... Pixel selection unit, 132 ... Local area setting unit, 133 ... Similarity calculation unit, 134 ... Judgment unit, 135 ... Characteristic change unit, 136 ... Convergence judgment unit, 137 ... Image segmentation part, 138 ... Seed reflection part

Claims (4)

A position information acquisition unit that acquires position information that represents the representative position of the specified area in the image,
An area detection unit that detects a specified area from location information,
With
The area detection unit
The designated area is detected based on the similarity between the texture around the pixel belonging to the designated area and the texture around the pixel to be determined whether or not it is included in the designated area.
An image segmentation portion that divides the image in which the representative position is set into blocks, and a local region that sets a first local region and a second local region as a range for calculating a texture in units of the blocks. An image processing device including a setting unit and a similarity calculation unit for calculating the similarity between the texture in the first local region and the texture in the second local region. Pixel setting for setting which pixel in the second local region is included in the designated region when it is detected that the pixel in the second local region is included in the designated region by the similarity. The image processing apparatus according to claim 1 , further comprising a unit. The similarity calculation unit further calculates the strength with which the second local region belongs to the designated region based on the similarity.
The first or second claim, wherein the designated region is detected by using the strength to determine whether or not the pixels in the second local region belong to the designated region. Image processing device. After detecting the designated area in units of the block, the area detection unit cancels the setting of the designated area in the boundary area of the designated area in the block as a unit, and the pixel in which the setting of the designated area is not canceled. The image processing apparatus according to claim 1 , wherein the designated area is detected again in the boundary area of the designated area based on the reference pixel selected from the above.

Images