JP2001043388A - Method and device for processing image and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform area dividing processing at high speed by reducing storage areas and a calculation quantity by decreasing the amount of data to be processed by updating processing by updating not all applied cells but only a cell satisfying prescribed conditions. SOLUTION: Concerning all the cells, whether they are brought into contact with the border line of areas or not is investigated and only the cells brought into contact with the border line of areas are registered on a processing list. Otherwise, it is searched whether the state value of a cell registered on the existent processing list is updated or not and the cells of the updated states are deleted from the list. Concerning the deleted cells and nearby cells, whether they are brought into contact with the border line of areas or not is investigated and only the cells brought into contact with the border line of areas are registered on the list. The probability for the cell registered on the list to belong to each class is calculated and the valve of the class with the maximum probability is defined as the state value of that cell. The state values of the cell before and after state updating are compared and which percentage of state values of cells is changed is found as a convergence degree.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] The present invention relates to an image processing method, an image processing apparatus, and a storage medium.

[0002]

2. Description of the Related Art In image processing and image recognition, extraction and determination of an object from a still image is a fundamental and important problem. Object extraction is used in a wide range of fields such as similar image retrieval using objects in still images, face recognition, data description in MPEG4, and human interface. The region division is an indispensable function as a pre-process of object extraction.

[0003] The region dividing process of a still image is a process of dividing a still image according to a certain standard. A method for realizing region division of a still image can be roughly classified into two. one,
This method focuses on the discontinuity of a region on a still image. Another method focuses on the continuity of regions on a still image.

An example of the former is an edge detection method. The edge detection method refers to a method of detecting an edge of an image and separating the image by the edge. The edge detection processing has an advantage that the calculation cost is low. However, the edge detection method has a disadvantage that there is no guarantee that an edge constitutes a closed region. As a countermeasure against this problem, there has been proposed a method of performing division by performing edge interpolation processing.

[0005] An example of the latter is region growing. Region growing is a method of examining the characteristics of a seed small area and expanding the area by taking in pixels having characteristics close to those of the small area among the pixels surrounding the small area. Is formed. The method that focuses on the continuity of the region is relatively widely used because it has a feature that a closed region can always be obtained.

A problem that arises when performing area division is the influence of noise. For example, when the edge detection method is used, it may be considered that an edge cannot be normally detected due to the influence of noise. Also, the division processing by region growing may fall into a local solution.

In order to reduce the influence of such noise, a Markov random field (MRF; Markov Randam
Field) is often used in the area division processing. MR
Since F includes a process for noise as a model, it is also applied to the restoration of an image to which noise has been added.

[0008]

However, the area dividing method using MRF has a problem that a huge storage area and a large amount of calculation are required. This is because the calculation of the probability that each pixel belongs to each region is performed for each pixel, and the state value update process of determining the state values of all pixels is performed by reducing the number of pixels whose state values are updated and reducing the number of pixels. It is necessary to repeat until the condition is stabilized.

An object of the present invention is to provide an image processing method, an image processing device, and a storage medium for solving the above problems.

Further, according to the present invention, in a region division process accompanied by an update process using an arbitrary probability density function, the amount of data to be processed in the update process is reduced to reduce the storage area and the amount of calculation, and the region division process is performed at high speed. The purpose is to do.

[0011]

In order to achieve the above object, the present invention provides a setting step of setting an initial state value of a cell constituted by image information corresponding to an image, and setting a state value of the cell to a predetermined value. A state updating step of updating using an evaluation function;
A merging step of merging the cells based on the updated state values of the cells and dividing the image into regions, wherein the updating step satisfies a predetermined condition instead of all the given cells. It is characterized in that a cell is updated.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS (First Embodiment) This embodiment will be described with reference to the drawings.

FIG. 1 is a flowchart showing the configuration of the first embodiment. Step S101 is pre-processing. Fig. 7
Illustrates a method for generating a cell.

Step S101 is a pre-process in which a set of pixels is generated by creating a quadtree according to the conditions given by the user. Here, for the sake of simplicity, a quadtree is used in the present embodiment, but the created tree may have an arbitrary number of branches. The set of pixels generated in this way is treated as a cell. Finally, the initial state value of the cell is set.

Step S102 is a state value updating process. A class is a set of cells having similar features.

In step S102, the probability that the cell belongs to the class is calculated, and the state value of the cell is updated.

In step S103, the number of cells whose state values have changed by the update processing is checked, and convergence is determined. If it is determined that the cell state value has converged,
Proceed to S104. If it is determined that the convergence has not occurred, the process returns to step S102.

In step S104, a unique area number is assigned to each area in consideration of the cell state value and the continuity of space.

In S105, an area merging process is performed. As the area merging processing, there are a method of merging based on the closeness of the average colors of adjacent areas, and a method of merging based on the amount of color change at the boundary between the areas.

FIG. 2 is a flowchart showing the configuration of the pre-processing S101 in FIG.

In step S101, a digital image is input, a pixel or a group of pixels is made to correspond to a cell, and the initial state of the cell and the class is set.

A cell has a class number to which the cell belongs as a state value. The initial value of the state value may be determined in any way. Here, as an example, the initial value of the state value is determined by a natural random number having an upper limit of the number specified by the user.

The generation of the quadtree is performed by defining a division condition for an input image and recursively dividing a partial image satisfying the division condition. For example, in the present embodiment, the following two criteria are used as division conditions. 1. Luminance difference in partial image 2. Variance of color in partial image (value of determinant of variance-covariance matrix of color vector)

In step S201, a threshold value d relating to the division condition is obtained. For example, in the present embodiment, the threshold value d is represented by the following equation using the weight Tdiv given by the user and the maximum value Fall of the luminance differences of all the pixels included in the original image.

[0025]

[Outside 1]

In step S202, a division parameter is calculated for the partial image. The initial state of the partial image is the entire still image. For example, in the present embodiment, the division condition is “division parameter> threshold d”.

In step S203, it is determined whether there is a partial image satisfying the division condition. If there is a partial image that satisfies the division condition, the process proceeds to step S204. If there is no partial image that satisfies the division condition, the process ends.

In this embodiment, the division condition is determined by a threshold value related to a certain reference, but the division condition may be determined by another condition.

In step S204, the partial image satisfying the condition D is divided into two equal parts in the x direction and two equal parts in the y direction to divide the image into four equal parts. If the size of the partial image in the x direction is 1 pixel, division in the x direction is not performed. If the size of the partial image in the y direction is 1 pixel, the division in the y direction is not performed.

The state update process S102 includes two processes, a first stage for creating a list of cells to be processed and a second stage for updating the status of cells registered in the list.

The flow of the process in step S102 will be described with reference to the flowchart shown in FIG.

First stage: In the present embodiment, the updating process can be executed at higher speed by processing only the cells that are in contact with the boundary of the area. FIG. 8 illustrates cells processed in step S102 and cells not processed. In the first stage, a list of cells to be processed is created. In the former stage, in step S301, 1. When the state update processing with high speed has not been performed at all 2. When the state update processing with high speed has been performed once or more and there is already a list of cells in contact with the boundary, Divided into two streams.

In the first case, steps S302 and S30
In step 3, it is checked whether or not all the cells are in contact with the boundary of the area, and only the cells in contact with the boundary of the area are registered in the processing list, and the process proceeds to the subsequent stage.

In the second case, in step S304, it is checked whether or not the state value of the cell registered in the existing processing list has been updated. Next, in step S305, the cell whose state has been updated is deleted from the list.

Finally, in step S306, it is checked whether or not the deleted cell and its neighboring cells are in contact with the boundary of the region, and only the cells that are in contact with the boundary of the region are registered in the list. Proceed to the next stage. For example, in the present embodiment, the neighborhood is a set of cells adjacent to the cell of interest. FIG. 5 illustrates the vicinity of the cell of interest.

Subsequent stage: In the subsequent stage, as one of the methods for realizing the region division of the still image using the MRF, the region division by the Heat Bath method is performed.

The Heat Bath method is a method for performing processing according to the following procedure. 1. For all cells: (a) Calculate the probability that a cell belongs to each class. (b) The class to which the cell belongs is determined according to the obtained probability. For example, in the present embodiment, the class with the highest probability is selected. 2. Update the class parameters. For example, in the present embodiment, the parameters are the average color of the cells belonging to the class, the variance of the color, and the inverse matrix of the variance-covariance matrix. 3. If the cell condition is not stable, return to 1. For example, in the present embodiment, a cell whose class has changed is examined, and if the ratio of the number of cells whose class has changed to the entire cell is larger than a certain ratio, it is determined that the state of the cell is not stable.

Here, the initial state of the cell may be determined by random numbers. The number of classes may be arbitrary. Step S307,
In step S308, the probability P (xc | yc) belonging to each class of the cell registered in the processing list is calculated, and the value of the class having the highest probability is set as the state value of the cell. A clique which is a group of pixels referred to for determining a state is, for example, a set of a cell of interest and all cells adjacent to the cell in the present embodiment. FIG. 6 illustrates a clique according to the present embodiment.

Assuming that c is a clique, x is an element of state X, and y is a color vector of a pixel, the probability P (xc | yc) is expressed by the following equation.

[0040]

[Outside 2] And In the present embodiment, an arbitrary space such as an RGB space L ^* a ^* b ^* space may be used as the color space of the color vector yij of the pixel. The edge in equation (8) is, for example, Sobel
Any edge detection method such as edge detection using a filter may be used.

Here, the meaning of the equation (2) will be briefly described. Equation (2) is composed of equations (3) and (5), equations (4) and (6), and Pyc-1. Equation (5) represents the distance in the color space between the average color of the cell and the average color of the class. However, this distance is normalized by the variance of the color of the class (Maharanobis distance).

Equation (6) is a part for calculating the continuity of the area by MRF. Λ and H in the equation (6) are respectively the equations (7) and
This is shown in (8).

Β is a parameter for determining the weight of the continuity of the area with respect to the Mahalanobis distance in the color space. Equation (7) is an equation that is set such that the higher the ratio of the state value of the surrounding cell to the state value of the target cell is, the higher the value of the entire equation (2) becomes.

Further, by introducing a line process for limiting the clique according to the equation (8), the edge is taken into account instead of simply considering the continuity of the area. formula
Equations (3) and (4) are distribution functions that normalize equations (5) and (6), respectively, and are used to calculate equation (2) as a probability. Pyc is always 1 because it is the pixel appearance probability.

The line process will be described with reference to FIG. The line process is a process of excluding, from the cells included in the clique, cells satisfying a certain condition from the calculation of the MRF. For example, in the present embodiment, a cell that satisfies a certain condition refers to a cell that is separated from a target cell by an edge. In determining the state of cell R in FIG.
It is assumed that the state values of cells A, B, C, and D, which are cells included in the clique, are 1, 2, 3, and 3, respectively. Also, cell R
It is assumed that an edge has been detected between cell C and cell R and cell D. Here, the value of β in equation (6) is very large,
If the continuity of the area is prioritized over the closeness of the colors, and if the line process is not introduced, the state value of the cell R is
It is 3 despite being separated by edges. On the other hand, cell C and cell D separated by edge
Is excluded from the calculation of the MRF of the equation (6) by the line process, so that an appropriate state value can be set for the cell R.

For example, the color of cell R is higher than the color of cell B in cell A.
When the color is close to the color of, the state value of the cell R becomes 1 as shown in FIG.

Next, in step S309, the state values of the cells before and after the state update are compared, and the percentage of the state value of the cell that has changed is determined as the degree of convergence.

The update processing step S102 is repeatedly executed until the degree of convergence exceeds a predetermined value in step S103 of FIG.

FIG. 4 is a flowchart showing the structure of the area number setting process S104. Step S104 is equivalent to step S1
For the cells classified by 02, a process of dividing the cells into regions in consideration of spatial continuity is performed.

In step S104, when determining the area number of a certain cell, the state values of the surrounding cells are referred to. The processing differs depending on whether the state value of the surrounding cells matches the state value of the cell of interest or not.

If the state values do not match: If the surrounding cells do not match the state values, a new area number is prepared in step S403 and set as the cell area number.

When the state values match: When all the area numbers of surrounding cells having the same state value as the cell of interest match, the area numbers of the cell of interest also match (FIG. 9). . If they do not match, match the area number
(FIG. 10). As the area number of the cell of interest, the area number after matching is set.

In this embodiment, since the cells belonging to the area are stored in a list structure, the matching of the area numbers in the area number setting and the merging processing in the area merging need only be performed by reconnecting the lists. Has become.

As described above, according to the first embodiment, a set of pixels is made to correspond to a cell, and only the cells in contact with the boundary of the area are processed without performing the update processing for all the pixels. It is possible to perform a digital image area dividing process at high speed.

(Second Embodiment) In the first embodiment, a method for processing a region of a digital image using MRF at high speed has been described. In the second embodiment, a method will be described in which a region of an image forming a frame of a moving image is divided at high speed using a motion vector calculated based on a difference between frames of the moving image. Note that the flow of all processes in the second embodiment is the same as in the first embodiment, and a description thereof will not be repeated.

In this embodiment, a method of dividing an area using only a motion vector will be described. In the case of dividing an area by a motion vector of a moving image, an image forming a frame is used, and color information of the image is used. Is also good.

In the area division using only the motion vector, it is not possible to perform the processing using the color-specific features as seen in the first embodiment. Therefore, preprocessing S1
In 01, an n-ary tree is generated by setting a threshold value d and defining a division condition with reference to a determinant of a variance-covariance matrix of a motion vector, and is associated with a cell.

In the update processing S102, the state value of the cell is updated by the same processing. In the present embodiment, equations (2), (4),
Y in equation (5) corresponds to the motion vector. The edge in equation (8) can be detected by calculating the amount of change in the motion vector.

The area number setting processing S104 is the same as in the first embodiment.

As described above, according to the second embodiment, it is possible to perform high-speed region segmentation of a moving image frame by MRF using a motion vector calculated based on a difference between frames of the moving image. Will be possible.

(Third Embodiment) In the first embodiment, the process of updating the state value of a cell is performed using a probability density function using an MRF. The update processing of the state value of the cell may be performed by using this.

(Other Embodiments) The functions of the above-described embodiment (for example, a computer connected to an apparatus or a system connected to the various devices so as to operate the various devices so as to realize the functions of the above-described embodiments) are provided. , A program code of software for realizing the functions realized by the flowcharts of FIGS. 2, 3, and 4; and a computer (CPU or MPU) of the system or the apparatus. What was implemented by operating is also included in the category of the present invention.

In this case, the program code itself of the software realizes the functions of the above-described embodiment, and the program code itself and means for supplying the program code to the computer, for example, the program code Is provided.

As a storage medium for storing such a program code, for example, a floppy disk, hard disk, optical disk, magneto-optical disk, CD-ROM, magnetic tape, nonvolatile memory card, ROM or the like can be used.

When the computer executes the supplied program code, not only the functions of the above-described embodiment are realized, but also the OS (operating system) running on the computer or another program. Needless to say, the program code is included in the present embodiment even when the functions of the above-described embodiment are realized in cooperation with application software and the like.

Further, the supplied program code is stored in a memory provided in a function expansion board of the computer or a function expansion unit connected to the computer, and then stored in the function expansion board or the function storage unit based on the instruction of the program code. It is needless to say that the present embodiment also includes a case where a provided CPU or the like performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.

A plurality of devices (for example, host computer, interface device, reader, printer, etc.)
Or a device composed of one device (for example, a copying machine, a facsimile machine, etc.).

[0068]

According to the first aspect, as described above,
In the area division processing accompanied by the update processing using the predetermined evaluation function, the area division processing can be performed at high speed by reducing the number of cells to be processed in the update processing.

The same effects can be obtained with the image processing device according to the thirteenth aspect and the storage medium according to the fourteenth aspect.

According to the second aspect of the present invention, in the updating process, not all the cells but the cells constituting the boundary of the region are processed, so that the region dividing process can be performed at high speed.

According to the third aspect, it is possible to perform the area division processing accompanied by the update processing using an arbitrary probability density function at a high speed.

According to the fourth aspect, it is possible to perform the region division processing accompanied by the update processing using the probability density function of the MRF at a high speed.

According to the seventh aspect, since the cells are generated by recursively dividing the image, the number of cells to be processed can be reduced. As a result, the area dividing process can be performed at high speed.

According to the eighth aspect, when the digital signal is a vector calculated based on a difference between frames of a digital moving image, a region dividing process of a moving image frame according to a motion vector can be performed.

According to the ninth aspect, when the digital signal is a pixel value, it is possible to perform a region dividing process of the digital signal according to the color represented by the pixel value.

According to the tenth aspect, the cell can be easily generated by obtaining the division condition based on the determinant of the variance-covariance value of the digital signal.

According to the eleventh aspect, when the digital signal is a pixel value, a cell can be generated according to the luminance information by forming the cell using the luminance difference.

According to the twelfth aspect, when the digital signal is a pixel value, the cell is formed using the determinant of the variance-covariance matrix of the color vector, thereby forming the variance-covariance matrix of the color vector. A cell can be generated according to the value of the determinant.

[Brief description of the drawings]

FIG. 1 is a flowchart illustrating a flow of a process according to an embodiment.

FIG. 2 is a flowchart illustrating a flow of a pre-processing step S101 according to the embodiment.

FIG. 3 is a flowchart showing a flow of a process of a status update process S102 in the embodiment.

FIG. 4 is a flowchart showing a flow of a region number setting process S104 in the embodiment.

FIG. 5 is a diagram showing a neighborhood system of a cell in the present embodiment.

FIG. 6 is a diagram illustrating a clique according to the present embodiment.

FIG. 7 is a diagram illustrating a cell generation method according to the present embodiment.

FIG. 8 is a diagram showing cells processed in the update processing of the embodiment.

FIG. 9 is a diagram illustrating an example of processing when setting the area number of a certain cell according to the present embodiment when the area numbers of cells having the same state value among neighboring cells are the same;

FIG. 10 is a diagram illustrating an example of a process when setting an area number of a certain cell according to the present embodiment when the area numbers of cells having the same state value among neighboring cells do not match;

When setting a state value of a cell according to the present embodiment,
It is a figure which shows the process at the time of introducing a line process.

Claims (14)

[Claims] A setting step of setting an initial state value of a cell composed of image information corresponding to an image; a state updating step of updating a state value of the cell using a predetermined evaluation function; Merging the cells based on the state values of the obtained cells, and performing a region division of the image.The updating step updates not all the given cells but cells satisfying a predetermined condition. An image processing method comprising: 2. The image processing method according to claim 1, wherein the cell satisfying the predetermined condition is a cell forming a boundary of an area constituted by the cells. 3. The image processing method according to claim 1, wherein the predetermined evaluation function is an arbitrary probability density function. 4. The image processing method according to claim 1, wherein the arbitrary probability density function is a probability density function represented by an integrable function using MRF. 5. The image processing method according to claim 1, wherein the status updating step updates the status value of the cell until a predetermined condition is satisfied. 6. The image processing method according to claim 5, wherein the predetermined condition is based on a ratio of the number of updated cells to the total number of cells. 7. The image processing apparatus according to claim 1, wherein the cell is generated by dividing the image into partial images and recursively dividing the partial image based on a division condition. Method. 8. The image processing method according to claim 1, wherein the image information is a motion vector calculated by a difference between frames of a moving image. 9. The image processing method according to claim 1, wherein the image information is a pixel value of the image. 10. The image processing method according to claim 7, wherein the division condition is based on a determinant of a variance-covariance matrix of the image information. 11. The image processing method according to claim 7, wherein the division condition is based on a luminance difference when the image information is a pixel value. 12. The image processing method according to claim 7, wherein when the image information is a pixel value, the division condition is based on a value of a determinant of a variance-covariance matrix of a color vector. 13. A setting unit for setting an initial state value of a cell constituted by image information corresponding to an image; a state updating unit for updating a state value of the cell by using a predetermined evaluation function; Merging means for merging the cells based on the state values of the obtained cells, and dividing the image into regions.The updating means updates not all the given cells but cells satisfying a predetermined condition. An image processing apparatus comprising: 14. A setting code for setting an initial state value of a cell constituted by image information corresponding to an image, a state update code for updating a state value of the cell using a predetermined evaluation function, A merging code for merging the cells based on the state values of the cells, and performing a region division of an image, wherein the update code updates not all the given cells but cells that satisfy a predetermined condition. A storage medium readable by a computer.