JP5022979B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program for performing appropriate image processing on image data.

  Conventionally, with the widespread use of color scanners and digital cameras, image data input devices and output devices may be different, for example, image data captured by a digital camera is output by a printing device. In the output device, for example, the image data is output after performing correction according to the feature of the image data such as correction of the background color. As described above, when the input device and the output device are different, the feature of the image is output. It may be difficult to identify.

  In order to solve such a problem, an image processing apparatus that performs appropriate image processing on each image data is known (see, for example, “Patent Document 1”). In this apparatus, for example, undo (cancel) processing is performed on color document image data input from an image device such as a scanner in order to perform data management and procedures for generating image data optimal for various applications. ) And Redo (redo) process history and status. In addition, an image processing apparatus is disclosed that allows the user to visually and intuitively grasp the state transition by outputting the state transition of the image processing (see, for example, “Patent Document 2”).

JP 2006-053690 A JP 2006-074331 A

  However, not only the types of image data to be subjected to image processing, but also user preferences and purposes of using image data have been diversified. For example, the background treatment includes removal of the background to make it white regardless of the color of the background, and background cleaning to remove dirt and show-through while maintaining the original color. Which one to select depends on the user's preference. However, when the configuration is such that the user selects the contents of the image processing and its parameters one by one, there are problems that the user's operation becomes complicated and the work efficiency also decreases.

  The present invention has been made in view of the above, and an object thereof is to provide an image processing apparatus, an image processing method, and an image processing program capable of performing image processing desired by a user with a simple operation.

In order to solve the above-described problems and achieve the object, an image processing apparatus according to the present invention includes a history holding unit that holds a combination of image data and processing contents for the image data in a first period , A designation accepting unit that accepts designation of the processing content for the target image data, a first period, and a second period during which processing is performed on image data similar to the image data in the first period. With respect to the processing content for the specified target image data, change detecting means for detecting whether there is a change from the processing content for the similar image data held in the history holding means, and a change in the processing content is detected. in this case, a set of the feature amount and the processing content of the image data in the first time period held in the history holding means, said second period Kicking and updating means for updating the set of the processing content and the feature amount, the image data acquisition means for acquiring the target image data, by using a combination of the history holding means held image data and process content The processing content acquisition means for acquiring the processing content for the target image data, and the processing content output means for outputting the acquired processing content.

Further, the present invention is an image processing apparatus described above, the processing content acquisition means, by using the combination of the processing contents of the image data held in said history holding means, the processing content from any of the feature A prediction function creating unit that creates a prediction function used for prediction; and a feature amount calculating unit that calculates a feature amount of the target image data. The feature amount calculated by the feature amount calculating unit, and the prediction The processing content for the target image data is acquired based on the prediction function created by a function creation means.

Further, the present invention is an image processing apparatus described above, the prediction function creating means, the prediction error for the combination of the processing content and the feature amount creating a prediction function that minimizes, characterized in that .

Further, the present invention is an image processing apparatus described above, the prediction function creating means, among the feature amounts, the degree of contribution to the prediction of the processing contents to identify a large feature quantity than the other feature amounts The prediction function is created such that a weight for the feature amount is larger than a weight for another feature amount.

Further, according to the present invention, in the above image processing apparatus, the prediction function creating means has a greater degree of contribution to prediction of the processing content among the combinations of the feature amount and the processing content than other combinations. A combination is specified, and the prediction function is created so that a weight for the combination is larger than other combinations.

Further, the present invention is an image processing apparatus described above, the prediction function creating means creates the prediction function by nearest neighbor method incorporating learning weighted distance, characterized in that.

Further, the present invention is an image processing apparatus described above, the prediction function creating means creates the prediction function in accordance with the steepest descent method, it is characterized.

Further, the present invention is an image processing apparatus described above, wherein said change detecting means, when said second period of time, to detect the presence or absence of said change, characterized in that.

Further, the present invention is an image processing apparatus described above, wherein said a second period, a period in which a combination of processing contents and a predetermined number of image data is registered, characterized in that.

Further, the present invention is an image processing apparatus described above, wherein said a second time period is a predetermined time, characterized in that.

According to the present invention, in the above image processing apparatus, an image processing unit that performs image processing according to the processing content predicted by the processing content prediction unit, an output unit that outputs a processing result by the image processing unit, Is further provided.

Further, the present invention is an image processing apparatus described above, wherein said history holding means, a combination of the processing content for the image data and the image data is stored for each user, characterized in that.

The image processing method of the present invention is an image processing method executed by an image processing apparatus, and the image processing apparatus includes a control unit and a storage unit, and the storage unit stores image data in a first period. And a history holding means for holding a combination of the processing content for the image data and a step of receiving a designation of the processing content for the target image data to be processed , executed by the control unit ; change detecting means, said first period, between said first second period for performing processing for the image data similar to the image data of the period for the processing content for the specified the target image data a step of detecting the presence or absence of a change from the processing content for the image data similar held in said history holding means, updating means, a change in processing contents When detected, a set of the feature amount and the processing content of the image data in the first time period held in the history holding means, and the processing content as the feature amount in the second period and updating the set, the image data acquisition unit, acquiring the target image data, processing content acquisition means, by using a combination of the held image data and the processing contents in the history holding means The processing content for the target image data is acquired, and the processing content output means outputs the acquired processing content.

Further, the program of the present invention designates the processing contents for the target image data to be processed by using a computer having history holding means for holding a combination of the image data in the first period and the processing contents for the image data. The processing for the specified target image data between the specification receiving means for receiving the image data, the first period, and the second period for processing the image data similar to the image data of the first period Regarding the contents, a change detecting means for detecting whether or not the similar image data held in the history holding means has changed from the processing contents, and when a change in the processing contents is detected, held in the history holding means It has been a set of the feature amount and the processing content of the image data in the first period, the features and the front in the second period And updating means for updating the set of the processing contents, by using an image data acquisition means for acquiring the target image data, the combination of the processing contents of the image data held in said history holding means, the target image data It is made to function as a processing content acquisition means which acquires the processing content with respect to, and a processing content output means which outputs the acquired said processing content.

According to the present invention, there is a history holding unit that holds a combination of the image data in the first period and the processing contents for the image data, and the designation of the processing contents for the target image data to be processed is received . Between the first period and the second period during which processing is performed on image data similar to the image data in the first period, the processing content for the specified target image data is similar to that held in the history holding unit. When the presence / absence of change from the processing content to the image data is detected, the set of the image data and the processing content in the first period held in the history holding unit is set as the image data and the processing content in the second period. And the processing content for the target image data is acquired by using the combination of the image data and processing content held in the history holding means. Since outputs the processing content, the user, when the desired processing contents is displayed, only selected, it can be subjected to desired image processing. Therefore, the image processing desired by the user can be performed by a simple operation without performing a complicated operation. Further, since the history holding unit holds the image data and the processing contents received by the designation receiving unit in association with each other, the history suitable for each user can be held, and the processing content suitable for the user can be predicted. There is an effect that can be done.

Further, according to the present invention, the processing content acquisition unit creates a prediction function used for prediction of processing content from an arbitrary feature amount using a combination of image data and processing content held in the history holding unit. A prediction function creating unit that calculates the feature amount of the target image data, and a feature amount calculated by the feature amount calculating unit and a prediction function created by the prediction function creating unit Thus, since the processing content for the target image data is acquired, the processing content can be automatically acquired.

In addition, according to the present invention, the prediction function creating means creates a prediction function that minimizes the prediction error with respect to the combination of the feature quantity and the processing content, so that the prediction function can be created with high accuracy. Play.

In addition, according to the present invention, the prediction function creating means identifies a feature amount that has a greater degree of contribution to the prediction of the processing content than the other feature amounts, and the weight for the feature amount is other than the feature amount. Since a prediction function that is larger than the weight for the feature amount is created, there is an effect that the prediction function can be created with high accuracy.

In addition, according to the present invention, the prediction function creating means identifies a combination that has a greater degree of contribution to the prediction of the processing content than the other combinations from among the combination of the feature amount and the processing content, and weights for the combination Since a prediction function is generated such that becomes larger than other combinations, the effect of being able to create a prediction function with high accuracy is achieved.

Further, according to the present invention, the prediction function creating means creates the prediction function by the nearest neighbor method incorporating weighted distance learning, so that it is possible to create the prediction function with high accuracy.

Further, according to the present invention, the prediction function creating means creates the prediction function by the steepest descent method, so that it is possible to create the prediction function with high accuracy.

Further, according to the present invention, since the change detection unit detects the presence or absence of the change when the second period has elapsed, there is an effect that the prediction function can be updated periodically.

Further, according to the present invention, the second period is a period in which a combination of a predetermined number of image data and processing contents is registered, so that the prediction function can be updated at an appropriate timing. Play.

Further, according to the present invention, since the second period is a predetermined time, there is an effect that the prediction function can be updated at an appropriate timing.

According to the present invention, the image processing means performs image processing according to the processing content predicted by the processing content prediction means, and the output means outputs the processing result by the image processing means. There is an effect that it is possible to view the image processing result according to the processed content.

Further, according to the present invention, since the history holding unit holds the combination of the image data and the processing content for the image data for each user, the processing content for the target image data is predicted and predicted for each user. There is an effect that the processing contents can be output.

  Exemplary embodiments of an image processing apparatus, an image processing method, and a program according to the present invention are explained in detail below with reference to the accompanying drawings.

(First embodiment)
FIG. 1 is a block diagram showing electrical connections of the image processing apparatus 1 according to the first embodiment of the present invention. As shown in FIG. 1, an image processing apparatus 1 is a computer such as a PC (Personal Computer), and includes a CPU (Central Processing Unit) 2 that centrally controls each unit of the image processing apparatus 1 and a ROM ( Secondary storage such as primary storage device 5 such as Read Only Memory (RAM) 3 and RAM (Random Access Memory) 4 and HDD (Hard Disk Drive) 7 as a storage unit for storing data files (for example, color bitmap image data). Information is transmitted by communication with other external computers via the network 6, a removable disk device 8 such as a CD-ROM drive for storing information, distributing information to the outside, and obtaining information from the outside, and a network 9. Network interface 10, CRT (Cathode Ray Tube), LCD (Liquid Crystal Display), etc. for displaying the process progress and results to the operator The display device 11 and a keyboard 12 for an operator to input commands and information to the CPU 2, a pointing device 13 such as a mouse, and the like. The data bus 14 arbitrates data transmitted and received between these components. Works.

  In the present embodiment, a general personal computer is applied as the image processing apparatus 1. However, the present invention is not limited to this, and a portable information terminal called PDA (Personal Digital Assistants). , PalmTopPC, mobile phone, PHS (Personal Handyphone System), etc.

  In such an image processing apparatus 1, when the user turns on the power, the CPU 2 activates a program called a loader in the ROM 3, loads a program for managing the hardware and software of the computer called the operating system from the HDD 6 into the RAM 4, and Start the system. Such an operating system starts a program, reads information, and performs storage according to a user operation. As typical operating systems, Windows (registered trademark), UNIX (registered trademark), and the like are known. An operation program running on these operating systems is called an application program.

  Here, the image processing apparatus 1 stores an image processing program in the HDD 6 as an application program. In this sense, the HDD 6 functions as a storage medium that stores the image processing program.

  In general, the application program installed in the secondary storage device 7 such as the HDD 6 of the image processing apparatus 1 is stored in an optical information recording medium such as a CD-ROM or DVD-ROM, or a magnetic medium such as an FD. The application program recorded on the medium 8 a and recorded on the storage medium 8 a is installed in the secondary storage device 7 such as the HDD 6. Therefore, the portable storage medium 8a such as an optical information recording medium such as a CD-ROM or a magnetic medium such as an FD can also be a storage medium for storing an image processing program. Further, the image processing program is stored on a computer connected to a network such as the Internet, and is installed in the secondary storage device 7 such as the HDD 6 by being downloaded from the outside via the network interface 10, for example. You may do it. The image processing program executed by the image processing apparatus 1 according to the present embodiment may be provided or distributed via a network such as the Internet.

  In the image processing apparatus 1, when an image processing program that operates on an operating system is started, the CPU 2 executes various arithmetic processes according to the image processing program and centrally controls each unit. Of various types of arithmetic processing executed by the CPU 2 of the image processing apparatus 1, normalization processing at the time of image data accumulation / transmission, which is a characteristic processing of the present embodiment, will be described below. Here, the normalization process is a process for converting digital image data received through an external device (for example, a scanner or a digital camera) connected to the image processing apparatus 1 or the network 9 into an ideal form. .

  In addition, when real-time property is regarded as important, it is necessary to speed up the processing. For this purpose, it is desirable to separately provide a logic circuit (not shown) and execute various arithmetic processes by the operation of the logic circuit.

  Here, image processing executed by the CPU 2 of the image processing apparatus 1 will be described. FIG. 2 is a functional block diagram illustrating functions related to image processing executed by the CPU 2 of the image processing apparatus 1. The image processing apparatus 1 includes an image data acquisition unit 100, a feature amount calculation unit 102, a designation reception unit 104, a history database (DB) 110, a prediction function creation unit 120, a processing content prediction unit ( Processing content acquisition unit) 122, processing content output unit 124, image processing unit 126, change detection unit 130, and update unit 132.

  The image data acquisition unit 100 acquires image data. Further, when the input image data is related to a document, the tilt of the document, that is, skew correction is performed.

  The feature amount calculation unit 102 calculates a feature amount in the entire image data. Here, the feature amount includes statistical information such as the ratio of characters in the entire image, the ratio of pictures in the entire image, the degree of scattering between characters and pictures, and the layout density. In addition, there are a spatial distribution of characters and pictures, a distribution of colors and edges, and a background color.

  The designation receiving unit 104 receives a designation of processing content to be performed on the image data acquired by the feature amount calculating unit 102. The processing content is designated by an input from the user. The processing content includes the type of processing and its parameters. Examples of the processing type include a background color correction process, a spatial filter process, and a resolution enlargement process.

  As the background color correction processing, there are background removal excluding the background color and background cleaning for correcting the background color. Background removal is described in Japanese Patent Application Laid-Open Nos. 2004-320701 and 2005-110184. Here, assuming that a set of algorithms and parameters is A, the background color correction process is defined as follows.

      A = {Skin removal, skin cleaning, do nothing}

  Spatial filter processing includes smoothing processing, edge enhancement processing, and adaptive filtering over the entire processing target image. Here, adaptive filtering performs different processing for each pixel. Details are described in JP-A No. 2003-281526. Spatial filtering is defined as follows.

      A = {smoothing process, edge enhancement process, adaptive filtering, nothing}

  Examples of the resolution enlarging process include a process for enlarging the resolution of characters as described in Japanese Patent Application Laid-Open No. 2005-063055, and normal image interpolation. The resolution enlargement process is defined as follows.

      A = {Character resolution enlargement, image interpolation, do nothing}

The history DB 110 associates the feature amount obtained by the feature amount calculation unit 102 with predetermined image data and the processing content designated by the user for the same image data for each user (user ID). Store). The history DB 110 stores combinations of feature amounts and processing contents in chronological order in the order in which the processing contents are specified. That is, the history DB 110 stores history information H as shown in (Equation 1).

  Here, x (k) is a feature amount extracted from the kth image data. α (k) (α ∈ A) is an algorithm or processing parameter suitable for the image data, that is, processing content.

  The prediction function creation unit 120 creates a prediction function for specifying the processing content to be performed on the newly acquired image data based on the relationship between the feature amount held in the history DB 110 and the processing content. .

  The processing content prediction unit (processing content acquisition unit) 122 includes a feature amount calculation unit 102 and a prediction function creation unit 120, and the feature amount calculated by the feature amount calculation unit 102 and prediction for target image data to be predicted. Based on the prediction function created by the function creation unit 120, the optimum processing content for the target image data is predicted.

  The processing content output unit 124 displays the optimal processing content predicted by the processing content prediction unit (processing content acquisition unit) 122 on the display screen.

  The image processing unit 126 performs image processing on the image data in accordance with the processing content selected by the designation receiving unit 104.

  The change detection unit 130 is registered in the history DB 110 in a predetermined period, that is, the relationship between the image data registered in the history DB 110 in the first period and the processing content, and in a second period that is a period after the first period. The presence or absence of a change between the image data and the relationship between the processing contents is detected. The image data includes the feature amount of the image data.

  The update unit 132 updates the contents of the history DB 110 based on the detection result by the change detection unit 130.

  FIG. 3 is a flowchart showing a history registration process for registering the feature vector and the processing content in the history DB 110 in the image processing apparatus 1. First, the image data acquisition unit 100 acquires image data (step S100). Next, the feature amount calculation unit 102 calculates a feature amount vector of the image data acquired by the image data acquisition unit 100 (step S102). Furthermore, the designation accepting unit 104 accepts designation of processing details of image processing for the image data acquired by the image data acquiring unit 100 (step S104).

  Next, the feature amount vector calculated by the feature amount calculation unit 102 and the processing content specified by the designation receiving unit 104 are stored in the history DB 110 in association with each other (step S106). That is, “x (k) α (k)” is added to the history information H. Then, the image processing unit 126 performs image processing of the specified processing content on the image data (step S108).

  FIG. 4 is a flowchart showing detailed processing in the feature amount calculation processing (step S102). The feature amount calculation unit 102 exclusively divides the image data acquired from the image data acquisition unit 100 into rectangular blocks having the same size (step S110). Specifically, the image data is divided into blocks of the same size, for example, a 1 cm × 1 cm rectangle (80 pixels × 80 pixels if the resolution is 200 dpi, 120 pixels × height 120 pixels if the resolution is 300 dpi). .

Next, each block is classified into one of three types of “picture”, “character”, and “other” (step S112). Specifically, as shown in FIG. 5, first, an image I obtained by reducing a block image to be processed to a low resolution of about 100 dpi is generated (step S120), and a resolution level number L is set (step S120). In step S122, the resolution reduction level k is initialized (k ← 0) (step S124). The reason why the processes in steps S120 to S124 are performed is to extract features from the image I and the resolution-reduced image as shown in FIG. Although details will be described later, for example, when the resolution level number L 2, the image I, the images I ₁ resolution 1/2, the resolution is the image I ₂ 1/4 image meter 3 Extract features from two images.

If the resolution reduction level k has not reached the resolution level number L (Yes in step S126), the image I _k (k = 0,...) Obtained by reducing the resolution to 1/2 ^k from the image I generated in step S120. ., L) is generated (step S128). Next, the image I _k is binarized (step S130). However, in a binary image, a black pixel has a value 1 and a white pixel has a value 0.

Then, from the image I _k of binarized resolution 1/2 ^k, after calculating the feature vectors f _k M-dimensional (step S132), the resolution reduction level k by "1" is incremented (k ← k + 1) (Step S134).

Here, a method for extracting features from an image obtained by binarizing the image I _k (k = 0,..., L) will be described. The “higher order autocorrelation function (Nth order autocorrelation function)”, which is an extension of the autocorrelation function to the higher order (Nth order), indicates that the displacement direction (S ₁ , S ₂ ,..., S _N ), defined by (Equation 2).

Here, the sum Σ is addition for the pixel r of the entire image. Therefore, an infinite number of high-order autocorrelation functions can be considered depending on the order and the direction of displacement (S ₁ , S ₂ ,..., S _N ). Here, for simplicity, the order N of the higher-order autocorrelation function is set to “2”. Further, the displacement direction is limited to a local 3 × 3 pixel region around the reference pixel r. Excluding equivalent features by translation, the total number of features is 25 for a binary image as shown in FIG. For the calculation of each feature, the product of the corresponding pixel values of the local pattern may be added to the entire image. For example, the feature corresponding to the local pattern of “No. 3” is calculated by taking the sum of products for the entire image of the gray value at the reference pixel r and the gray value at the point immediately adjacent to the reference pixel r. In this way, an M = 25-dimensional feature vector f _k is calculated by (Equation 3) from an image having a resolution of 1/2 ^k .

  The processes in steps S128 to S134 as described above are repeated until the resolution reduction level k incremented in step S18 exceeds the resolution level number L (No in step S126).

If incremented resolution reduction level k has exceeded the number of resolution levels L at step S134 (step S126, No), feature vector f _0, · · ·, based on f _L, the block, " It is classified into one of three types of picture, “character” and “other” (step 136).

ここで、ｅは誤差ベクトルである。また、Ｆは（式５）により定義される。

最小二乗法により、最適な結合係数ベクトルａは、（式６）により与えられる。

Here, the block classification method will be described in detail. First, from the aforementioned M = 25-dimensional feature vector f _k = (g (k, 1),..., G (k, 25)) (k = 0,..., L) to (25 × L ) Dimension feature vector x = (g (0,1),..., G (0,25),..., G (L, 1),. Generate. In order to perform classification using such a block feature quantity vector x, it is necessary to perform learning in advance. Therefore, in the present embodiment, the feature amount vector x is calculated by dividing the learning data into two types, one containing only characters and one not containing characters. Thereafter, the feature quantity vector p ₀ of the character pixel and the feature quantity vector p ₁ of the non-character pixel are calculated in advance by taking the respective averages. Then, if the feature vector x obtained from the block image to be classified is decomposed into a linear combination of the known feature vectors p ₀ and p ₁ , the coupling coefficients a ₀ and a ₁ become character pixels and non-characters. It represents the ratio of pixels or the “characteristic” and “non-characteristic” of the block. Such decomposition is possible because the feature based on the higher-order local autocorrelation is invariant to the position of the object in the screen, and is additive with respect to the number of objects. The feature vector x is decomposed as in (Equation 4).

Here, e is an error vector. F is defined by (Formula 5).

By the least square method, the optimum coupling coefficient vector a is given by (Equation 6).

Each block is classified into “picture”, “not a picture”, and “undecided” by performing threshold processing on the parameter a ₁ representing “non-characteristic”. Each block is classified as “undecided” or “not a picture”, and is classified as “character” if the parameter a ₀ representing the character character is greater than or equal to a threshold value, and “other” otherwise. FIG. 8 shows an example of block classification. In the example of FIG. 8, the black portion represents “character”, the gray portion represents “picture”, and the white portion represents “other”.

The description returns to FIG. 4 again. After classification into any of the three types of “picture”, “character” and “other”, about 20 image feature quantities are calculated based on the classification results of all blocks (step S114). Image features include, for example, the ratio of characters and pictures, the density ratio: how the layout is crowded (the degree to which it is packed in a narrow space), and the degree of scattering of letters and pictures: the characters and pictures are scattered throughout the paper. There is a degree. In this case, specifically, the following five values are calculated as image feature amounts. Note that the feature amount calculation unit 102 calculates various image feature amounts other than this. The feature quantity calculation unit 102 extracts about 20 types, that is, about 20 dimensions. It is preferable to use as many feature quantities as possible from the viewpoint of creating a prediction function that selects appropriate process contents according to the history of process contents designation by various users.
1. Character ratio Rtε [0,1]: Ratio of blocks classified as “character” in all blocks 2. Non-character ratio Rpε [0, 1]: Ratio of blocks classified as “picture” in all blocks Layout density Dε [0, 1]: the sum of the areas of the blocks of “character” and “picture” divided by the area of the drawing area 4. Character scattering degree St (> 0): the spatial distribution of character blocks in the x and y directions, normalized by the area of the image of the determinant of the variance / covariance matrix Non-character scattering degree Sp (> 0): the spatial distribution of picture blocks in the x and y directions, normalized by the area of the image and the determinant of the variance / covariance matrix

  FIG. 9 is a diagram illustrating an example of history information registered in the history DB 110. In the example shown in FIG. 9, the image data, the feature amount obtained from the image data, and the processing content designated for the image data are associated with each other. Further, the processing content designated for the image data includes background color correction processing (background removal, background cleaning), spatial filter processing (smoothing processing, edge enhancement processing, adaptive filtering), as shown in FIG. The contents include resolution enlargement processing (character resolution enlargement, image interpolation) and the like. Further, as shown in FIG. 10, the processing content designated for the image data includes not only the processing itself but also parameters. “Skin removal 3”, “Edge enhancement 1”, and the like shown in FIG. 10 are parameters.

  FIG. 11 is a flowchart showing a process content prediction process for presenting appropriate process contents to the user using a prediction function. An appropriate prediction function can be obtained when the history information H is used for a predetermined number of image data such as 20, for example. Therefore, when the history information H of a specified number or more is stored, the process content prediction process is started.

  First, the prediction function creation unit 120 creates a prediction function for predicting processing contents from image data based on the history information H stored in the history DB 110 (step S200). The prediction function is a function obtained by learning from the history information H. That is, the history information H stored in the history DB 110 is set as a learning data set for creating a prediction function. The prediction function is created using a nearest neighbor method incorporating weighted distance learning.

When a feature value set F representing image content, an algorithm set A of processing contents and parameters, and history information H are given, an element uεU of the user set U and given unknown image data are observed. A function f representing the suitability f _H (α, x, u) of the algorithm α∈A for the feature quantity x is generated as a prediction function. Here, f _H is expressed as (Equation 7).

However, a different function f _H is constructed for each different algorithm set A. That is, the function f _H of the algorithm set A of the "background color correction processing" function f _H of the algorithm set A of the "spatial filtering" is constructed separately.

However, there are the following technical problems. In other words, considering the Bayesian identification framework, f _H (α, x, u) can be formulated as a problem for obtaining the probability represented by (Equation 8) for the user u, the image feature x, and the process α.

Here, P (x | u) is a normalization factor for the image. Therefore, it may be ignored when determining the priority order of the plurality of processes α. Therefore, f _H (α, x, u) is obtained by (Equation 9).

  It is easy to obtain P (α | u) from the history information H. For the processing content α, the number of times the processing is used may be recorded for each user. The history information H is the distribution p (x | α, u) of the feature quantity for the image to which the user u has applied the processing content α.

However, the following four points are further considered when creating the prediction function.
1. The history information H stored in the history DB 110 depends on the user's preference and task of the image processing apparatus 1. Therefore, on-site online learning is required.
2. In the image processing apparatus 1, it must be assumed that the history information that can be used for learning is relatively small (several tens to hundreds). This is due to the fact that on-site it is necessary to read user preferences and tasks from as little data as possible and adapt immediately.
3. The feature space is multidimensional (about 20). Therefore, a feature selection mechanism for selecting only features suitable for prediction and removing disturbance factors, or weighting to each feature dimension is necessary. In addition, it is necessary to consider that the feature subsets suitable for prediction differ depending on individual selection targets and users.
4). The feature quantity is continuous and multidimensional. Furthermore, the number of data is small. In this case, it is practically difficult to obtain p (x | α, u). This is because it is difficult to estimate the probability distribution p (x | α, u) by using the non-parametric Parzen window method or the EM method assuming a mixed Gaussian distribution due to the problem of “curse of dimension”.

  On the other hand, the nearest neighbor method is suitable for on-site learning and is an identification method that does not assume the form of a probability distribution function. The nearest neighbor method is a prediction method using a past case that is most similar to the one currently being processed, and the prediction accuracy improves as the number of similar data increases. In addition, the identification method does not require estimation of probability distribution such as Gaussian distribution.

  Furthermore, for problems with less learning data and multidimensional features, the distance measure in the nearest neighbor method is weighted according to the contribution to the prediction of each feature dimension, and learning data (combination of feature quantity and processing content) The dilemma between the number of data and the number of dimensions can be eliminated by weighting according to the importance of each. Therefore, in this embodiment, such a nearest neighbor method, that is, a nearest neighbor method incorporating weighted distance learning is used.

  The nearest neighbor method incorporating weighted distance learning is basically based on the nearest neighbor method. Furthermore, when calculating the distance between the prototype point and the prediction target point, the prototype point is not a simple Euclidean distance. The weighted distance is calculated according to the importance of each and the importance of each prediction target point.

Here, the i-th prototype point x _i is defined as in (Expression 10), and the prediction target point, that is, an arbitrary point y to be identified is defined as in (Expression 11). Here, the prototype point x _i corresponds to each feature quantity α included in the history information H. An arbitrary point y to be identified corresponds to the feature amount α obtained from the target image data.

Further, let c be a class. Here, the class is an element of the set A, that is, an index of an applied algorithm or processing parameter. Each prototype point x _i is accompanied by class information indicating the class applied by the user.

The square (Δ) of the distance between x _i and y is calculated by (Equation 12) using the weight v _i for the i-th prototype point and the weight w _cj for the j-th feature dimension for class c.

  Assuming that the number of feature dimensions is d, the number of data is N, and the number of classes is C, the number of parameters is N for the prototype weight and Cd for the weight of each dimension determined for each class. That is, N + Cd in total.

The weights v _i and w _cj in (Equation 9) are automatically learned from the data. The criterion for learning is to minimize the error rate evaluated by Leave-One-Out. That is, the weight is learned by the steepest descent method according to the following four criteria.
1. It is assumed that the same class of points as the predetermined prototype points are sparsely distributed, and the identification result changes due to the absence of the prototype points. In this case, it can be determined that this prototype point has a large influence on the prediction function of the processing content, that is, a prototype point having a high importance. Therefore, the weight v _i is set to a larger value so that the range affected by the prototype point becomes wider.
2. It is assumed that points of the same class as the predetermined prototype points are dense, and the presence of the prototype points has a low degree of influence on the identification result. In this case, it can be determined that the importance of this prototype point is low. Therefore, the weight v _i is set to a smaller value.
3. For a given class c, the weight w _cj takes a larger value if the influence of the jth feature dimension on the prediction is greater.
4). For class c, if the j-th feature dimension is a factor that disturbs the prediction, the weight w _cj is close to zero.

  For the nearest neighbor method incorporating weighted distance learning, see R. Paredes & E. Vidal, “Learning weighted metrics to minimize nearest-neighbor classification error,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 7, pp. 1100-1110, July 2006.

  After the prediction function is created, when the image data acquisition unit 100 acquires target image data to be predicted (step S202, Yes), the feature amount calculation unit 102 acquires the target image data acquired by the image data acquisition unit 100. A feature amount is calculated (step S204). Note that the process for calculating the feature amount is the same as the feature amount calculation process in step S102 described with reference to FIG.

Next, the processing content prediction unit (processing content acquisition unit) 122 predicts the optimal processing content for the target image data (step S206). Specifically, the feature amount for the target image data acquired from the feature amount calculation unit 102 is input. Then, the optimal processing content, that is, the algorithm and processing parameters for the target image data is predicted using the prediction function. The square (Δ) of the distance between the i-th prototype point x _i (its class label is c) and the feature quantity y obtained for the target image data is calculated by (Equation 12). At this time, the weight v _i for the i-th prototype point obtained by the prediction function creation unit 120 and the weight w _cj of the j-th feature dimension are used for the class c. The processing content prediction unit (processing content acquisition unit) 122 identifies a prototype point that minimizes the distance between the prototype point and the prediction target point. Then, a recommended algorithm or a recommended parameter is specified for the class label of the feature amount.

  That is, the feature quantity that most closely approximates the feature quantity of the target image data is specified from the feature quantities of each image data stored in the history DB 110. Then, the processing content α associated with the feature amount is predicted and acquired as the optimal processing content for the target image data.

  Next, the processing content output unit 124 presents the optimal processing content α acquired by the processing content prediction unit (processing content acquisition unit) 122 to the user (step S208).

  As described above, since the optimum processing content is presented to the user, the user can perform the corresponding processing content by selecting the desired processing content if presented. Thus, it is possible to save the trouble of inputting various processing contents and parameters every time image data is input.

  Furthermore, when the presented processing content is not what is desired, the user may specify appropriate processing content and parameters using a user interface such as a keyboard and a mouse.

  When the designation receiving unit 104 receives a change specification of the processing content from the user together with the user ID (Yes at Step S210), the specification receiving unit 104 changes the content of the history DB 110 according to the change specification (Step S212). Here, the change designation is designation for applying processing contents different from the presented processing contents to the target image data. The change specification includes the processing content after the change. Specifically, the feature amount obtained for the target image data and the processing content included in the change designation are newly added to the history information H. If there is no change designation (step S210, No), the process proceeds to step S214.

  Next, image processing is performed on the target image data in accordance with the specified processing content (step S214). Next, when it is the update timing of the learning data set (step S216, Yes), the change detection unit 130 detects the presence / absence of preference change and updates the data set (step S218). Then, the process returns to step S202 again. Note that the update timing of the learning data set is, for example, after a predetermined period has elapsed since the previous update timing of the learning data set.

  As another example, for example, after a learning data set is set or a learning data set is updated (described later), a combination of feature amounts and processing contents for a predetermined number of image data is newly added to the history DB 110. It may be when it is added to.

  As another example, the learning data set may be updated each time a new combination of feature amount and processing content is added. Thus, the timing may be a predetermined timing and is not limited to the embodiment.

  FIG. 12 is a flowchart showing detailed processing in preference change presence / absence detection and learning data set update processing (step S218). Whether or not there is a preference change by the change detection unit 130 is determined whether processing content different from processing content (processing content or algorithm) for the specified target image data is combined with image data having similar feature amounts. This is done by detecting this.

First, as a premise, the learning data set T ₀ used when the previous prediction function was created (step S200 or step S224 described later) is set. Furthermore, after creating the previous prediction function, the data set T ₁ added to the history information H of the history DB 110 is set (step S220). Furthermore, let P ₀ be a prediction function created from T ₀ . Here, T ₀ is expressed as (Equation 13).

Next, error data F is detected (step S222). Specifically, by using the nearest neighbor method, for each element of T ₀ ∪T ₁ , the closest data obtained by excluding itself is found (one-point exclusion method). Here, T ₀ ∪T ₁ is indexed. Specifically, 1 to | T ₀ | is an element of T ₀ . Let | T ₀ | +1 to | T ₁ | be an element of T ₁ .

Then, error data F indicated by (Equation 14) is obtained. Here, the error data F is data that could not be correctly recognized by the nearest neighbor method. In other words, the error data F is different from the selection example (processing content) recorded in association with the closest data excluding itself and the processing content actually selected.

Next, the contradictory data combination S is detected (step S224). Here, the combination S is a combination of data whose selection is different between T ₀ and T ₁ though the distance is short. In other words, it is a combination of data that is similar in feature quantity but different in associated processing content.

The combination S is defined as (Equation 15).

That is, i and k in the combination S are the nearest neighbor points. I and k belong to T ₀ and T ₁ , respectively. However, the selection cases are different between i and k, and when T ₀ and T ₁ are merged, they adversely affect each other. That is, the data combination is inconsistent.

FIG. 13 is a diagram for explaining the combination S. FIG. In FIG. 13, white dots indicate T ₀ data. Black dots shows the data of T _1. The distance between the data A at T ₀ and the data B at T ₁ is d. Assume that there is no other data in a circle with a radius d centered on A and B, respectively, and the selection examples at points A and B are different. In this case, a combination of data A and data B is added to the set S.

The description returns to FIG. 12 again. After detecting the combination S (step S224), from among the data included in the T _0, deleting old data contained in the combination S (step S226). That is, the processing of (Expression 16) is performed.

Next, data obtained by merging T ₀ and T ₁ after deletion is used as a learning data set. That is, the learning data set is updated (step S228). Next, the prediction function creating unit 120 creates a prediction function using the newly set learning data set. That is, the prediction function is updated (step S230). Note that the process of creating a prediction function is the same as the process of creating a prediction function (step S200) described with reference to FIG.

  Next, when the number of S is equal to or larger than a predetermined value (step S232, Yes), it is determined that there is a preference change, and output is made to notify the user to that effect (step S234). On the other hand, if the number of S is less than the specified value (No at step S232), it is determined that there is no preference change and is output to notify the user to that effect (step S236). This completes the preference change presence / absence detection process (step S218).

  As described above, the present invention has been described using the embodiment, but various changes or improvements can be added to the above embodiment.

(Second Embodiment)
Next, an image processing apparatus 20 according to the second embodiment will be described. The image processing apparatus 20 according to the second embodiment performs processing content prediction processing on a plurality of pieces of target image data in batches. FIG. 14 is a functional block diagram illustrating functions related to image processing executed by the CPU 2 in the image processing apparatus 20 according to the second embodiment. As functions of the image processing apparatus 20 according to the second embodiment, in addition to the functions of the image processing apparatus 20 according to the first embodiment, a processing result holding unit 140 and a processing result display unit 142 are further provided. I have. The processing result holding unit 140 holds the processing result of the image processing performed by the image processing unit 126. The processing result display unit 142 displays the processing result held in the processing result holding unit 140.

  FIG. 15 is a flowchart illustrating the processing content prediction processing according to the modification example. After predicting the optimum processing content (step S206), the target image data is actually subjected to image processing according to the predicted processing content (step S240). Next, the processing result is stored. The processing result is stored in the primary storage device 5 or the like. If target image data that has not been subjected to image processing remains after the above processing (step S244, Yes), the processing from step S204 to step S242 is performed. If image processing has been performed on all target image data (step S244, No), then processing results for all target image data are presented (step S246). Then, the process after step S210 is performed.

  As described above, in the image processing apparatus 20 according to the second embodiment, the processing result of the image processing is displayed. Therefore, the user determines whether to actually browse the processing result and change the processing content. be able to. Further, the processing efficiency can be improved by performing the processing in batches in this way.

(Third embodiment)
FIG. 16 is a diagram illustrating a schematic configuration of a multi-function device 50 according to the third embodiment. As shown in FIG. 16, the multi-function device 50 includes a scanner unit 51 that is an image reading unit and a printer unit 52 that is an image printing apparatus. The multi-function device 50 has the functions of the image processing apparatus 1 described in the other embodiments. More specifically, the image data acquisition unit 100 acquires a scan image read by the scanner unit 51 as image data and target image data, and performs various processes related to process content prediction.

  Other configurations and processes of the multi-function device 50 according to the third embodiment are the same as the configurations and processes of the image processing apparatus 1 according to the other embodiments.

(Fourth embodiment)
FIG. 17 is a diagram illustrating an overall configuration of an image processing system 60 according to the fourth embodiment. The image processing system 60 is a server client system, and a server computer S and a plurality of client computers C are connected via a network N. The server computer S performs processing in the image processing apparatus 1. Each client computer C transmits an image to the server computer S. The server computer S has each function of the image processing apparatus 1. Further, a network scanner NS is provided on the network N. The image data acquisition unit 100 of the server computer S acquires image data from each client computer S or network scanner NS.

  The history DB 110 may be held in a server (not shown) other than the server computer S.

  Other configurations and processes of the image processing system 60 according to the fourth embodiment are the same as the configurations and processes of the image processing apparatus 1 according to the other embodiments.

1 is a block diagram showing electrical connection of an image processing apparatus according to a first embodiment. It is a functional block diagram which shows the function concerning the image processing which CPU of an image processing apparatus performs. It is a flowchart which shows the history registration process which registers a feature-value vector and process content in log | history DB in an image processing apparatus. It is a flowchart which shows the detailed process in a feature-value calculation process. It is a flowchart which shows the detailed process in a classification | category process. It is a schematic diagram which shows multi-resolution processing. It is a figure which shows an example of the mask pattern for high-dimensional autocorrelation function calculation. It is a schematic diagram which shows the example of a block classification | category. It is a figure which shows an example of the log | history information registered into log | history DB. It is a figure which shows an example of the processing content registered into log | history DB. It is a flowchart which shows the process content prediction process which shows a suitable process content to a user using a prediction function. It is a flowchart which shows the detailed process in a preference change presence-and-absence detection and learning data set update process. It is a figure for demonstrating the combination S. FIG. It is a functional block diagram which shows the function concerning the image processing which CPU in the image processing apparatus concerning 2nd Embodiment performs. It is a flowchart which shows the process content prediction process concerning the example of a change. It is a figure which shows schematic structure of the multifunctional device concerning 3rd Embodiment. It is a figure which shows the whole structure of the image processing system concerning 4th Embodiment.

Explanation of symbols

1,20 Image processing device 2 CPU
5 Primary storage device 6 Secondary storage device 8 Removable disk device 8a Storage medium 10 Network interface 11 Display device 12 Keyboard 13 Pointing device 14 Bus controller 50 Multifunction device 51 Scanner unit 52 Printer unit 60 Image processing system 100 Image data acquisition unit 102 Features Quantity calculation unit 104 Specification reception unit 110 History DB
DESCRIPTION OF SYMBOLS 120 Prediction function production | generation part 122 Process content prediction part 124 Process content output part 126 Image processing part 130 Change detection part 132 Update part 140 Process result holding part 142 Process result display part

Claims (14)

History holding means for holding a combination of image data in the first period and processing contents for the image data ;
A designation accepting unit that accepts designation of the processing content for target image data to be processed;
It said first time period, between said first second period for performing processing for the image data similar to the image data of the period for the processing content for the specified the target image data, the history holding means Change detection means for detecting the presence or absence of a change from the processing content with respect to similar image data held in
When a change in processing content is detected, a set of the feature amount of the image data and the processing content in the first period held in the history holding unit is set as the feature amount in the second period. And updating means for updating to a set of the processing contents;
An image data acquisition means for acquiring the target image data,
Processing content acquisition means for acquiring processing content for the target image data using a combination of image data and processing content held in the history holding means;
Processing content output means for outputting the acquired processing content;
An image processing apparatus comprising: The processing content obtaining unit creates a prediction function used to predict the processing content from an arbitrary feature amount using a combination of the image data and processing content held in the history holding unit. And a feature amount calculation means for calculating a feature amount of the target image data, and based on the feature amount calculated by the feature amount calculation means and the prediction function created by the prediction function creation means Obtaining the processing content for the target image data,
The image processing apparatus according to claim 1. The prediction function creating means creates a prediction function that minimizes a prediction error for a combination of the feature quantity and the processing content;
The image processing apparatus according to claim 2. The prediction function creating means identifies a feature amount that has a greater degree of contribution to prediction of the processing content than the other feature amounts, and the weight for the feature amount is a weight for the other feature amount. Create the prediction function that is larger than
The image processing apparatus according to claim 3 . The prediction function creating means identifies a combination having a larger degree of contribution to the prediction of the processing content than the other combinations among the combination of the feature amount and the processing content, and the weight for the combination is another combination Create the prediction function that is larger than
The image processing apparatus according to claim 3 or 4, characterized in that. The prediction function creating means creates the prediction function by a nearest neighbor method incorporating weighted distance learning.
The image processing apparatus according to claim 4 or 5, characterized in that. The prediction function creating means creates the prediction function by a steepest descent method.
The image processing apparatus according to any one of claims 3 5, characterized in that. The change detecting means detects the presence or absence of the change when the second period has elapsed;
The image processing apparatus according to any one of claims 1 to 7, characterized in that. The second period is a period in which a predetermined number of combinations of image data and processing contents are registered.
The image processing apparatus according to any one of claims 1 to 8, characterized in that. The second period is a predetermined time.
The image processing apparatus according to any one of claims 1 9, characterized in that. Image processing means for performing image processing according to the processing content predicted by the processing content prediction means;
Output means for outputting a processing result by the image processing means;
The image processing apparatus according to any one of claims 1 to 1 0, characterized in that it further comprises a. The history holding unit holds a combination of the image data and the processing content for the image data for each user.
The image processing apparatus according to any one of claims 1 1 1, characterized in that. An image processing method executed by an image processing apparatus,
The image processing apparatus includes a control unit and a storage unit,
The storage unit
A history holding means for holding a combination of the image data in the first period and the processing content for the image data;
Executed in the control unit,
A step of accepting designation of the processing content with respect to target image data to be processed;
Change detecting means, said first period, between said first second period for performing processing for the image data similar to the image data of the period for the processing content for the specified the target image data Detecting the presence or absence of a change from the processing content for similar image data held in the history holding means ;
When a change in processing content is detected by the update unit, a set of the feature amount of the image data and the processing content in the first period held in the history holding unit is set in the second period. Updating to the set of the feature quantity and the processing content in
Image data acquisition means, acquiring the target image data,
Processing content acquisition means, using a combination of image data and processing content held in the history holding means, to obtain processing content for the target image data;
A processing content output means for outputting the acquired processing content;
An image processing method comprising: A computer having history holding means for holding a combination of image data in the first period and processing contents for the image data ,
A designation accepting unit that accepts designation of the processing content for target image data to be processed;
It said first time period, between said first second period for performing processing for the image data similar to the image data of the period for the processing content for the specified the target image data, the history holding means Change detection means for detecting the presence or absence of a change from the processing content with respect to similar image data held in
When a change in processing content is detected, a set of the feature amount of the image data and the processing content in the first period held in the history holding unit is set as the feature amount in the second period. And updating means for updating to a set of the processing contents;
An image data acquisition means for acquiring the target image data,
Processing content acquisition means for acquiring processing content for the target image data using a combination of image data and processing content held in the history holding means;
Processing content output means for outputting the acquired processing content;
A program characterized by functioning as

Images