JP3492990B2 - Image processing apparatus, image processing method, and recording medium - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and an image processing method for extracting a sensitive characteristic amount from image data.

[0002]

2. Description of the Related Art In recent years, with the spread of the Internet centering on the WWW, an environment where anyone can easily access enormous amounts of information on various media is being established. Under such circumstances, a technology for providing information based on the perspective of each user and the interpretation according to the purpose has become indispensable. As for media, which is a typical image in multimedia information, there are great expectations for its applications such as searching for similar images and image creation support based on sensitivity, and research results have attracted attention from various industries.

In the interpretation of images, due to its nature, modeling that transforms into an abstract representation is essential. Conventionally, as a method of modeling an image, an object extraction using an image processing technique such as binarization, thinning, and edge processing (Reference 1) and contour extraction using mathematical morphology (Reference 2) are mainstream. Met. In recent years, in addition to these, in addition to these, a kansei engineering approach in which the amount of features such as color (references 3 and 4), composition (references 5 and 6), texture (references 7 and 8), and an index that characterizes images are used. A method of converting text information into words has been attempted.

The above documents are as follows. Reference 1: “Computer Vision, New Technology Communications, 1998” Reference 2: “J. Serra: Image Analysis
sis and Mathematic Morp
holology, Academic Press, L
ondon, 1982] Reference 3: "Yuudai Morohara et al .: Selection of image color in textile design image, IPSJ Transactions, V"
ol. 36, No. 2, pp. 329-337, 199
5 ”Reference 4:“ Akihiro Kuroda et al .: Integrated System for Design Applying Kansei Search of Image Database, 9th NI
COGRAPH Proceedings, pp. 113-122, 199
3 ”Document 5:“ Hidenori Yamamoto et al .: Similar image retrieval considering spatial distribution of color information, IEICE technical report, EID98
-171, IE98-162, 1999-02 "Document 6:" Hideyuki Kobayashi et al .: Image Retrieval System Integrating Characteristic Values and Approaching Human Sensitivity, IEICE Technical Report, "
PRMU97-261, 1998-03 ”Document 7:“ Shigeru Mase et al .: Morphology and Image Analysis [1] ”, IEICE Journal, Vol.74, No.
2, pp. 166-174, 1991] Reference 8: "Shusuke Ueda et al .: Morphology and Image Analysis [2], The Institute of Electronics, Information and Communication Engineers, Vol.74, No.
3, pp. 271-279, 1991 "

[0005]

The conventional techniques described above have the following problems. -Regarding object extraction, it is difficult to extract objects due to the time and effort of each processing step and noise removal work. -Extraction of colors and compositions is general, but flexible modeling suitable for each image has not been reached. -Texture analysis is effective depending on the processing purpose and target image, but it is not general purpose. -The conventional method cannot interpret an image according to each user's viewpoint and purpose.

[0006] That is, in the modeling by the conventional image processing technique, it is difficult to extract an object for each processing step, work such as noise removal, which requires high accuracy and precision, and labor required for the work. Further, the extracted object is not a feature amount that expresses the impression received from the image, and is not sufficient as an element that embodies the human sense.

On the other hand, the kansei engineering approach of expressing with features such as color and composition, and the modeling method with index words that characterize the image are suitable as elements for expressing the impression received from the image. However, flexible modeling that matches the features of each image has not been realized. In other words, a flexible modeling framework corresponding to each image or adapted to the purpose of use has not yet been established. Furthermore, no technique has been established for preserving the cuts and viewpoints that characterize the modeling specialized for those images for later use.

The present invention has been made in consideration of the above circumstances, and when extracting a sensible feature amount from image data,
An object of the present invention is to provide an image processing device and an image processing method that can be flexibly adapted to the purpose, individual, situation, and the like.

[0009]

SUMMARY OF THE INVENTION The present invention is an image processing apparatus for inputting image data and extracting a feature amount indicating the emotional expression content when the image data is displayed. It is possible to specify the angle and position of a dividing line that divides the image data as the feature amount by performing a dividing process that divides the input image data by a first input unit that inputs and the input image data on the basis of sensitivity. Extracting means for extracting the dividing line information, the first constraint condition regarding the dividing order in the dividing direction in the dividing process, and the second constraint regarding the range of the angle allowed for the dividing line in each dividing direction. And a second input unit for inputting constraint condition information indicating a condition, wherein the extracting unit extracts the dividing line information within the range of the first and second constraint conditions. .

[0010]

[0011]

[0012]

[0013]

It should be noted that the present invention relating to the apparatus also holds as the invention relating to the method, and the present invention relating to the method also holds as the invention relating to the apparatus. Further, the present invention relating to an apparatus or a method is provided for causing a computer to execute a procedure corresponding to the present invention (or for causing a computer to function as means corresponding to the present invention, or for a computer to have a function corresponding to the present invention. It also holds as a computer-readable recording medium in which a program (for realizing it) is recorded.

In the present invention, the following means are used. Image data is modeled based on color and / or composition, and image color and / or composition information of the image is extracted as a feature amount. Setting corresponding parameters for each image and / or parameters for operating the entire modeling system, and operating these to extract, for example, image color of the image and / or composition information having an arbitrary shape as a feature amount. Save the feature quantity of the image modeled based on the parameters (may be saved with the parameter set).

As the color modeling method, for example, the following method can be used. Considering one similar color appearing in the image as one, the number of target colors is narrowed down, and the image color is determined from the used colors based on the degree of conspicuity with respect to each target color. In narrowing down the number of colors, a threshold value of color mixture for determining two colors as similar colors and a limited number of colors can be used as parameters. The method for narrowing down the number of colors may be another method such as the PA method or the MCA method. Also, in image color selection, the size of the appearance area of the color and the ratio to the high attractiveness and the high contrast feeling are used as constraint parameters to calculate the degree of conspicuousness, and the high attractiveness that is the factor is calculated. To seek
Hue, saturation, and ratio to lightness can be used as constraint parameters. The number of colors selected to determine the final image color can also be a parameter.

In the modeling method based on the composition, for example, the following method can be used. Region segmentation by a straight line is the basic segmentation method, and each segment is freely segmented under pre-specified constraints, and those segments are recursively repeated as long as the segment conditions are satisfied. It should be noted that not only a straight line but also a circular or radial area dividing method is conceivable, and the dividing condition may be another method such as comparing the color averages of divided neighboring areas. In the linear area division, the division direction and the position can be used as constraint parameters.

According to the present invention, when a set of still image data is modeled on the basis of color or composition, each image data is adjusted by adjusting various parameters.
It enables flexible modeling corresponding to individual image data. It is also possible to save those parameter sets and adapt the image data to a particular purpose, individual or situation.

[0019]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

The present invention adjusts various parameters in modeling and converts the image data into a feature amount having an abstract representation. In the following, in the case of converting into a color-based feature amount, and A case of converting into a feature amount based on composition will be described as an example. It should be noted that the specific example of the process, the specific example of the parameter, the specific example of the image data, the specific example of the processing result, and the like illustrated in the present embodiment are merely examples, and are not of course limited to those illustrated in the present embodiment. .

FIG. 1 shows an example of the configuration of the image modeling apparatus of this embodiment.

As shown in FIG. 1, this image modeling apparatus includes an image data input section 1, a parameter input section 2, a color list creating section 3, a registered color limiting section 4, a conspicuous degree calculating section 5, and an image color extracting section. It includes a unit 6, a constraint condition reading unit 7, a division direction / position determination unit 8, a recursive division application unit 9, a region feature amount calculation unit 10, an extraction result display unit 11, and a data storage unit (not shown).

The data storage unit stores the input image data,
It stores the generated image color feature amount, the input parameter, the input constraint condition, and the generated composition feature amount, and is composed of, for example, a hard disk, an optical disc, a semiconductor memory, or the like. The data storage unit may be configured by a separate storage device for each type of data, or may be configured to store all or part of various types of data in the same storage device.

The image modeling device may have a function of exchanging information with a communication partner via a network.

This image modeling device can be realized by executing software on a computer. In this case, an OS having a desired function, software such as driver software, packet communication software, encryption software, and the like are installed as needed. In addition, in this case, a graphical user interface (GUI) is used for inputting information from the user and presenting information to the user.
Is preferably used.

FIG. 2 shows an example of the processing procedure of the image modeling apparatus. The image modeling device in FIG. 1 has a function of converting image data into a feature amount based on color and a function of converting image data into a feature amount based on composition, but the procedure example of FIG. Is an example of a case in which both conversion into a feature amount based on color (steps S2 to S4) and conversion into a feature amount based on composition (steps S5 to S7) are performed. Note that the conversion to the feature amount based on the color and the conversion to the feature amount based on the composition may be performed one first and the other later, or may be performed alternately in predetermined processing units, You may process simultaneously or in parallel. Figure 3
FIG. 2 shows an example of a detailed processing procedure for image color extraction in the processing of step S2 in FIG. 2, that is, in color-based modeling. FIG. 4 shows an example of a detailed processing procedure for cutting out a composition in the processing of step S5 in FIG. 2, that is, modeling based on the composition.

First, in this image modeling apparatus, the image data (12) is input by the image data input unit 1 (step S1). Various input forms are possible such as keyboard input, online input via a network, reading from a magnetic tape or a magnetic disk or an optical disk which is a medium for transmitting information (the input form is fixed to one type and is configured. The input form may be selected from a plurality of input forms).

As image data, a film photograph is digitized, a digital photograph, an analog VTR.
1 frame of the digital VTR, 1 frame of the digital VTR, CG data, digitalized hand-drawn picture or drawing, etc. may be used.
Further, in the present embodiment, it is assumed that the image data is a color image, but of course, the black and white image data also has a characteristic amount relating to color, and the black and white image data is not excluded from the processing target. . In the present embodiment, the case where the digitized image data is input from the image data input unit 1 has been described as an example, but the image modeling apparatus is the source of the image data and is still digitized. You may provide the function which digitizes and takes in what is not done.

FIG. 5 shows an example of image data. This example is image data mainly including color images of "coast", "road (along coast)", and "automobile (along road)".

After the image data is input, the image color is determined (step S2) and the composition is cut out (step S5).

First, the determination of the image color (step S2) will be described with reference to FIG.

First, the parameter input section 2 inputs the parameter information (14) in a predetermined input form such as keyboard input, online input via a network, or reading from a magnetic tape which is a medium of information transmission.
It is input (step S11).

FIG. 6 shows an example of parameter information. The parameter r is a threshold value of color mixture, the parameter c is a limited number of colors, and the parameter v (s, t, u) is an influence ratio (hue, saturation, lightness) for calculating the attractiveness. And the parameter m (e, f,
g) is an influence ratio for calculating the degree of conspicuity (the influence ratio on the size of the color area, the high attractiveness, and the high contrast feeling), and the parameter i is the specified number of colors.

Next, the color list creating section 3 creates a color list for the input image data.

First, the color list creating section 3 divides the image data as shown in FIG. 5 into pixels, and reads the color information corresponding to each pixel (step S1).
2). Each color has three primary colors R (red), G (green), B (blue),
It is represented by each 8-bit gradation. These values are converted into the L * a * b * space based on a known coordinate conversion formula.

Next, the color list creating section 3 uses L * a *.
A color list composed of registered colors represented by b * space coordinate values and the number of appearances obtained by summing the number of pixels for each of them is created (step S13).

However, in the present embodiment, when a color list is created, a process of color mixture is performed instead of simply collecting the same number of vectors after being converted into the L * a * b * space. Try to do it.

First, L * of the color of each pixel of the input image data
One a * b * space vector value is read, and the three-dimensional vector value and the number of appearances = 1 are registered in the color list.

After that, L of the color of each pixel of the input image data is set.
Regarding the * a * b * space vector values, one is read from the unprocessed one, and the vector value of the newly read color and the vector value of the registered color that has already been registered in the processing up to that point in the color list , Euclidean distance in L * a * b * space (when there are multiple registered colors, the minimum value)
If the calculated Euclidean distance is larger than the color mixture threshold (parameter r), it is determined that there is no color close to the color list, and the three-dimensional vector value of the newly read color and the number of appearances = 1 Register in the color list. On the other hand, if the calculated Euclidean distance is equal to or less than the color mixture threshold (parameter r), it is determined that the colors are close to each other, and the newly read color and the registered color having the Euclidean distance are mixed as described later. When color mixing is performed, the three-dimensional vector value of the registered color before the color mixing and its appearance number are invalidated, and instead, the color after the color mixing is set as the registered color, and the three-dimensional vector value and its appearance number = color mixture Register the previous number of appearances + 1 in the color list.

The above is the L of the color of each pixel of the input image data.
This is repeated until all the * a * b * space vector values are read and processed.

The color of each pixel of the input image data is L *
The order of reading the a * b * space vector values one by one may be appropriately determined. Further, in the above, when there are a plurality of registered colors, the minimum value of the Euclidean distance is calculated.However, the Euclidean distance becomes equal to or less than the threshold value of the color mixture only while the Euclidean distance with each registered color is being calculated. It is also possible to cancel the calculation when the color is changed and to adopt the registered color at that time. Further, in the above, the color list is created by reading all the L * a * b * space vector values of the color of each pixel of the input image data. L * a *
It is also possible to read the b * space vector value and create a color list.

Here, the color mixing method will be described.

Registered color B mixed with newly read color A
Let a and b be three-dimensional coordinate vectors in the space of
When the coordinate vector of the color X (new registered color after color mixing) X generated as a result of the color A and the color B being mixed is x, x is defined by the following color. _{^{x → = w A · a →}} + w B · b → _{here, w A = N A ÷ (} N A + N B) w B = N B ÷ (N A + N B) N A: the number of occurrences N _B of color A : The number of appearances of color B. Further, the number of occurrences N _x of the generated result of the mixing color X is _{N x = N A + N B} . However, in the present embodiment, N _A = 1 because the colors are read and processed one by one.

In the above description, one color is read for each input image data to create one color list. However, for example, a color list is created by using half the pixels of the input image data. A color list created by the same method as above using the remaining half of the pixels of the input image data (by further performing color mixing processing, etc.) into a single color list, etc. , Other methods are possible.

Next, in the created color list,
Registered colors whose number of appearances is equal to or less than a reference (for example, equal to or less than the number corresponding to 0.5% of the total number of pixels) are regarded as noise, and information regarding the corresponding registered color is deleted from the color list (step S14).

Next, at this time, the number of registered colors registered in the color list is checked, and if it is larger than the limited number of colors (parameter c) (step S15), the process of limiting the registered colors is executed. Perform (step S16). If the number of registered colors is less than or equal to the limited number of colors (parameter c), the process of limiting the registered colors is skipped.

In the process of limiting the registered colors, the registered color limiting unit 4 performs the color mixing process until the number of registered colors in the color list becomes equal to or less than the limited number of colors (parameter c) (in one color mixing process, registration is performed. One color can be reduced). That is, at each time point, the two closest colors among all registered colors, that is, L * a *
The two colors that minimize the Euclidean distance on the b * space are searched for, the colors are mixed according to the color mixing method described above, and the color list is updated. This is repeated until the number of colors registered in the color list becomes equal to or less than the limited number of colors (parameter c).

Next, the registered color limiting unit 4 determines the used color for the input image data from the registered colors registered in the color list.

The processes of steps S17 to S19 are repeated.

First, the registered color limiting section 4 divides each axis of L * a * b * into regions, and creates a histogram according to the number of appearances in each region for each axis of L * a * b * ( Step S17). It should be noted that the method of area division may be appropriately determined.

Next, in the histogram corresponding to the L axis, the number of convexes formed by the graph connecting the number of appearances of each area (FIG. 7 shows an example in which the number of convexes is 3) and the histogram corresponding to the a axis In, the number of convexes formed by the graph connecting the number of appearances of each area and the number of convexes formed by the graph connecting the number of appearances of each area in the histogram corresponding to the b-axis are examined, and The number is used as the number of colors used in the input image data, and the number of colors used is stored (step S1).
8).

Next, the number of colors used this time is compared with the number of colors used last time, and if there is a change (step S19),
It returns to step S17. In the first time, since there is no previous number of used colors, it is considered that the number of used colors has changed (or, for example, the initial value of the number of used colors may be set to 0).

When the process returns to step S17, step S18 is repeated until the number of used colors obtained in step S18 is reached.
Similar to step 16, the closest colors in the color list are mixed, a histogram is created again, and in step S18, the number of used colors of the input image data is again determined and stored.

The above process is repeated until it is determined in step S19 that the number of colors used has not changed.

If it is determined in step S19 that the number of colors used this time has not changed from the number of colors used last time, this loop processing ends, and the colors registered in the color list at this point are It has been decided as the color to be used for the input image data.

Next, the salient degree calculation unit 5 calculates the salient degree with respect to the used color of the input image data (that is, the color registered in the color list).

First, the conspicuousness degree calculation unit 5 targets the colors registered in the color list from the L * a * b * coordinate system and considers that the HSV is closer to the human sense.
(Hue: Hue, Saturation: Saturation, Val
ue: brightness) conversion to the coordinate system (step S2)
0).

Next, based on the influence ratio (parameter v (s, t, u)) on each of hue, saturation, and lightness, the attractiveness C ₂ which is an element of the degree of conspicuity is calculated as follows. (Step S21). C ₂ = (s × influence value by hue + t × saturation + u × lightness) ÷ (s + t + u) (1) The influence value used here corresponds to the range of the hue H value.

Next, the image color extracting section 6 selects an image color.

First, the image color extraction unit 6 determines the degree of conspicuousness L with respect to the colors registered in the color list, and the influence ratios on the size of the color area, the high attractiveness, and the high contrast feeling ( Based on the parameter m (e, f, g)), the calculation is performed as follows (step S22). _{L = ((e · C 1} ) 2 + (f · C 2) 2 + (g · C 3) 2) 1/2 ... (2) where, in the region of interest color for each of the registered color colorist The size C ₁ and the contrast C ₃ are respectively
It is defined as the ratio of the number of appearances to the total number of pixels and the difference in brightness from other colors. C ₂ is the above-mentioned high attractiveness.

As a result, the noticeable degree L for each registered color in the color list is obtained.

Then, the image color extracting section 6 selects the registered colors of the color list for the input image data from the ones having a high degree of conspicuousness L, as many as the image colors designated by the designated color number (parameter i). (Step S23).

The method of narrowing down the number of colors is not limited to the above-described color mixing method, but is the PA method (reference 9 “P. Heckbe”).
rt: Color Image Quantizati
on for Frame Buffer Display
y, Computer Graphics, Vol.
16, No. 3, 1982 ") and the MCA method (Reference 1)
0 "Suzuki Suzuki, et al .: Limited color display method with flexibility in color selection, Journal of Television Society, Vol. 43, No. 3,
pp. 268-275, 1989 ") and other techniques can also be used.

Further, in the above, the conspicuous degree is used as the evaluation value for selection, but other methods are possible.

Subsequently, the composition cutting (step S5) will be described with reference to FIG.

First, the input image data (see FIG. 5) is divided within the constraint conditions.

First, the constraint condition reading unit 7 inputs the constraint condition (15) in a predetermined input form such as keyboard input, online input via a network, or reading from a magnetic tape which is a medium for transmitting information ( Step S31).

FIG. 8 shows an example of constraint conditions.

In this example, [vert, ...] Specifies vertical division (upper and lower division), [horiz, ...] specifies horizontal division (left and right division), and [daig, ...] specifies diagonal division. Assuming that [vert, *, [horiz, *, ni
l, nil, *], [daig, *, nil, nil,
*], *] Can be specified in a nested structure. [Vert, *, [horiz, *, nil, n
il, *], [daig, *, nil, nil, *],
*] First performs division according to the outermost described vert, that is, upper and lower division, and then precedes the divided area corresponding to the upper side of the input image data by one inside thereof. Division according to horiz, that is, left-right division, and division according to daig described below, that is, diagonal division, is performed on the divided area corresponding to the lower side of the input image data. This is an example of instructing to do. The outermost hor
If iz or daig is described, hor
The division according to iz or daig is performed, then the division area corresponding to the left side of the input image data is divided according to the preceding description inside, and the division area corresponding to the right side of the input image data is applied. It suffices to divide the divided area according to the following description into the divided area.
Further, such a nested structure can be described in any number of layers. Note that nil means that no further division is performed, and * may be omitted.

In this example, "vert": vertical division (0≤angle≤30, 150≤angle≤18)
0), “horiz”: horizontal division (60 ≦ angle ≦ 1
20), “daig”: diagonal division (30 ≦ angle ≦
60, 120 ≦ angle ≦ 150) is a constraint on the angle attached to the dividing line in each division.

Next, the division direction / position determining unit 8 determines the division direction and position (parameter) based on these pieces of information designated in advance (within the range of the designated list structure and constraint angle). It is determined (step S32).

When the constraint condition illustrated in FIG. 8 is specified, the angle is provided with three types of constraints of vertical, horizontal, and diagonal, which are defined as follows. Vertical division: 0 ≦ angle ≦ 30, 150 ≦ angle ≦
180 Horizontal division: 60 ≦ angle ≦ 120 Diagonal division: 30 ≦ angle ≦ 60, 120 ≦ angle
e ≦ 150 In the first division, vertical division is performed according to the list structure of FIG. Here, when the vertical division is performed within the range of the above-mentioned angle constraint, the position of the dividing line that maximizes the difference between the hue of one of the divided areas and the hue of the other area is obtained, and the division parameter is set to decide.

Here, the division parameter is a predetermined one point (eg, x) on the division line based on the angle of the division line and relative coordinates from the reference point (eg, upper left point) of the input image data.
Section or y-section). Of course, other formats such as two predetermined points (for example, x-intercept and y-intercept) on the dividing line based on the relative coordinates from the reference point (for example, the upper left point) of the input image data may be used.

In the next division, according to the list structure of FIG.
Horizontal division within the above angle constraint range is performed on the upper divided area obtained by vertical division, and diagonal division within the above angle constraint range is performed on the lower divided area. Is done. Again, in each division,
The position of the dividing line that maximizes the difference between the hue of one of the divided areas and the hue of the other area is determined, and the division parameter is determined.

If the following nesting is present in the list structure, the same processing is repeated.

By the way, if the dividing line is determined under the constraint condition and there is no further nesting in the list structure (that is, the division within the constraint condition is completed) (step S33), the process is repeated. The process ends.

The obtained division parameter (14) is stored in a predetermined format.

It is also possible to describe the obtained division parameter in the corresponding portion of the list structure of FIG. FIG. 9 shows an example in which the angle and position of the obtained dividing line are described in the list structure of FIG. In FIG. 9, for the vertical dividing line, angle = 10.0, y intercept = 5
0, for the horizontal division line in the lower layer, angle = 80.
0, x intercept = 50, for diagonal dividing line, angle = 4
It is described that 5.0 and y-intercept = 200.

FIG. 10 shows an example in which the input image data of FIG. 5 is divided.

Subsequently, the above division result is subjected to a division process which is not restricted by the condition.

Here, in step S31, the parameter input section 2 reads the parameter information (14) for each divided area from the magnetic tape, which is a keyboard input, online input via a network, or an information transmission medium. It is assumed that the input is made in a predetermined input form. Note that this input may be performed after the end of step S33.

FIG. 11 shows an example of parameter information.
The parameter p1 is the change angle width within the angle constraint, the parameter p2 is the movement width of the dividing line, and the parameter p is
3 is the minimum indivisible area area, and the parameter p4
Is the threshold of division strength. In this example,
The parameter p1 and the parameter p2 are common to the vertical, horizontal, and diagonal divisions, but the parameter p1 and the parameter p2 may be specified for each of the vertical, horizontal, and diagonal divisions.

The recursive division applying section 9 changes the angle in the vertical direction within a given constraint range (parameter p1), and also inserts a division line for each given movement width (parameter p2). The position of the dividing line that maximizes the strength of division (described later) between one of the divided areas and the other area is obtained (step S34). The recursive division application unit 9 also performs the same processing in the vertical direction (step S35). Also, the same processing is performed in the diagonal direction (step S36). Note that the three processes may be performed in sequence, or may be performed alternately in predetermined process units,
You may perform simultaneously or in parallel.

Then, of the division strengths corresponding to the division lines respectively obtained by the divisions in the three directions, the position of the division line having the maximum division strength is selected (step S37). .

Here, both of the two divided regions are
Is less than a given constant area (parameter p3),
Alternatively, if the division strength corresponding to the selected dividing line is
If it is less than or equal to the given threshold value (parameter p4) (step S38), the division is ended. On the other hand, in other cases (step S38), for each divided area,
The division is continued in the same manner as above.

The above judgment is performed for each of the divided areas, and for the areas satisfying the condition for continuing the division (when the number of conditions is 0, 1 and 2). In some cases), the division may be continued in the same manner as above.

The obtained division parameter (14) is stored in a predetermined format in the same manner as in the case of the division processing within the constraint conditions described above.

As described above, when the obtained division parameter is described in the corresponding part of the list structure of FIG. 8, it corresponds to the corresponding part of the list structure. Just insert a nested structure. For example,
In the corresponding nil part in FIG. 9, [angle, position, ni
l, nil, *] may be inserted.

FIG. 12 shows an example of a result obtained by performing a condition-independent division process on the division result of FIG.

Here, a processing example in the recursive division applying section 9 will be described in more detail.

The division strength S (t) is defined as follows, for example. Region C at position t with respect to certain area I
When divided into ₁ and region C _2, the dispersion of color in the entire area region I, the division to maximize the dispersion of color between the region C ₁ and the region C _2, and good split. That is, with respect to the area C ₁ , the area C _2, and the area area I, the average value of the RGB spatial coordinates of all the pixels existing in the area is μ
₁ (t), μ ₂ (t), μ _T, and region C ₁ and region C ₂
If the variance between and is σ _B and the overall variance is σ _T , then
The strength of division is calculated as follows. S (t) = σ _B ² _/ σ _T ² σ _B ² = w ₁ (t) | μ ₁ (t) −μ _T | ² + w
₂ (t) | μ ₂ (t) −μ _T | ² where w ₁ (t) and w ₂ (t) are the sizes of the region C _{1 and the} region C ₂ with respect to the entire area region I, respectively. .

The recursive division application section 9 performs the following processing. (1: Vertical division) With respect to the area area I of the input image data of size M × N, a dividing line is inserted for each width m in the horizontal direction to divide the area. The division strength S _m (t) of each division line is obtained. Angle constraint (for example, 0 ≦ angle ≦ 30, 150
The angle is changed within the range of ≦ angle ≦ 180) and this is repeated. The maximum vertical division strength S _M and the division position t at that time are obtained.

(2: Horizontal division) Similarly, regarding the vertical direction,
A dividing line is inserted for each width n, and the dividing strength S of each dividing line is
_{Find n} (t). Angle constraint (60 ≦ angle ≦ 12
This is repeated by changing the angle within the range of 0). The maximum horizontal division strength S _N and the division position t at that time are obtained.

(3: Diagonal division) Similarly, in the diagonal direction, a dividing line is inserted for each width k, and the dividing strength S of each dividing line is S.
_{Find k} (t). Angle constraint (30 ≦ angle ≦ 6
The angle is changed within the range of 0, 120 ≦ angle ≦ 150), and this is repeated. The maximum strength S _{L of the} diagonal division,
The division position t at that time is obtained.

(4) S _M (t), S _N (t), S
_If S _M (t) is the maximum of _L (t), then S _M
(T) is divided at the position t, and if S _N (t) is maximum, it is divided in the vertical direction at the position t that gives S _N (t), and S
_{If L} (t) is the maximum, division is performed at a position t that gives S _L (t) in an oblique direction.

(5) Repeat until no further division is possible (the area is below a certain area or there is no division position).

Next, the area feature amount calculation section 10 determines, for all or some of the partial areas of the input image data, which are divided according to the finally obtained division parameter (14), attribute information relating to the partial areas. Is calculated as a feature amount (step S39). The partial region is, for example, a divided region on one side divided into two in any one of the division stages. The attribute information is, for example, the average of RGB spatial coordinates of all pixels existing in a predetermined divided area.

The attribute information of this partial area may be described in the corresponding portion of the list structure of FIG. An example of this case is shown in FIG. In FIG. 13, the average of the RGB vector values of all pixels in the upper half part of the first vertical division is (128: 60:
80), and RG of all pixels in the lower half partial area
It shows that the average of the B vector values is (255: 128: 25).

In the above description, the case where the line is divided vertically, horizontally and diagonally has been described as an example. However, a circle or radial region dividing method is also possible, and the dividing condition is that the color average of the divided neighboring regions is used. It is also possible to use other methods such as comparison.

Further, in the present embodiment, the input image data is divided within the constraint condition, and then the condition-independent division is performed.
It may be possible to specify whether the former is performed and the latter is not performed, the latter is performed without performing the former, or both are performed.
Further, the image modeling device may be provided with a function of performing either one of the division of the input image data within the constraint condition and the division regardless of the condition.

When the image color is determined as described above (step S2), the obtained image color feature amount (13) is associated with the input image data or its identification information, for example. , Is stored in a predetermined database or the like (step S3). The image color feature amount is, for example, a vector value of the selected registered color, an evaluation value such as a degree of conspicuity, information on the number of appearances,
All or part of information such as ranking. For the vector value of the registered color, for example, a method using HSV space coordinate values,
There are a method of using RGB space coordinate values, a method of using Lab space coordinate values, and a method of writing any two or three of them.

Similarly, when the composition is cut out (step S5), the obtained composition feature amount (16) is stored in a predetermined database or the like, for example, in association with the input image data or its identification information. (Step S
6). The composition feature amount is, for example, a dividing line parameter and attribute information about the partial area. Depending on the purpose,
Only one of the dividing line parameter and the attribute information regarding the partial area may be stored (in the case of storing only the former, the area feature amount calculation unit 10 may be omitted).

When the image color characteristic amount and the composition characteristic amount are stored, the parameter information from which the characteristic amount is obtained or the information for specifying the parameter information may be stored together.

The image color feature amount and the composition feature amount created and saved as described above are generated when they are created, when they are designated by the user, or when a specific application program is operated.
The extraction result display unit 11 displays it in a predetermined display form (steps S4 and S7). When both the image color feature amount and the composition feature amount are stored for a certain image data, both may be displayed sequentially or simultaneously, for example, only the one designated by the user may be displayed. May be.

The result of color-based modeling may be displayed as illustrated in FIGS. 14 and 15, for example. 14
Is an example of actual color display (note that the hatching difference in FIG. 14 is for explaining the difference in display color), and FIG. 15 is an example of numerical display. Further, color display and numerical display may be used together.

Also, for example, the user may be allowed to specify the number of colors to be displayed as image colors. Also,
For example, the user may be allowed to select a desired one from a plurality of display modes. In addition, various variations of the display method are possible.

The result of modeling based on the composition may be displayed as shown in FIG. 12, for example. In this case, the dividing line is actually displayed on the image data. There is also a method of filling the corresponding partial area with the average value of the corresponding RGB space coordinates. In addition, various variations of the display method are possible.

By the way, in the above, both the conversion into the feature amount based on the color and the conversion into the feature amount based on the composition are performed, but for example, the user only performs the conversion into the feature amount based on the color. It is also possible to adopt a configuration in which either of the conversion into the feature amount based on the color, the conversion into the feature amount based on the color, and the conversion into the feature amount based on the composition can be selected.

It is also possible to adopt a configuration in which the image modeling device is capable of performing only either conversion into a feature amount based on color or conversion into a feature amount based on composition. When the function to convert to the feature amount based on the color is provided but not to the function to convert to the feature amount based on the composition, the constraint condition reading unit 7, the division direction / position determination unit 8, the recursive in the configuration described so far. Automatic division application unit 9, area feature amount calculation unit 10,
The extraction result display unit 11 may be omitted. When the function for converting to the feature amount based on the composition is provided and the function to convert to the feature amount based on the color is not provided, in the configuration described so far, the color list creating unit 3, the registered color limiting unit 4, and the prominence degree calculation The unit 5 and the image color extraction unit 6 may be omitted.

In the above, an example of obtaining the feature amount based on the color or the feature amount based on the composition is shown. However, the feature amount based on the object, the feature amount based on the texture, etc.
It is also possible to obtain another feature amount.

The feature quantity obtained by the image modeling apparatus can be used for various information processing such as retrieval and classification.

Each of the above functions can be realized as software. Further, the present embodiment is readable by a computer in which a program for causing a computer to execute a predetermined means (or for causing a computer to function as a predetermined means or for causing a computer to realize a predetermined function) is recorded. It can also be implemented as a recording medium.

Note that the configuration illustrated in the present embodiment is an example, and is not intended to exclude other configurations, and a part of the illustrated configuration may be replaced with another one or one of the illustrated configurations may be omitted. Other configurations that are obtained by omitting parts, adding other functions to the exemplified configurations, or combining them are possible. Further, another configuration that is logically equivalent to the illustrated configuration, another configuration including a portion that is logically equivalent to the illustrated configuration, and another configuration that is logically equivalent to the main part of the illustrated configuration are possible. is there. In addition, another configuration that achieves the same or similar purpose as the exemplified configuration,
Other configurations that have the same or similar effects as the exemplified configurations are possible. Further, in the present embodiment, various variations of various components can be appropriately combined and implemented. Further, each embodiment is an invention as an individual device, an invention about a component inside the individual device, or an invention of a method corresponding to them, etc.
It is intended to include and embrace inventions relating to various viewpoints, stages, concepts or categories. Therefore, the invention disclosed in the embodiments of the present invention can be extracted without being limited to the exemplified configurations.

The present invention is not limited to the above-described embodiments, but can be implemented with various modifications within the technical scope thereof.

[0115]

According to the present invention, the modeling method can be manipulated by adjusting various parameters to adapt the image data to a particular purpose, individual or situation.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration example of an image modeling device according to an embodiment of the present invention.

FIG. 2 is a flowchart showing an example of a processing procedure according to the embodiment.

FIG. 3 is a flowchart showing an example of a processing procedure for extracting an image color in color-based modeling according to the first embodiment.

FIG. 4 is a flowchart showing an example of a processing procedure for cutting out a composition in modeling based on the composition in the embodiment.

FIG. 5 is a diagram showing an example of an image which is an input to the image modeling device in the embodiment.

FIG. 6 is a diagram showing an example of parameter information which is an input to the color-based modeling in the embodiment.

FIG. 7 is a diagram for explaining an example of processing related to a histogram according to the same embodiment.

FIG. 8 is a diagram showing an example of a constraint condition which is an input to modeling based on the composition in the embodiment.

FIG. 9 is a diagram showing an example of describing a processing result in a list structure according to the same embodiment.

10 is a diagram showing an example in which the input image data of FIG. 5 is divided.

FIG. 11 is a diagram showing an example of parameter information which is an input to modeling based on the composition in the embodiment.

FIG. 12 is a diagram showing an example in which the input image data of FIG. 10 is further divided.

FIG. 13 is a diagram showing an example of describing a processing result in a list structure according to the same embodiment.

FIG. 14 is a view showing a display example of image color feature amounts in the same embodiment.

FIG. 15 is a diagram showing a display example of image color feature amounts in the same embodiment.

[Explanation of symbols]

1 ... Image data input section 2 ... Parameter input section 3 ... Color list creation section 4 ... Registered color limited section 5 ... Conspicuousness calculation section 6 ... Image color extraction unit 7 ... Constraint condition reading unit 8 ... Division direction / position determination unit 9 ... Recursive division application unit 10 ... Area feature amount calculation unit 11 ... Extraction result display section

─────────────────────────────────────────────────── ─── Continuation of the front page (56) References Yudai Morohara, 3 people, Selection of image color for textile design images, Journal of Information Processing Society of Japan, February 15, 1995, Vol. 36, No. 2, pp. 329-337 Hidenori Yamamoto, 3 people, Similar image retrieval considering spatial distribution of color information, IEICE Technical Report IE98-155-169, February 3, 1999, Vol. 98, No. 576, pp. 45-50 Kaoru Yoshida, Proposal of database retrieval system considering subjective background, IEICE Technical Report MVE97-40-56, July 24, 1997, Vol. 97, No. 206, pp. 17-24 (58) Fields surveyed (Int.Cl. ⁷ , DB name) G06T ⁷ /00-7/60 G06T 1/00 G06F 17/30

Claims (9)

(57) [Claims] 1. Inputting image data and displaying the image data
Extracts the features that indicate the emotional expression content
In the image processing apparatus, the first input unit for inputting the image data and the dividing process for dividing the input image data by a criterion relating to the sensitivity are used as the feature amount.
It is possible to specify the angle and position of the dividing line dividing the data
The extraction means for extracting the dividing line information, the first constraint condition regarding the dividing order in the dividing direction in the dividing process, and the dividing line relating to each dividing direction.
And the second constraint condition regarding the range of allowable angles.
And a second input means for inputting to constraint information, the extracting means, within said first and second constraints, you and extracting the divided WASH paper image Image processing device. Wherein said extraction means, claim 1, characterized in that repeated between predetermined splitting condition is satisfied, the recursively division
The image processing device according to item 1. 3. The division condition is that the area of the divided area is equal to or larger than a threshold value, or the distribution of the hue in the area before the division and the hue between the two areas after the division. The image processing apparatus according to claim 2 , wherein an evaluation value that indicates the strength of division defined by variance is equal to or greater than a threshold value. 4. The image processing apparatus according to claim 1 , wherein the extraction means extracts attribute information about a color of a partial area obtained by dividing the image data. 5. The first constraint condition is that the first division is performed.
Such division direction and each partial area obtained by this division
Describe the division direction for the next division for each area.
The image according to claim 1, characterized in that
Image processing device. 6. The first constraint condition is division performed in advance.
Direction of division, and each partial obtained by this division
Division direction for subsequent division for each area
Is described over multiple layers.
The image processing apparatus according to claim 1, wherein the image processing apparatus is provided. Wherein said dividing direction is a direction to divide the direction of dividing the image data next to the dividing line, the direction to divide the image data in the vertical dividing line, or the image data at an oblique dividing lines, wherein the second constraint is to describe a range of angle allowed to the next acceptable angle range dividing line, acceptable angular range to the vertical dividing line, and the diagonal dividing line
The image processing device according to claim 1, characterized in that
Processing equipment. 8. Inputting image data and displaying the image data
Extracts the features that indicate the emotional expression content
An image processing method including: a first input step of inputting image data, and dividing the input image data by a criterion relating to sensitivity
Image data as the feature amount.
It is possible to specify the angle and position of the dividing line that divides the data
The extraction step of extracting the dividing line information and the division order in the division direction in the division processing
Regarding the first constraint condition regarding the
And the second constraint condition regarding the range of allowable angles.
Second input step for inputting constraint condition information
However, in the extracting step, the range of the first and second constraint conditions is
An image characterized by extracting the dividing line information in the area
Image processing method. 9. Inputting image data and displaying the image data
Extracts the features that indicate the emotional expression content
To make a computer function as an image processing device
A computer-readable record that records the program
A medium, a first input step of inputting image data, and dividing the input image data by a criterion relating to sensitivity.
Image data as the feature amount.
It is possible to specify the angle and position of the dividing line that divides the data
The extraction step of extracting the dividing line information and the division order in the division direction in the division processing
Regarding the first constraint condition regarding the
And the second constraint condition regarding the range of allowable angles.
The second input step for inputting the constraint condition information.
Computer, and in the extracting step, the first and second constraint conditions
Program for extracting the dividing line information within the range
A computer-readable recording medium in which a program is recorded.