JP2015049748A - Image processing apparatus, image processing method and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus capable of determining a parameter for defining the colors of texture of an arbitrary object through a color space constituting an image by a simple user input.SOLUTION: The image processing apparatus includes color space analysis means for analyzing a color space of an image, color group division means for dividing the color space into color groups on the basis of an analysis result of the color space, color group allocation means for allocating the respective divided color groups to a component of an inputted three-dimensional model or a component group, and parameter determination means for determining a parameter for defining the colors and texture of the three-dimensional model on the basis of features of the respective allocated color groups.

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program.

  In the field of 3D computer graphics, a 3D object is a geometric surface consisting of a flat surface such as a polygon, a free-form surface such as a NURBS (Non-Uniform Rational B-Splines) surface, or a volume such as a voxel. A two-dimensional image is generated by expressing a set of elements and observing these geometric elements. Hereinafter, this geometric element is referred to as a primitive, a three-dimensional object composed of a set of primitives is referred to as an object, and observation is referred to as rendering.

  A primitive may be a very simple point, surface, or solid, or it may be a curved surface or solid represented by a complicated mathematical expression. In rendering, the appearance of an object, that is, the color and texture of the surface, is determined depending on the observation direction and the light source direction based on the optical characteristics of the object. As a method for simply expressing the optical characteristics of an object, generally, a method of defining a certain reflection model for a primitive constituting the object and calculating a three-dimensional appearance based on the reflection model is widely used. Depending on the reflection model, there may be a single parameter or a plurality of parameters used in this reflection model, and the optical characteristics of the object can be expressed by controlling the parameters. Hereinafter, this set of parameters is referred to as material. The parameters constituting the material vary depending on the reflection model and the rendering method, and in the simplest case, the parameters may represent a single object color.

Here, an example in which the primitive is a polygon will be described with respect to the expression of the optical characteristics by the material. A polygon is a three-dimensional plane and is generally represented by a set of three or more vertices having three-dimensional coordinates. In the following, when rendering one polygon based on the Phong reflection model under one light source, the point I _p on the polygon having the normal vector N is in the direction of the vector V from the observation viewpoint, and the light source from I _p When the direction to is represented by a vector L and the complete reflection direction is R, Equation (1) defining the color and shadow at I _p is shown.
I _p = k _a i _a + k _d (L · N) i _d + k _s (R · V) ^∝ i _s (1)

(1) In the _{formula, i a, i d, i} s is a coefficient of a light source color, respectively environmental reflection, diffuse reflection, determining the source color in each specular reflection. k _a , k _d , and k _s are coefficients indicating the reflection color of the polygon, and determine reflection colors in environment reflection, diffuse reflection, and specular reflection. ∝ determines the reflection intensity of the polygon. When the reflection characteristics of the object are defined in this way, the appearance of the color or shadow of the object can be arbitrarily determined by giving arbitrary values to k _a , k _d , k _s , and ∝. Here, a set of k _a , k _d , k _s , and cocoon is a material.

  However, the values of parameters constituting such a material are not intuitive values, and skill is required to provide a material that reproduces an arbitrary appearance. Furthermore, in general, objects to be expressed in general three-dimensional computer graphics are composed of a large number or complex primitives, and there is a requirement to have different materials for each group composed of each primitive or a plurality of primitives. Many. In such a case, it is necessary to define a material for each or each group, and providing such a set of materials (hereinafter, a set of multiple materials is referred to as a material set) is also a task that requires skill and labor. It is.

  In response to the requirement to determine the color scheme and reflection characteristics of an object by providing a material set with simple input that does not require skill, it is also possible to select a favorite one from a plurality of preset material sets. However, it is difficult to prepare in advance an exhaustive material set that can sufficiently meet the user's requirements for any arbitrary object, and the more options, the more time is required for the selection action.

  As a solution to this problem, when multiple points or primitives on an object and a color corresponding to the point are specified as user input, the color specification is automatically changed to an unspecified point on the object or There is a method called Mesh Colorization that is reflected in the primitive (for example, see Non-Patent Document 1). In this method, a material set can be given without special techniques by intuitively selecting a color to be given to an object and a location to be given. In this case, each material has a value indicating a single object color as a parameter.

  However, since it is necessary for the user to clearly determine the color arrangement, that is, the combination of the colors and the correspondence between each color and the primitives constituting the object, it is still necessary to perform the color arrangement according to the image to some extent. Requires experience.

G. Leifman and A. Tal, “Mesh Colorization,” Computer Graphics Forum, 2012.

  As described above, there is a problem in that the user specifies not only the color and reflection characteristics of an arbitrary object, or the combination thereof, but also controls the appearance of the object, which requires a lot of work and experience.

  The present invention has been made in view of such circumstances, and image processing that can determine parameters that define the color and texture of an arbitrary object via a color space that constitutes an image by simple user input. It is an object to provide an apparatus, an image processing method, and an image processing program.

  The present invention provides a color space analyzing unit that analyzes a color space of an image, a color group dividing unit that divides the color space into color groups based on the analysis result of the color space, and components of the input three-dimensional model Alternatively, color group assigning means for assigning the divided color groups to the component groups, and parameters for defining the color and texture of the three-dimensional model based on the characteristics of the assigned color groups And parameter determining means for determining.

  The present invention is characterized in that the color group assigning means assigns the color groups based on an area ratio or a volume ratio of the constituent elements or constituent element groups of the three-dimensional model.

  The present invention is characterized in that the color group assigning means assigns the color group to a component or a component group of the three-dimensional model so as to be similar to the distribution of the color group in the image. .

  In the present invention, the parameter determination means determines the parameter using at least one of the colors included in the color group and a statistic for the set of colors as a characteristic of the color group. It is characterized by.

  The present invention is characterized in that the parameter determination means determines the parameter using a value obtained by calculation from an image area corresponding to the color group as a characteristic of the color group.

  The present invention is characterized by further comprising image acquisition means for inputting information capable of specifying an image and acquiring the image based on the information capable of specifying the image.

  The present invention provides a color space analyzing step for analyzing a color space of an image, a color group dividing step for dividing the color space into color groups based on the analysis result of the color space, and components of the input three-dimensional model Alternatively, a color group assigning step for assigning the divided color groups to the component groups, and parameters for defining the color and texture of the three-dimensional model based on the characteristics of the assigned color groups. And a parameter determining step for determining.

  The present invention is an image processing program for causing a computer to function as the image processing apparatus.

  According to the present invention, it is possible to easily determine parameters that define the color and texture of an arbitrary object via a color space that constitutes an image.

It is a block diagram which shows the structure of 1st Embodiment of this invention. It is a flowchart which shows operation | movement of the apparatus shown in FIG. It is a block diagram which shows the structure of 2nd Embodiment of this invention. It is a flowchart which shows operation | movement of the apparatus shown in FIG.

<First Embodiment>
Hereinafter, an image processing apparatus according to a first embodiment of the present invention will be described with reference to the drawings. Terms used in the following description are defined. A color group is a group made up of one or more colors, and is a subset of the set of all colors that make up the target color space. Each color group may or may not be independent. The union of all color groups may not be equivalent to the set of all colors constituting the color space. The primitive group is a primitive composed of a target object or a group composed of a plurality of primitives, and is a subset of a set of all primitives composing the object. Each primitive group may or may not be independent. In general, the union of all primitive groups is equivalent to the set of all primitives that make up the object.

  FIG. 1 is a block diagram showing the configuration of the first embodiment. In FIG. 1, reference numeral 100 denotes a material set generation device configured by a computer device. Reference numeral 101 denotes an image input unit that inputs one or more images. Reference numeral 102 denotes an object input unit for inputting one or more objects. Reference numeral 103 denotes a color group determination unit that divides the color space of the input image into a plurality of color groups. Reference numeral 104 denotes a color arrangement unit that associates each color group with a primitive group constituting an object. Reference numeral 105 denotes a material generation unit that determines one or more parameters from the corresponding color group for each primitive group and generates a material. Reference numeral 106 denotes a material output unit that outputs the generated material set.

  Next, the operation of the material set generation apparatus 100 illustrated in FIG. 1 will be described with reference to FIG. FIG. 2 is a flowchart showing the operation of the material set generation apparatus 100 shown in FIG. First, the image input unit 101 inputs one or more images, and the object input unit 102 inputs one or more objects (step S101). An object is composed of vertices, polygons, free-form surfaces, voxels and other primitives, and is composed of primitive groups composed of the primitives. Each primitive group is a set of one or more primitives. When each primitive group does not include two or more primitives, or when a primitive group is adaptively set in the apparatus, the primitive group may not be explicitly indicated at the time of input.

  Next, the color group determination unit 103 analyzes the color space of the input image input by the image input unit 101 and divides it into a plurality of color groups (step S102). The color space analysis is to perform distribution and other statistical analysis on the pixel value of each pixel constituting the image, and is represented by histogram analysis or the like. In the present invention, not only analysis methods generally used in image processing, such as histogram analysis, but also multivariate analysis using information obtained from images or additional information obtainable by any other method, etc. This is called color space analysis.

  The color space analysis and division may be performed in any way, but can be performed using, for example, histogram analysis. By generating a histogram with a certain range for any of the components constituting the color space and performing peak detection, it is possible to identify one or more ranges of significant values constituting the image for that component. The values included in this range may be combined to form one color group, or a representative value may be selected.

  Further, a range wider than the histogram range may be set as a range of values included in the color group, or a narrow range may be used. If a plurality of peaks are detected, color groups may be generated for all of the peaks, or if the number of primitive groups is known, it may be generated according to the number of pixels, and pixels constituting the peak A threshold value may be provided for the total amount, and the color group may be generated only from the peak composed of pixels of a certain amount or more.

  In addition, when the number of color groups is less than the number of primitive groups, the color groups may be further divided, or a peak that does not satisfy the criterion may be newly set as a candidate, and one color group may be a plurality of primitive groups. You may assign to. Further, colors included in different color groups may overlap.

  Any color space may be defined, any component may be used as a component to be analyzed, a plurality of different color spaces or components may be analyzed separately, and a comprehensive analysis may be performed. It doesn't matter. For example, by analyzing hue components in the HVS color space, grouping by similar hues is performed, and a gorgeous material set composed of a plurality of hues can be generated.

  Alternatively, by further analyzing the saturation component, it is possible to generate a material set with a sense of unity by dividing into high hue and low saturation while grouping together with similar hues. These color spaces and components may be determined, may be changed by user input, or may be determined automatically from the tendency of the image. For example, when the input image is biased to a specific hue range as a whole, the analysis target may be adaptively changed by performing saturation analysis.

  There may be one or more input images. When a plurality of images are input, a color group may be created separately by performing color space analysis for each image. In this case, all the color groups created may be assigned to primitive groups, or only some color groups may be selected and used. For example, correlation may be obtained between color groups generated from different input images, and color groups with low correlation between images may be excluded.

  Alternatively, the color space of all the images may be integrated for analysis. In that case, the color space may be integrated in any way. The statistical population may simply be the pixels of all images. Further, all images and their pixel values or their analysis values may be regarded as worthy, for example, weighting may be performed according to the ratio of image sizes. For example, when the color spaces of the image A having the size L and the image B having the size S are integrated, an S / L weight may be applied to the image B. Any other integration may be performed.

  Next, the color arrangement unit 104 associates each color group with each primitive group (step S103). The color group and the primitive group may be associated in any way. For example, the color group is assigned in the descending order of the number of pixels constituting each color group, and the area ratio or volume ratio of the primitives constituting the primitive group is assigned in descending order. May be.

  Alternatively, a method of assigning the pixel groups constituting each color group so that the distribution in the original image in the original image and the distribution of the primitive groups on the surface of the object are similar may be considered. In this method, for example, when the pixels of the color groups A and B are distributed close to each other and the color group C is distributed far away, for example, the distance between the centers of the respective distributions of A, B, and C is set. The input image is obtained by assigning the ratio of the mutual distances between the centroids of the primitives constituting the corresponding primitive groups a, b, and c to be equal to the mutual distance ratio of the distribution centers in the color group. A color scheme that gives a close impression can be given.

  Further, when the number of primitive groups is unknown or when grouping is not performed in advance, the objects may be adaptively divided into primitive groups according to the number of color groups. As a group division method, primitive grouping may be performed by a method such as the above-described color arrangement method, or another method may be used.

  Next, the material generation unit 105 determines one or more parameters constituting the material from the corresponding color group for each primitive group (step S104). Each material may be composed of any component. In the simplest example, it consists of values representing a single color. Further, as used in general computer graphics expression, it may have a specular reflection coefficient, a diffuse reflection coefficient, a reflection intensity value, or any other component.

  Any method may be used to obtain the parameters from the color group. For example, a statistic such as an average value, an intermediate value, or a maximum value of colors constituting the color group may be used. For example, the average value in the saturation component of the color group may be the diffuse reflection coefficient, and the maximum value may be the specular reflection coefficient. Further, a value indicating the reflection intensity may be obtained from a brightness histogram in the color group.

  Alternatively, the parameters may be determined by performing some image processing on a partial region including pixels belonging to the color group to be calculated among the pixels of the original image. For example, a visually conspicuous region may be detected in an image using a Gabor filter or the like, and the pixel value of the region may be used to determine a coefficient having a conspicuous effect such as a specular reflection coefficient.

  Further, it may be determined in consideration of the correlation with other color groups, or may be determined independently. For example, if a remarkably bright area is detected when analyzed over the entire image, the reflection intensity of the color group including that area is set high and set low for the other color groups, so that the entire original image For example, a method for obtaining a parameter for expressing a rendering result having an atmosphere close to the impression it has may be used.

  Alternatively, the characteristics of the primitive group may be reflected in the material generation. For example, based on the dispersion of the normals of the primitives that make up a group, when the dispersion is small, the reflection intensity is set high so that the group has a smooth texture, and when the dispersion is large, the texture is reflected as a rough texture. The strength may be set low. Any other characteristics may be reflected.

  Next, the material output unit 106 outputs information indicating the correspondence to the generated material set and primitive group as material information (step S105). The material information may be described in any way, information indicating the details of the material and information indicating the correspondence may be described at the same time, or may be described separately.

  As a description method widely used in the computer graphic field, there is a method in which information indicating details of a material is described independently as material information, and information indicating correspondence is described together with information indicating an object. When such a description method is adopted, an object output unit for outputting input object information together with correspondence information may be provided separately. In addition, there is a method in which information indicating the details of the material is not described independently, and there is a method of describing the information together with the information indicating the object together with the information indicating the correspondence. In this case, an object output unit is provided instead of the material output unit 106 You may do it.

  In the above description, the color space that composes the image is first divided into color groups and assigned to each primitive group, and then the material is generated. You can assign it.

  Alternatively, color group division may be performed so as to optimize a likely color group combination corresponding to each primitive group. For example, as described in the description section of the color arrangement unit 102, color group division can be performed so that the distribution of colors included in the color group in the image is close to the distribution of primitives included in the primitive group in the object.

  Alternatively, any additional information may be used for color space analysis or color arrangement. For example, by specifying one or more colors to be mainly used, a color group having such colors as representative values such as an average value and an intermediate value can be configured from the color space. Furthermore, when one color group including mainly used colors is configured, the other color groups can be configured as color groups including complementary colors of the main color. In order to input such additional information as user input, an additional information input unit may be further provided, or it may be input simultaneously with the image as metadata accompanying the image.

Second Embodiment
Next, an image processing apparatus according to a second embodiment of the present invention will be described with reference to the drawings. FIG. 3 is a block diagram showing the configuration of the second embodiment. In FIG. 3, the code | symbol 100a is the material set production | generation apparatus comprised with a computer apparatus. In FIG. 3, the same parts as those in the apparatus shown in FIG. The apparatus shown in FIG. 3 is different from the apparatus shown in FIG. 1 in that input information for acquiring, as input information, a keyword that can acquire any image representing a user's image, an image feature amount, or another similar image An input unit 101a is newly provided, and instead of the image input unit 101, an image acquisition unit 101b that acquires one or more images using input information is provided. The color group determination process is performed using the above image as an input image.

  Next, the operation of the material set generation apparatus a shown in FIG. 3 will be described with reference to FIG. FIG. 4 is a flowchart showing the operation of the material set generation apparatus a shown in FIG. In FIG. 4, the same parts as those in the processing operation shown in FIG.

  First, the input information input unit 101a inputs a keyword / image feature amount or another similar image as input information, and the object input unit 102 inputs one or more objects (step S101a). Subsequently, the image acquisition unit 101b acquires one or more images using the input information (step S101b). Hereinafter, the keyword / image feature amount or another similar image is referred to as input information.

  The image may be acquired in any way, but for example, a query may be created from the input information, and a method such as image search using the created query may be used. In the simplest case, it is possible to perform an image search that can use the input information as a query as it is and acquire an image from the search result. An input query may be created from a plurality of pieces of information as a plurality of input information or a combination of different information, or an image search may be performed, or a plurality of input queries may be created and a plurality of image searches may be performed. I do not care.

  Alternatively, a database that defines the relationship between these input information and images may be prepared and used in advance. In this case, a database unit may be further provided. Further, a plurality of samples may be prepared in advance and any one of them may be output, or may be generated, corrected, or synthesized adaptively according to input information and output. For example, as a simple example, by preparing a large number of sample images and a large number of sample keywords in advance and defining the correspondence between them, by preparing a vector space spanned by a base capable of constructing an image corresponding to the keyword as a database, A base coefficient in the vector space can be acquired for an arbitrary keyword, and a new image corresponding to the keyword can be reconstructed using the coefficient and the base.

  Further, as another example, when a keyword as input information includes a hue keyword indicating hue and other keywords, a sample image is specified using a keyword other than the hue keyword, and the hue indicated by the hue keyword is the main image. It is also possible to shift the hue of the entire image so that it is a hue and output an image that has been hue shifted. Any other modification or synthesis may be performed to generate the output image.

  One or more images may be acquired. When multiple candidates are obtained and ranked by image search etc., multiple images may be output in the order of highest ranking or highest ranking, and narrowed down to a different scale from the multiple candidates. And the narrowing down result may be output. Alternatively, a combination or combination of a plurality of images may be output.

  Next, the color group determination unit 103 analyzes the color space of the input image input by the image input unit 101 and divides it into a plurality of color groups (step S102). Then, the color arrangement unit 104 associates each color group with each primitive group (step S103).

  Next, the material generation unit 105 determines one or more parameters constituting the material from the corresponding color group for each primitive group (step S104). Then, the material output unit 106 outputs information indicating the correspondence to the generated material set and primitive group as material information (step S105).

  As described above, the material set generation apparatus determines each parameter constituting the material set to be given to an arbitrary object by analyzing the color space constituting the input image. In order to realize this, the material set generation device is provided with a color group determining unit that divides the color space of the input image into a plurality of color groups, a color arrangement unit that associates each color group with each primitive group, and a correspondence with each primitive group. And a material generating means for determining one or more parameters from the color group to generate the material. Further, as another form of the material set generation device, a keyword or image feature amount used for image search or another similar image is used as an input, and one or more images are acquired using an input query, and the color group determination is performed. The input image used in the means was used.

  As described above, as a method of controlling the color and reflection characteristic of an arbitrary object or a combination thereof by simple user input, an arbitrary image close to the user's desired atmosphere / image or a key that can specify such an image With a query or the like as an input, a material set as a color and reflection characteristic of an arbitrary object or a combination thereof can be generated through a color space constituting an image. In addition, a material set corresponding to an arbitrary object can be generated from an arbitrary image.

  Also, the reflection characteristics are not estimated clearly for the objects present in the input image, so the colors that make up the input image when there is no relationship between the input image and the object to be represented. A material consisting of parameters that determine one or more plausible color groups from space, assign the color group to a primitive group that constitutes an arbitrary object, and specify reflection characteristics for the assigned combination. This makes it possible to specify intuitive colors and reflection characteristics that reflect the atmosphere and image of the input image on an arbitrary object. Also, instead of inputting an image, a keyword that can acquire any image representing the user's image, an image feature amount, or another similar image is used as input information, a query is generated from the input information, and the query is used to generate a query. By acquiring two or more images and giving them color and reflection characteristics as input images, it becomes possible to specify them more intuitively. As a result, it is possible to provide a color and reflection characteristic designation method suitable for rendering in the field of three-dimensional computer graphics.

  You may make it implement | achieve the material set production | generation apparatus in embodiment mentioned above with a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system serving as a server or a client in that case may be included and a program held for a certain period of time. Further, the program may be for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in the computer system. It may be realized using hardware such as PLD (Programmable Logic Device) or FPGA (Field Programmable Gate Array).

  As mentioned above, although embodiment of this invention has been described with reference to drawings, the said embodiment is only the illustration of this invention, and it is clear that this invention is not limited to the said embodiment. is there. Therefore, additions, omissions, substitutions, and other modifications of the components may be made without departing from the technical idea and scope of the present invention.

  It can be applied to applications where it is essential to specify color and reflection characteristics suitable for rendering in the field of three-dimensional computer graphics.

  DESCRIPTION OF SYMBOLS 100, 100a ... Material set production | generation apparatus, 101 ... Image input part, 101a ... Input information input part, 101b ... Image acquisition part, 102 ... Object input part, 103 ... Color group Determination unit, 104 ... Color arrangement unit, 105 ... Material generation unit, 106 ... Material output unit

Claims (8)

Color space analysis means for analyzing the color space of the image;
Color group dividing means for dividing the color space into color groups based on the analysis result of the color space;
Color group assigning means for assigning the divided color groups to the components or component groups of the input three-dimensional model;
An image processing apparatus comprising: parameter determination means for determining parameters for defining the color and texture of the three-dimensional model based on the characteristics of the assigned color groups. The color group assigning means includes
The image processing apparatus according to claim 1, wherein the color group is assigned based on an area ratio or a volume ratio of a component or a component group of the three-dimensional model. The color group assigning means includes
The image processing apparatus according to claim 1, wherein the color group is assigned to a component or a component group of the three-dimensional model so as to be similar to a distribution of the color group in the image. The parameter determination means includes
2. The parameter according to claim 1, wherein the parameter is determined using at least one of a color included in the color group and a statistic for the set of colors as a characteristic of the color group. Image processing device. The parameter determination means includes
The image processing apparatus according to claim 1, wherein the parameter is determined using a value obtained by calculation from an image region corresponding to the color group as a feature of the color group.   6. The image acquisition device according to claim 1, further comprising an image acquisition unit that inputs information capable of specifying an image and acquires the image based on information capable of specifying the image. Image processing device. A color space analysis step for analyzing the color space of the image;
A color group dividing step of dividing the color space into color groups based on the analysis result of the color space;
A color group assigning step for assigning the divided color groups to the components or component groups of the input three-dimensional model;
And a parameter determining step for determining parameters for defining the color and texture of the three-dimensional model based on the assigned characteristics of each of the color groups.   An image processing program for causing a computer to function as the image processing apparatus according to claim 1.

Images