CN104537658A - Modeling system and extracting method for primal sketch of color image - Google Patents

Abstract

The invention discloses a modeling system and extracting method for a primal sketch of a color image. The modeling system comprises a color space alternation module, a structural motif modeling module, a texture primitive modeling module and a color image primal sketch modeling module, wherein the color space alternation module transforms an original RGB image into a Lab color space through a color space, the structural motif modeling module divides a structure part in the image into a set of K disjointed image blocks, each set of image blocks are represented by an image primitive, and the structure part of the image is represented by a structure chart through a set of image primitives; the texture primitive modeling module clusters a texture part in the color image trough response of a set of selected filters and divides the texture part into a set of disjointed homogeneous texture areas, each homogeneous texture area is represented by a set of histograms in an implicit expression mode, and finally the texture areas in the image are marked; the color image primal sketch modeling module obtains a probability mathematic model represented by a primal sketch of a gray level image, and constructs a model for the primal sketch of the color image. Through the modeling system and extracting method for the primal sketch of the color image, the primal sketch neglected by the gray level primal sketch image can be extracted.

Description

The main primitive graph model of coloured image sets up system and extracting method

Technical field

The present invention relates to human-computer interaction technique field, the main primitive graph model particularly relating to a kind of coloured image sets up system and extracting method.

Background technology

Machine vision (Machine Vision, MV), replace human eye measure and judge with machine exactly, it is the requisite acquisition of information passage of any intelligent system.According to three layers of vision computation model of Marr, be the ground floor that vision calculates from original 2D image to main primitive figure (Primal Sketch).Main primitive figure is a kind of very important image table representation model in machine vision, its object is to the expression of structure and texture in unified image.

Marr proposes main primitive representation theory in its vision computation model, but does not provide a complete mathematical model and extraction algorithm.Main primitive extracting method main is at present divided into two classes: (one) is based on the main primitive extracting method of zero crossing; (2) the main primitive graph model of the gray level image of Corpus--based Method method and extraction algorithm.

But above-mentioned main primitive extracting method has following shortcoming: the main primitive extracting method 1, based on zero crossing needs to select suitable wave filter.But for dissimilar image, there is no unified rules selection wave filter; 2, the main primitive graph model of the gray level image of Corpus--based Method method and extraction algorithm, gray level image can only be processed, from visually-perceptible theory, color is a very important information source in image perception, particularly there is very important impact to the perception of the profile of object, thus also have impact on the expression of main primitive figure.

Summary of the invention

For overcoming the deficiency that above-mentioned prior art exists, the object of the present invention is that the main primitive graph model providing a kind of coloured image sets up system and extracting method, its main primitive figure of coloured image extracted meets perception more, can extract by the main primitive of gray scale main primitive figure " undetected ", the main primitive figure that the present invention proposes, may be used for image content-based compression of images side and in the middle of the application such as high-rise object identification and image understanding.

For reaching above-mentioned and other object, the main primitive graph model that the present invention proposes a kind of coloured image sets up system, comprising:

Color notation conversion space module, for by original RGB image by color notation conversion space to Lab color space;

Structural motif MBM, the structure division in image is divided into one group of K disjoint image block, each image block is represented by a kind of picture element, and the structure division of image forms a structural drawing by one group of picture element and represents, realizes structural motif modeling;

Texture primitive MBM, first by the filter response selected a group, cluster is carried out to the texture part in image, be divided into one group of disjoint homogeneous texture region, each homogeneous texture region is by one group of histogram implied expression, finally the texture region in image is marked, realize the modeling of texture primitive;

The main primitive MBM of coloured image, obtains according to this structural motif MBM and this texture primitive MBM the probabilistic mathematical models that the main primitive figure of gray level image represents, and utilizes the probability model obtained to the main primitive modeling of coloured image.

Further, each image block be expressed as follows by a kind of picture element:

${\mathrm{\Λ}}_{\mathrm{sk}}={\∪}_{k=1}^K{\mathrm{\Λ}}_{\mathrm{sk},k},$

Wherein, k is the index of picture element, k=(θ _top, θ _geo, θ _pho), θ _toprepresent the type of picture element, θ _georepresent the geometric position information of picture element, θ _phorepresent gray-scale intensity or the tone intensity information of picture element.

Further, the structure division in image forms a structural drawing by one group of picture element and is expressed as follows:

S _sk＝(K，(Λ _sk，k，B _k，a _k)，k＝1，…，K)

Wherein, B _krepresent the image block that picture element k is corresponding, a _kbe its address variable, be used for representing B _kat structural drawing S _skneutralize the connection of other image blocks, Λ _skfor the structure division in image.

Further, to the texture part Λ in image _nsk, by carrying out cluster to one group of selected filter response, be divided into one group of size to be disjoint homogeneous texture region of M=3 ~ 7, wherein, and

Further, each homogeneous texture region is by one group of histogram h _mi(m=1 ..., M, i=1 ..., n) implied expression is as follows:

$h_i\left(I_{{\mathrm{\Λ}}_{\mathrm{nsk},m}}\right)=h_{\mathrm{mi}},m=1,...,M$

Further, this texture primitive MBM modeling is as follows:

$S_{\mathrm{nsk}}=(M,({\mathrm{\Λ}}_{\mathrm{nsk},m},h_{\mathrm{mi}}\↔{\mathrm{\β}}_{\mathrm{mi}}),m=1,...,M,i=1,...,n)$

Wherein, β _mibe the parameter of texture region model, solved by minimax entropy method.

Further, the modeling of this coloured image main primitive MBM is as follows:

$\begin{array}{c}p(I_{\mathrm{\&Lambda;}},S_{\mathrm{sk}},S_{\mathrm{nsk}})=\\ \frac{1}{z}\exp \{-\frac{1}{2{\mathrm{\&sigma;}}^2}{\mathrm{\&Sigma;}}_{k=1}^K{\mathrm{\&Sigma;}}_{(u,v)\&Element;{\mathrm{\&Lambda;}}_{\mathrm{sk},k}}{(I(u,v)-B_k(u,v))}^2-{\mathrm{\&Sigma;}}_{m=1}^M{\mathrm{\&Sigma;}}_{i=1}^n<{\mathrm{\&beta;}}_{\mathrm{mi}},h_i\left(I_{{\mathrm{\&Lambda;}}_{\mathrm{nsk},m}}\right)>-E\left(S_{\mathrm{sk}}\right)-E\left(S_{\mathrm{nsk}}\right)\}\end{array}$

Wherein, E (S _sk) and E (S _nsk) represent the priori energy of structural drawing and texture part.

For achieving the above object, the invention provides a kind of main primitive figure extracting method of coloured image, comprising the steps:

Step one, selected one group of wave filter, to original input picture, produces one group of proposal figure, as the basis of selecting structure picture in picture as primitive, makes simplify processes to texture part simultaneously, obtains the simplified model of the main primitive graph model of coloured image;

Step 2, according to maximizing quantity of information or minimizing the principle describing length, selects picture element and histogram to describe one by one, extracts the main primitive figure of coloured image, until convergence from proposal figure.

Further, this simplified model is

$\begin{array}{c}p(I_{\mathrm{\&Lambda;}},S_{\mathrm{sk}},S_{\mathrm{nsk}};{\mathrm{\&Delta;}}_{\mathrm{sk}})\&ap;\\ \frac{1}{z}\exp \{-\frac{1}{2{\mathrm{\&sigma;}}^2}({\mathrm{\&Sigma;}}_{k=1}^K{\mathrm{\&Sigma;}}_{(u,v)\&Element;{\mathrm{\&Lambda;}}_{\mathrm{sk},k}}{(I(u,v)-B_k(u,v))}^2+{\mathrm{\&Sigma;}}_{(u,v)\&Element;{\mathrm{\&Lambda;}}_{\mathrm{nsk}}}{(I(u,v)-\mathrm{\&mu;}(u,v))}^2\left)\right\}\end{array}.$

Further, the primitive utilizing matching pursuit algorithm at every turn to select image coding information increment maximum in primitive to be selected, adds structural drawing S to _skin, by upgrading this simplified model and comparing, obtain the current information delta brought getting primitive to be selected, and by being calculated as follows the main primitive figure of formulas Extraction coloured image:

$\begin{array}{c}\mathrm{\&Delta;L}=\log \frac{p(I_{\mathrm{\&Lambda;}},S_{\mathrm{sk}},S_{\mathrm{nsk}};{\mathrm{\&Delta;}}_{\mathrm{sk}})}{p(I_{\mathrm{\&Lambda;}},{S^{\&prime;}}_{\mathrm{sk}},{S^{\&prime;}}_{\mathrm{nsk}};{\mathrm{\&Delta;}}_{\mathrm{sk}})}=\\ \frac{1}{2{\mathrm{\&sigma;}}^2}{\mathrm{\&Sigma;}}_{(u,v)\&Element;{\mathrm{\&Lambda;}}_{\mathrm{sk},k+1}}[{(I(u,v)-B_k(u,v))}^2-{(I(u,v)-\mathrm{\&mu;}(u,v))}^2]\end{array}.$

Compared with prior art, the main primitive graph model of a kind of coloured image of the present invention sets up system and extracting method for coloured image, the expression of structure and texture in unified image, the main primitive figure achieving coloured image extracts, its main primitive figure of coloured image extracted meets perception more, can extract by the main primitive of gray scale main primitive figure " undetected ", the present invention may be used for the compression of images side of image content-based, and in the middle of the application such as high-rise object identification and image understanding.

Accompanying drawing explanation

Fig. 1 is the system architecture diagram that the main primitive graph model of a kind of coloured image of the present invention sets up system;

Fig. 2 is the flow chart of steps of the main primitive figure extracting method of a kind of coloured image of the present invention;

Fig. 3 is the extraction FB(flow block) of the main primitive figure of coloured image in present pre-ferred embodiments.

Embodiment

Below by way of specific instantiation and accompanying drawings embodiments of the present invention, those skilled in the art can understand other advantage of the present invention and effect easily by content disclosed in the present specification.The present invention is also implemented by other different instantiation or is applied, and the every details in this instructions also can based on different viewpoints and application, carries out various modification and change not deviating under spirit of the present invention.

Before explaining the present invention, first illustratively theoretical foundation of the present invention: image represents can be divided into two parts, i.e. the structure division of image and the texture part of image.Intuitively, from the process of imaging, because object is different from the distance of camera, object on hand, defines the structure division in image, problem a long way off, and its structure is objectively no longer distinguishable in the picture, just defines the sensation of texture.The technical problem to be solved in the present invention is exactly for coloured image, the expression of structure and texture in research unified image, and main primitive figure represents and wants to unify these two kinds different mathematical models.

Fig. 1 is the system architecture diagram that the main primitive graph model of a kind of coloured image of the present invention sets up system.As shown in Figure 1, the main primitive graph model of a kind of coloured image of the present invention sets up system, comprising: the main primitive MBM 104 of color notation conversion space module 101, structural motif MBM 102, texture primitive MBM 103 and coloured image.

Wherein, color notation conversion space module 101 for by original RGB image by color notation conversion space to Lab color space.Lab color space is made up of three key elements, and key element is brightness (L), a and b is two Color Channels.The color that a comprises is again to bright pink (high luminance values) from bottle green (low brightness values) to grey (middle brightness value); B is again to yellow (high luminance values) from sapphirine (end brightness value) to grey (middle brightness value).Distance metric due to Lab color space meets the perception of the mankind to color more, by original RGB image by color notation conversion space to Lab color space, maximum operation is gone to Lab tri-passages, thus extracts the primitive that originally can not embody in gray level image.

Structural motif MBM 102, the structure division of image is divided into one group of K disjoint image block, each image block is represented by a kind of picture element, and the structure division of image forms a structural drawing by one group of picture element and represents, realizes structural motif modeling.

In present pre-ferred embodiments, image lattice is designated as Λ, as 256 × 256 pixels; The image be defined on Λ is designated as I _Λ.In main primitive figure represents, Λ is divided into structure division and texture part, is designated as Λ respectively _skand Λ _nsk, and meet:

Λ＝Λ _sk∪Λ _nsk，

Structure division Λ _skone group of K disjoint image block, wherein each image block can be divided into (as edge section) is expressed by a kind of picture element,

${\mathrm{\&Lambda;}}_{\mathrm{sk}}={\&cup;}_{k=1}^K{\mathrm{\&Lambda;}}_{\mathrm{sk},k},$

Wherein, k, as the index of picture element, is an implicit variable (hidden variable), needs from given input picture, to carry out reasoning by extraction algorithm.

k＝(θ _top，θ _geo，θ _pho) (3)

Wherein, θ _toprepresent the type (e.g., blob, angle point etc.) of picture element, θ _georepresent the geometric position information of picture element, θ _phorepresent gray-scale intensity or the tone intensity information of picture element.Structure division in image forms a structural drawing (sketch graph) by this group picture element and represents.

S _sk＝(K，(Λ _sk，k，B _k，a _k)，k＝1，…，K) (4)

Wherein, B _krepresent the image block that picture element k is corresponding, a _kbe its address variable, be used for representing B _kat structural drawing S _skneutralize the connection of other image blocks.By being similar to the expression of the production model of sparse coding, have:

$I_{{\mathrm{\&Lambda;}}_{\mathrm{sk},k}}=B_k+n,k=1,...,K---\left(5\right)$

N represents random Gaussian.

Texture primitive MBM 103, first by the filter response selected a group, cluster is carried out to the texture part in image, be divided into one group of disjoint homogeneous texture region, each homogeneous texture region is by one group of histogram implied expression, finally the texture region in image is marked, realize the modeling of texture primitive.

To the texture part Λ in image _nsk, the filter response usually first by selecting a group carries out cluster, is divided into disjoint homogeneous texture region, one group of M=3 ~ 7 (homogeneous texture region), namely ${\mathrm{\&Lambda;}}_{\mathrm{nsk}}={\&cup;}_{m=1}^M{\mathrm{\&Lambda;}}_{\mathrm{nsk},m}$ And each homogeneous texture region is by one group of histogram h _mi(m=1 ..., M, i=1 ..., n) implied expression, has,

$h_i\left(I_{{\mathrm{\&Lambda;}}_{\mathrm{nsk},m}}\right)=h_{\mathrm{mi}},m=1,...,M---\left(6\right)$

The carrying out of the texture region in image is marked, then has:

$S_{\mathrm{nsk}}=(M,({\mathrm{\&Lambda;}}_{\mathrm{nsk},m},h_{\mathrm{mi}}\&LeftRightArrow;{\mathrm{\&beta;}}_{\mathrm{mi}}),m=1,...,M,i=1,...,n)---\left(7\right)$

Wherein, β _mibe the parameter of texture region model, solved by minimax entropy method.

The main primitive MBM 104 of coloured image, obtains according to structural motif MBM 102 and texture primitive MBM 103 probabilistic mathematical models that the main primitive figure of gray level image represents, wherein, and E (S _sk) and E (S _nsk) represent the priori energy of structural drawing and texture part, and utilize the probability model obtained to the main primitive modeling of coloured image.

By structural motif MBM 102 and texture primitive MBM 103, the probabilistic mathematical models that the main primitive figure of gray level image represents can be obtained, wherein, E (S _sk) and E (S _nsk) represent the priori energy of structural drawing and texture part.This probability model has effectively unified the expression of structure and texture two parts.

$\begin{array}{c}p(I_{\mathrm{\&Lambda;}},S_{\mathrm{sk}},S_{\mathrm{nsk}})=\\ \frac{1}{z}\exp \{-\frac{1}{2{\mathrm{\&sigma;}}^2}{\mathrm{\&Sigma;}}_{k=1}^K{\mathrm{\&Sigma;}}_{(u,v)\&Element;{\mathrm{\&Lambda;}}_{\mathrm{sk},k}}{(I(u,v)-B_k(u,v))}^2-{\mathrm{\&Sigma;}}_{m=1}^M{\mathrm{\&Sigma;}}_{i=1}^n<{\mathrm{\&beta;}}_{\mathrm{mi}},h_i\left(I_{{\mathrm{\&Lambda;}}_{\mathrm{nsk},m}}\right)>-E\left(S_{\mathrm{sk}}\right)-E\left(S_{\mathrm{nsk}}\right)\}\end{array}$

Wherein σ is the variance of gauss hybrid models, and Z is normalization probability model.

Fig. 2 is the flow chart of steps of the main primitive figure extracting method of a kind of coloured image of the present invention.Fig. 3 is the extraction FB(flow block) of the main primitive figure of coloured image in present pre-ferred embodiments.As shown in Figures 2 and 3, the main primitive figure extracting method of a kind of coloured image of the present invention, comprises the steps:

Step 201, selected one group of wave filter, to original input picture, produce one group " proposing figure " (proposalmaps), as the basis of selecting structure picture in picture as primitive, simplify processes is done to texture part simultaneously, obtain the simplified model of the main primitive graph model of coloured image.

Specifically, selected one group of wave filter, comprise the Gabor filter of different scale and different directions, DoG (difference of Gaussian), LoG (Laplace of Gaussian) etc., to original input picture, produce one group " proposing figure " (proposal maps), as the basis of selecting structure picture in picture as primitive, a simplify processes is first done to texture part simultaneously, use Gauss model to replace.Simplified model is:

$\begin{array}{c}p(I_{\mathrm{\&Lambda;}},S_{\mathrm{sk}},S_{\mathrm{nsk}};{\mathrm{\&Delta;}}_{\mathrm{sk}})\&ap;\\ \frac{1}{z}\exp \{-\frac{1}{2{\mathrm{\&sigma;}}^2}({\mathrm{\&Sigma;}}_{k=1}^K{\mathrm{\&Sigma;}}_{(u,v)\&Element;{\mathrm{\&Lambda;}}_{\mathrm{sk},k}}{(I(u,v)-B_k(u,v))}^2+{\mathrm{\&Sigma;}}_{(u,v)\&Element;{\mathrm{\&Lambda;}}_{\mathrm{nsk}}}{(I(u,v)-\mathrm{\&mu;}(u,v))}^2\left)\right\}\end{array}---\left(9\right)$

Δ _skfor one group of K disjoint image block of structure division.

Step 202, according to maximizing quantity of information or minimizing the principle describing length, from " proposing figure ", select picture element (to structure division) and histogram to describe (to texture part) one by one, extract the main primitive figure of coloured image, until convergence.

The primitive that matching pursuit algorithm selects image coding information increment maximum at every turn in primitive to be selected, adds structural drawing S to _skin, i.e. S _sk=S _sk∪ S _{sk, k+1}and Λ _nsk=Λ _nsk-Λ _{sk, k+1}, by Renewal model (9), and compare, obtain the current information delta that may bring getting primitive to be selected,

$\begin{array}{c}\mathrm{\&Delta;L}=\log \frac{p(I_{\mathrm{\&Lambda;}},S_{\mathrm{sk}},S_{\mathrm{nsk}};{\mathrm{\&Delta;}}_{\mathrm{sk}})}{p(I_{\mathrm{\&Lambda;}},{S^{\&prime;}}_{\mathrm{sk}},{S^{\&prime;}}_{\mathrm{nsk}};{\mathrm{\&Delta;}}_{\mathrm{sk}})}=\\ \frac{1}{2{\mathrm{\&sigma;}}^2}{\mathrm{\&Sigma;}}_{(u,v)\&Element;{\mathrm{\&Lambda;}}_{\mathrm{sk},k+1}}[{(I(u,v)-B_k(u,v))}^2-{(I(u,v)-\mathrm{\&mu;}(u,v))}^2]\end{array}---\left(10\right)$

Computing formula (10), thus the main primitive figure that can extract coloured image.

In sum, the main primitive graph model of a kind of coloured image of the present invention sets up system and extracting method for coloured image, the expression of structure and texture in unified image, the main primitive figure achieving coloured image extracts, its main primitive figure of coloured image extracted meets perception more, can extract by the main primitive of gray scale main primitive figure " undetected ", the present invention may be used for the compression of images side of image content-based, and in the middle of the application such as high-rise object identification and image understanding.

Above-described embodiment is illustrative principle of the present invention and effect thereof only, but not for limiting the present invention.Any those skilled in the art all without prejudice under spirit of the present invention and category, can carry out modifying to above-described embodiment and change.Therefore, the scope of the present invention, should listed by claims.

Claims (10)

1. the main primitive graph model of coloured image sets up a system, comprising:

Color notation conversion space module, for by original RGB image by color notation conversion space to Lab color space;

Structural motif MBM, the structure division in image is divided into one group of K disjoint image block, each image block is represented by a kind of picture element, and the structure division of image forms a structural drawing by one group of picture element and represents, realizes structural motif modeling;

Texture primitive MBM, first by the filter response selected a group, cluster is carried out to the texture part in image, be divided into one group of disjoint homogeneous texture region, each homogeneous texture region is by one group of histogram implied expression, finally the texture region in image is marked, realize the modeling of texture primitive;

The main primitive MBM of coloured image, obtains according to this structural motif MBM and this texture primitive MBM the probabilistic mathematical models that the main primitive figure of gray level image represents, and utilizes the probability model obtained to the main primitive modeling of coloured image.

2. the main primitive graph model of a kind of coloured image as claimed in claim 1 sets up system, it is characterized in that, each image block be expressed as follows by a kind of picture element:

Wherein, k is the index of picture element, k=(θ _top, θ _geo, θ _pho), θ _toprepresent the type of picture element, θ _georepresent the geometric position information of picture element, θ _phorepresent gray-scale intensity or the tone intensity information of picture element.

3. the main primitive graph model of a kind of coloured image as claimed in claim 2 sets up system, it is characterized in that, the structure division in image forms a structural drawing by one group of picture element and is expressed as follows:

Wherein, B _krepresent the image block that picture element k is corresponding, a _kbe its address variable, be used for representing B _kat structural drawing S _skneutralize the connection of other image blocks, for the structure division in image.

4. the main primitive graph model of a kind of coloured image as claimed in claim 3 sets up system, it is characterized in that: to the texture part in image , by carrying out cluster to one group of selected filter response, be divided into one group of size to be disjoint homogeneous texture region of M=3 ~ 7, wherein, and

5. the main primitive graph model of a kind of coloured image as claimed in claim 4 sets up system, it is characterized in that, each homogeneous texture region is by one group of histogram h _mi(m=1 ..., M, i=1 ..., n) implied expression is as follows:

。

6. the main primitive graph model of a kind of coloured image as claimed in claim 5 sets up system, it is characterized in that: this texture primitive MBM modeling is as follows:

Wherein, β _mibe the parameter of texture region model, solved by minimax entropy method.

7. the main primitive graph model of a kind of coloured image as claimed in claim 6 sets up system, it is characterized in that, the modeling of this coloured image main primitive MBM is as follows:

Wherein, E (S _sk) and E (S _nsk) representing the priori energy of structural drawing and texture part, σ is the variance of gauss hybrid models, and Z is normalization probability model.

8. a main primitive figure extracting method for coloured image, comprises the steps:

Step one, selected one group of wave filter, to original input picture, produces one group of proposal figure, as the basis of selecting structure picture in picture as primitive, makes simplify processes to texture part simultaneously, obtains the simplified model of the main primitive graph model of coloured image;

Step 2, according to maximizing quantity of information or minimizing the principle describing length, selects picture element and histogram to describe one by one, extracts the main primitive figure of coloured image, until convergence from proposal figure.

9. the main primitive figure extracting method of a kind of coloured image as claimed in claim 8, it is characterized in that, this simplified model is

Wherein σ is the variance of gauss hybrid models, and Z is normalization probability model, Δ _skfor one group of K disjoint image block of structure division.

10. the main primitive figure extracting method of a kind of coloured image as claimed in claim 9, is characterized in that: in step 2, and the primitive utilizing matching pursuit algorithm at every turn to select image coding information increment maximum in primitive to be selected, adds structural drawing S to _skin, by upgrading this simplified model and comparing, obtain the current information delta brought getting primitive to be selected, and by being calculated as follows the main primitive figure of formulas Extraction coloured image: