WO2008093029A2

WO2008093029A2 - Methods for the analysis and modelling of an image containing unit elements to be detected, and corresponding system and device

Info

Publication number: WO2008093029A2
Application number: PCT/FR2007/052570
Authority: WO
Inventors: Julien Ricard; Jérôme Royan; Patrick Gioia
Original assignee: France Telecom
Priority date: 2006-12-21
Filing date: 2007-12-19
Publication date: 2008-08-07
Also published as: FR2910674A1; WO2008093029A3

Abstract

The invention relates to a method for analysing an image containing unit elements aligned along horizontal and vertical directions, that comprises the steps of: detecting (ai) outlines and obtaining an outline image; calculating (a3) a histogram along a vertical or horizontal direction, representing said outline image, and respectively called vertical outline histogram or horizontal outline histogram; wherein said method is characterised in that it comprises the following additional steps: calculating (a4) a background belonging image from said image containing the unit elements to be detected; calculating (a5) a histogram along a vertical or horizontal direction, representing said background belonging image, and respectively called vertical background histogram or horizontal background histogram; merging (a6) vertical or horizontal background histogram with said vertical outline histogram or said horizontal outline histogram.

Description

Methods of analyzing and modeling an image containing unit elements to be detected, corresponding system and device

The present invention generally relates to the fields of image analysis and rendering, and more specifically to the visualization of urban scenes in two or three dimensions, made up of a large number of buildings. One of the applications of the invention is in fact the analysis of real images of building facades in order to model them and then allow an approximate but rapid reconstruction of these images during a navigation in a virtual environment.

Currently, existing geographic information systems are able to model buildings over a large area and offer fluid navigation interfaces that make it easy to understand an urban environment in two or three dimensions. In order to make the urban environment realistic, for example to virtually visit Paris on a terminal, it is necessary that the textures displayed on each building are similar to real facades and correct resolutions. However, the data of such environments are generally stored on a remote server, so it is not possible to send the actual images of the facades of each building to the terminal as and when the visit, because the amount of information to to transmit would be too important. To overcome this problem, current systems propose to display generic texture images by repeating them several times within the virtual urban environment. This allows a small amount of information to be sent, but results in a significant decrease in overall navigation quality because the textures displayed are not visually close to the actual buildings.

In order to reconstruct more granular building facades, modeling languages to define facade elements, such as windows or balconies, regardless of a background texture, have been proposed in different works. The article "Modeling urban scenes for LBMS" by AF Coelho, AA de Sousa and FN Ferreira, published in 2005 on the occasion of the tenth international conference "3D Web Technology", proposes an XML grammar, according to the English "Extensible Markup Language", which allows to fully describe large urban environments in three dimensions, and from these descriptions to reconstruct buildings. The cities are thus modeled with many details.

Similarly, to ensure both navigation in a realistic urban environment, a good resolution and an acceptable amount of data to transfer, it has been proposed in the standard "Motion Picture Expert Group 4" (MPEG4), in its part "Animation Framework extension", to characterize the facade of a building, not by a single texture image, but by "procedural models of facades". These contain facade reconstruction information, from a number of facade element models, such as windows, doors, balconies, and background textures, which are stored in a window. catalog. These models make it possible to recreate images of façades visually close to the original facades, of any size, by transmitting a small amount of information to the terminal.

These visualization techniques of scenes are however limited because they require the intervention of an operator to manually model the buildings and the facades, which is very expensive when the environment to be modeled is of urban dimension in particular. As a result, the analysis of facade images to automatically fill procedural models of facades has become an important issue in the field of image analysis.

However, there are few methods to automatically analyze such images. Given the diversity of façade architectures, types of exterior cladding and types of façade elements, one of main difficulties to be overcome is to isolate the facade elements in a facade image.

K.Schindler and J.Bauer, in their article "A Model-based Method for Building Reconstruction", published in 2003 at a "Higher-Level Knowledge in 3D Modeling and Motion" workshop at an international conference Conference on Computer Vision (ICCV) ", presents a method of facade analysis, in which the windows of a facade to be analyzed are detected by a scanning algorithm based on the density of points not incident to the plane of the facade. This method thus detects the windows as being objects behind the plane of the facade, but requires several starting images of the facade, and their combination according to the parameters of the taking of images, which is complex and impractical to implement .

Similarly, the article by H. Mayer and S. Reznik, entitled "Building Facade Interpretation from Image Sequences", and published in the archives of the international society "International Society for Photogrammetry and Remote Sensing (ISPRS)", uses a method based on on that of K.Schindler and J.Bauer to insulate the windows of a facade before positioning them and detecting them more finely on the plane of the facade. This method therefore has the same disadvantages as the previous method, in terms of complexity and implementation.

A different method of facade image analysis is developed in the article "Extraction and Integration of Window in a 3D Building Model from Ground View Image" by SC Lee and R. Nevatia, published at the conference ICCV 2004. This method allows to isolate facade elements of a facade from a single photographic image taken from the street. This method analyzes histograms of vertical and horizontal contours of the facade image, to cut the image into cells each containing a facade element. However the analysis of these histograms does not allow to cut without error and precisely the facade images in cells. Indeed, some irregularities of the facades, such as decorative edges or decorative lines, give very sharp histograms and very difficult to analyze. The histograms used are therefore not very robust to the variations existing in the images of facades.

It should be noted that these image analysis techniques are applied to facade images, but they could be applied more generally to the detection of unit elements in structured images, i.e. images having two directions in which are arranged elements distinguished from a background or background. Indeed, these techniques exploit the fact that in a facade image, the facade elements are aligned vertically and horizontally with respect to the axes of the facade on a background image.

Similarly, if one of the applications of the invention is the analysis of images of building facades, to extract elements such as windows, doors and balconies to build the procedural models of associated facades, the invention is also applicable to any image containing elements aligned along two axes.

It is an object of the present invention to overcome the disadvantages of the prior art by providing a method and apparatus for analyzing such an image, as well as a facade image modeling method and system. To this end, the invention proposes a method of analyzing an image containing unitary elements to be detected, aligned in two horizontal and vertical directions, comprising the steps of:

- Detection of contours and obtaining an outline image,

Calculation of a histogram in a vertical or horizontal direction, representative of said contour image, respectively called vertical contour histogram or histogram of horizontal contours, said method being characterized in that it comprises the additional steps of: an image of belonging to the background corresponding to the distances between the colors of the pixels of said image containing unitary elements to be detected and an average background color corresponding to a uniformly distributed color in an area of said image containing unitary elements to be detected, - Calculation of a histogram in a vertical or horizontal direction, representative of said image of belonging to the background, called respectively vertical background histogram or horizontal background histogram,

- Merging said vertical or horizontal background histogram respectively said vertical contour histogram or said histogram of horizontal contours, in a histogram called respectively vertical window histogram or horizontal window histogram.

Thanks to the invention, an image containing elements aligned along two axes is automatically analyzed in a robust and efficient manner.

Indeed, the invention merges several types of highly correlated histograms, those obtained from a contour image, and those obtained from a background image, to obtain so-called "window" histograms. are easy to analyze. In other words, the joint use of the background color information of the image and the contour information makes the image analysis more robust. These histograms then make it possible to cut the image containing elements aligned along two axes in cells which each contain an element to be detected, in a fast, precise and efficient manner with respect to the prior art. This division into cells is a preliminary step to the finer detection of the isolated elements in each cell, then to the "procedural" modeling of the image when it is a facade image. In addition to performing the analysis of a facade, the invention requires a single photographic image thereof.

According to a preferred characteristic, in the melting step the vertical or horizontal background histogram is respectively multiplied by the histogram of vertical or horizontal contours. The fusion of a contour histogram with a background histogram is feasible in several ways, for example by linear combination or by multiplication. The best results in terms of efficiency and robustness were obtained by multiplying these two types of histograms.

According to a preferred feature, the method of analyzing an image according to the invention further comprises a vertical or horizontal color change detection step, for separating respectively the vertical window histogram or the horizontal window histogram. in two sub-histograms each representative of a color of said image containing unit elements to be detected.

This additional step, performed prior to analyzing the window histograms for cell clipping, makes it possible to split a window histogram at locations where there are disturbances due to color changes in the image. Thus, sub-histograms are obtained in which these disturbances are not troublesome when these sub-histograms are analyzed separately. Since the ground floors of buildings often have a facade color different from that of the upper floors, this step is useful for separating a facade image to be analyzed into two sub-images to be analyzed separately, one containing the floor of the building. -chaussée and the other the upper floors.

According to a preferred characteristic, the histograms of vertical or horizontal contours are obtained after respectively horizontal or vertical filtering of said contour image. This additional filtering step makes it possible to make the histograms of vertical or horizontal contours more characteristic of the unitary elements to be detected, when these unitary elements are composed of vertical or horizontal lines, for example windows.

According to a preferred characteristic, the vertical color change detection step comprises the steps of: - Hough transformation of an image of the luminances of said image containing unit elements to be detected,

Calculating a histogram in a vertical direction, representative of the resulting image of said transformation, called vertical Hough histogram,

Calculating a histogram called a ground floor histogram by linear combination of said vertical background histogram, said vertical Hough histogram, said histogram of vertical contours, and a histogram in a vertical direction representative of said contour image filtered vertically.

The ground floor histogram obtained by these substeps has a peak corresponding to the location of the color change in the image, which makes it possible to detect this change. The use of such a histogram is more robust than a classical study of the color variations on the image, because the use of colorimetric information and information on the presence or absence of horizontal lines on the image is associated. the image.

According to a preferred characteristic, the step of calculating said ground floor histogram uses the following formula:

H _RDC (y) = 3 • H _FY (y) + 2 • H _{y γ} (y) + H _m (y) - H _m (y)

OR

- H _RDC (y) is the value of said ground floor histogram in a variable y of line of said image, - H _fY (y) is the value of said vertical background histogram in said variable y,

- H _{1 γ} (y) is the value of said histogram representative of said vertically filtered contour image, in a vertical direction, in said variable y, H _m {y) is the value of said vertical Hough histogram in said variable y,

and H _m {y) is the value of said histogram of vertical contours in said variable y. This linear combination of different histograms obtained from the image makes it possible to maximize the peak value of the window histogram corresponding to the color change to be detected, compared to the values of the other peaks of the histogram.

According to another characteristic, both a horizontal window histogram and a vertical window histogram are computed, and the analysis method according to the invention comprises the additional steps of:

segmentation of said image containing unitary elements to be detected into cells each comprising one of said unitary elements of said image, by analysis of said horizontal and vertical window histograms,

detecting said unitary elements by using a rectangular region growth process using said background image and said contour image.

The joint use of the image belonging to the background and the contour image during the detection of the unitary elements improves the effectiveness of this detection, prior to for example a procedural facade modeling.

According to a preferred characteristic, the cell segmentation step uses a frequency filtering of at least one of said horizontal and vertical window histograms.

This frequency filtering facilitates the division into cells because it smooths the histograms, which allows to use segmentation thresholds without requiring complex analysis.

The invention also relates to a method of modeling an image containing unitary elements to be detected, comprising the steps of: inverse perspective transformation of said image,

analysis according to the invention of said image thus transformed,

classification of the elements of said image, detected at the end of said analysis, and background textures of said image, from a catalog of unitary elements and background textures,

modeling said image from said catalog.

In the case of the analysis of a facade image, the unitary elements to be detected are facade elements such as doors, balconies or windows. The façade image is decomposed, in this modeling, into facade elements superimposed on a background texture image that characterizes its coating.

This modeling process provides procedural facade models, which are used to reconstruct a facade texture image visually close to the actual facade. Thanks to the invention this obtaining of models is automatic, effective, and requires only one photograph per building facade. Only the most characteristic facade elements of the façades to be modeled are retained in the catalog of facade elements and background textures. The visualization on a remote browser of models in two or three dimensions of buildings faithful to reality, with textured facades of high resolution, is possible.

The invention also relates to a device implementing the method of analyzing an image according to the invention, or the method of modeling an image according to the invention.

The invention also relates to a system for modeling an image containing unitary elements to be detected, comprising:

image analysis means implementing the method for analyzing an image according to the invention,

and means for classifying unitary elements and background textures. The invention also relates to a signal representative of visibility data, carrying an image model obtained by analyzing an image. containing unitary elements aligned in two horizontal and vertical directions, said analysis having resulted in an image of contours and histograms of horizontal and vertical contours, representative of said contour image in a respectively horizontal and vertical direction said signal being characterized in that a background image having been calculated from said image, and so-called vertical and horizontal background histograms representative of said background image in a respectively vertical and horizontal direction; having been calculated, said analysis uses a merge of said vertical background histogram with said histogram of vertical contours and said horizontal background histogram with said histogram of horizontal contours.

The invention finally relates to a computer program comprising instructions for implementing one of the methods according to the invention, when it is executed on a computer.

The device, the modeling system, the signal and the computer program have advantages similar to those of the methods.

Other features and advantages will appear on reading a preferred embodiment described with reference to the figures in which:

FIG. 1 represents various steps of the facade image modeling method according to the invention, as described in this embodiment,

FIG. 2 represents treatments operated on a "perspective" façade image, that is to say taken from the street in a manner that is not orthogonal to the plane of the image;

FIG. 3 represents different steps of the image analysis method according to the invention,

FIG. 4 represents the obtaining of contour histograms from an outline image, FIG. 5 represents steps of a step of the image analysis method according to the invention,

FIG. 6 shows the frequency filtering of a window histogram obtained by the image analysis method according to the invention; FIG. 7 represents an analysis of the periodicity of a horizontal window histogram obtained by the method; According to the invention, FIG. 8 represents an embodiment of the facade image modeling system according to the invention.

According to a preferred embodiment of the invention, the image analysis method according to the invention is used to analyze facade images, and is integrated in the facade image modeling method according to the invention. However, this embodiment is easily adaptable to the analysis of structured images other than facade images, the principles of the invention remaining unchanged. The facade image modeling method is represented in FIG. 1 in the form of a flowchart comprising seven steps E1 to E7, the steps E2 to E5 being part of the image analysis method according to the invention.

The first step E1 of the modeling method is the inverse perspective transformation of an IP facade image to be analyzed, represented in FIG. 2. This step is necessary because the IP facade image is conventionally acquired by a system of photography photographic from the street. The facade represented on the IP image is then deformed according to perspective transformations, and contains elements foreign to the facade, such as sidewalk ends. It is therefore necessary to crop the IP image on the edges of the facade and straighten it, in order to obtain a spelling image IO facade. The initially colored IO image is transformed into a grayscale image. For a good performance of the modeling method according to the invention, this IO image must be of good resolution but not too large in order to limit the computation times necessary for the analysis of the image. In this embodiment, it is for example of resolution 700 ^* 500 pixels.

This step E1 is supervised and requires the manual selection of the four corners of the facade. The position of the four points within the image and the assumption that the facade is flat, allow to calculate the perspective transformation parameters undergone by the facade and to apply the inverse transformation to recover the IO rectified image.

As a variant, this step is performed automatically, since the exact position of the shooting corresponding to the IP image is known, the parameters of the camera and the dimensions of the facade represented on the camera. IP image.

The second step E2 is the creation of histograms called "window", from the image IO obtained in step E1. These histograms are characteristic of the positions of windows or other elements aligned with the IO facade image. Their analysis, described later in step E4, thus makes it possible to segment the IO image into cells each containing only one facade element. This step E2 of the modeling method according to the invention is broken down into steps a1 to a6 shown in FIG.

The first step ai is an edge detection on the facade image IO, making it possible to obtain an image IC of outlines, represented in FIG. 4. Examples of methods making it possible to extract the outlines of an image are given in FIG. the following articles:

- "Image Field Categorization and Edge / Comer Detection from Gradient Covariance", by S. Ando, published in 2000 in the magazine "Institute of

Electrical and Electronics Engineers (IEEE) Transactions on Pattern Analysis Machine Intelligence,

- and "Edge Detection with Embedded Confidence" by P. Meer and B. Georgescu, published in 2001 in the same magazine. The second step a2 is a directional filtering of this IC contour image: the vertically filtered image IC gives a vertically filtered contour image IFV, represented in FIG. 4, freed from vertical lines,

and the horizontally filtered IC image gives a horizontally filtered contour image IFH, free of horizontal lines.

The third step a3 is the calculation of contour histograms from the IFV and IFH filtered contour images. The pixel values of these images are summed line by line to give histograms in a vertical direction, and column by column to give histograms in a horizontal direction. We thus obtain four histograms:

a histogram of HVX horizontal contours, obtained by column-by-column summation of the pixels of the IFV filtered image, which characterizes the facade by having high values in the zones of the image containing facade elements, and troughs in the inter-element zones,

an HHX contour histogram, obtained by column-by-column summation of the pixels of the IFH filtered image, which has peaks in the vertical edge zones of the facade elements, or in the zones of vertical lines on the facade,

a histogram of vertical contours HHY, obtained by row-by-line summation of the pixels of the horizontally filtered image IFH, which characterizes the facade by having strong values in the zones of the image containing facade elements, and troughs in the inter-element zones,

and an HVY contour histogram, representative of the vertically filtered contour image IFV, obtained by row-by-line summation of the pixels of the IFV filtered image, and which has peaks in the horizontal edge zones of the facade elements, or in areas of horizontal lines on the facade. Only horizontal HVX and vertical HHY contour histograms are used for calculating the window histograms in this step E2, the other histograms being used for the detection of a color change on the facade in step E3. The fourth step a4 is the calculation of an image belonging to the background IAF, represented in FIG. 2. Each pixel of this image represents the distance, coded in gray levels, between the average background color of the image of the image. IO facade, and the corresponding pixel of the IO facade image. This distance is defined as a difference in gray level values. Thus, the more the areas of the IO facade image are close to the middle color of the background, the more black they appear on the IAF background image, because the distance between the corresponding pixels and the average background color is small. .

To characterize the average background color, the algorithm "Largest Empty Rectangle", described in the article "Automatic Techniques for Texture Mapping in Virtual Urban Environments", by RG Laycock and AM Day, published in 2004 on the occasion of an international conference "Computer Graphics International", or in the article "Computing the largest empty rectangle Source" by B. Chazelle, RL Drysdale and DT Lee, published in 1986 in the magazine "Society for Industrial and Applied Mathematics" (SIAM ) Journal on Computing ", is used. For a set of randomly distributed points on the image, this algorithm looks for rectangles of uniform colors. These regions make it possible to analyze the uniformly distributed colors in the image and to detect the average background color.

It should be noted that the "Largest Empty Rectange" algorithm is applied to the initial IO facade image in color, transposed into the L ^* a ^* b * coding system created by the International Commission on Illumination (CIE). ). The distances calculated between the average background color of the facade image I0, and the corresponding pixel of the facade image I0 are distances expressed in the coding system L * a ^* b ^* , because this coding system color makes it possible to code a distance between colors coherent with the distances perceived by the human eye.

The fifth step a5 is the calculation of so-called "background" histograms representative of the image of belonging to the IAF background in the vertical and horizontal directions:

the vertical background histogram HFY, represented in FIG. 2, is obtained by row-by-row summation of the values of the pixels of the image belonging to the background IAF,

and the horizontal background histogram HFX is obtained by summation column by column of the values of the pixels of the image belonging to the background IAF.

These background histograms characterize the suitability of a row or column of pixels with the background color of the facade: they contain strong values on areas of the image containing facade elements, and low values on the inter-element zones.

The sixth step a6 is the fusion of histograms of vertical and horizontal contours previously calculated, with the vertical and horizontal background histograms respectively. These four histograms are relatively correlated, which makes it possible to merge them to sum up the information and increase the robustness of the analysis of the IO facade image. The great diversity of façade images requires the provision of a robust cell segmentation method to ensure a high rate of correct results. We then obtain two so-called "window" histograms, one vertical and the other horizontal, defined by the formulas: H _Window (y) = H _FY (y) - H _m {y)

H _Window (x) = H _FX (x) - H _rx (x) WHERE

y is a variable designating a pixel line of the IO facade image, x is a variable designating a column of pixels of the IO facade image,

- _remjιe H (y) is the value of the vertical window histogram pixel line y - H _Hnetιe (x) is the value of the horizontal window of the histogram by pixels of column x,

- H _Fγ {y) is the value of the vertical background histogram HFY in the row of pixels y,

H _HX (χ) is the value of the horizontal histogram HFX in the column of pixels x,

- H _HY (y) is the value of the histogram of vertical contours HHY in the line of pixels y,

and H _vx (x) is the horizontal contour histogram value HVX in the pixel column x. It should be noted that in this embodiment the values of the histograms are multiplied with one another, but other modes of fusion can be used, for example using weighted sums.

These window histograms will then be used in step E4 of segmenting the IO facade image into cells, because they contain peaks on the zones representing facade elements and valleys in the inter-element zones.

The third step E3 of the modeling method according to the invention is the detection of a color change on the facade image IO. In fact, the window histograms obtained in step E2 are disturbed at the levels of color changes on the facade. In this embodiment of the invention, only a color change is detected in a vertical direction, because the buildings very often have ground floors coated with a different color from that of the upper floors. However, it is easy to adapt this embodiment to the detection of a color change in a horizontal direction, or with several color changes. This step E3 is broken down into four steps b1 to b4 shown in FIG.

Step b1 is a Hough transformation of an image of the luminances IL obtained from the initial IO facade image in color, and shown in FIG. 2. The image of the luminances IL is calculated by changing the space color of the IO image. The pixels of the IO image are initially represented in the Red / Green / Blue RGB color coding system according to the English "Red Green Blue", by triplets representing the values of these three components. These colors encoded in the RGB system are converted into coded colors in the CIE coding system L * a ^* b ^* . The components a * and b ^* represent the chrominance plane while the L * axis represents the luminance variations. The image of the luminances IL is obtained by keeping only the L * component of the colors L ^* a * b * thus calculated from the initial color IO image.

It should be noted that the CIE L ^* a * b ^* color space is used in the invention to calculate distances between colors and in particular during the "Largest Empty Rectange" algorithms during the detection of the background color and when constructing Background histograms. The Hough transform of the IL luminance image is performed according to a known method described in Hough PVC US3069654 entitled "Method and Means for Recognizing Complex Patterns". This transform makes it possible to detect the presence of lines within a two-dimensional signal. The resulting ITH image of this transform contains much more marked lines than those present on the IC contour image.

Step b2 is the calculation of a HTH histogram called "Hough", in a vertical direction of the resulting image ITH. This histogram is obtained by summing the pixel values of the ITH image line by line. It has very sharp peaks at horizontal lines on the ITH image and smaller peaks at the horizontal edges of elements facade. It should be noted that the Hough HTH histogram calculation does not require the complete construction of the Hough image, but only the portion of the Hough image corresponding to the vertical and horizontal lines. Step b3 is the calculation of a so-called "ground floor" histogram, obtained by linear combination of the HHY vertical contour histogram, the HFY vertical background histogram, the Hough vertical HTH histogram, and the contour histogram HVY, according to the formula:

H ^. (.y) = 3 • H _FY (y) + 2 - H _vγ (y) + H _m (y) - H _HY (y) where

y is a variable designating a pixel line of the IO facade image,

- H _RDC (y) is the value of the ground floor histogram in the variable y, - H _FY (y) is the value of the vertical background histogram HFY in the variable y,

- H _vγ (y) is the value of the contour histogram HVY in the variable y.

- H _m (y) is the value of the Hough vertical histogram HTH in the variable y,

and H _HY (y) is the value of the histogram of vertical contours HHY in the variable y.

This ground floor histogram was constructed to obtain a peak on the boundary between two areas of different colors. Indeed:

on the one hand, we add the histograms which have strong values on such transition zones, ie the HVY contour histogram and the Hough HTH histogram, because they detect the horizontal lines, and the HFY background histogram because it has high values on the ground floor area, and on the other hand subtract the histogram of vertical contours HHY because it must be zero on the transition zones, which has the effect of attenuating the other peaks of the window histogram. Similarly, if it is desired to detect a color change in a horizontal direction, it suffices to perform the symmetrical linear combination thereof, with the horizontal contour histogram HVX, the horizontal histogram HFX, a histogram representative of the resulting ITH image of the Hough transform in a horizontal direction, and the HHX contour histogram. Step b4 is the detection of the peak marking the boundary between the ground floor and the floors of the facade of the image IO, on the histogram of ground floor previously calculated. We are looking for a maximum in the lower half of the histogram. Indeed, not being able to make hypothesis on the number of stages in the facade, one supposes simply that there are at least two levels in the building. The boundary between the ground floor and the floor or floors is therefore in the lower half of the IO façade image. So we go through the histogram of ground floor and, we look, on all the maxima of its lower half, the peak of greater amplitude satisfying the following criteria: H _RDC (i)> 2/3,

7 = 0 and ΣH _Fe , _{e /} , _e U)> 0.8 * ΣH _Fenelιe (j)

or

i is a pixel line variable of the IO facade image corresponding to a peak on the lower half of the ground floor histogram, j is an index traversing the N values of the vertical window histogram from top to bottom with respect to the facade image 10,

N is the size of the vertical window histogram,

- H _RDC (i) is the value of the histogram of the ground floor histogram in variable i,

and H _fenetιe (j) is the value of the vertical window histogram in the variable j.

This determines the line of the facade image IO corresponding to the peak ground floor. This additional criterion of selection of the maxima of the ground floor histogram makes it possible to eliminate the peaks not being on zones of variations of the average color of the bottom.

The fourth step E4 of the modeling method according to the invention is the cell segmentation of the IO facade image. For this, the window histograms obtained in step E2 are analyzed. The vertical window histogram is separated into two sub-histograms corresponding to two sub-images of the IO facade image, one containing the ground floor of the building whose facade is being analyzed, and the other containing the upper floors. This separation uses the previous E3 detection step, which provides the separation line of the vertical window histogram. It eliminates disturbances on the vertical window histogram due to the color change on the IO facade image.

As noted above, window histograms have "valleys" in the inter-element areas. The analysis of the minima of the vertical window sub-histogram representative of the stages thus makes it possible to determine the boundaries between stages. Similarly, the analysis of the minima of the horizontal window histogram makes it possible to determine the boundaries between facade elements. To facilitate the search for minima, the high frequencies of these histograms are filtered on a wavelet basis. More precisely, the signals of the window histograms are broken down into frequencies on a Daubechies wavelet basis, and are filtered by suppressing the high frequencies. This makes it possible to smooth the histograms, as shown, for example, in the diagram of FIG. 6, having on the abscissa the axis AX1 of lines of pixels and on the ordinate the axis AY1 of values of the vertical window sub-histogram representative of the stages: The unfiltered signal SO of this sub-histogram has many peaks due to irregularities of the facade elements, while the corresponding SF1 filtered signal has only a few minima and maxima remaining to be analyzed.

Since the horizontal window histogram is generally more sensitive to noise and image disturbances than the vertical window histogram, the horizontal window histogram undergoes in this step E4, in addition to this frequency filtering, a delaying process. periodic. This process analyzes the periodicity of the horizontal window histogram by a covariance study and, if applicable, applies a set of Gaussians centered on the minimums of the curves satisfying the criterion of periodicity. The diagram of FIG. 7, having on the abscissa the axis AX2 of lines of pixels and the ordinate axis AY2 of values of the horizontal window histogram shows the signal filtered in SF2 frequencies of this histogram, a Gaussian SG applied to this filtered signal SF2, and the histogram resulting from this application SR. The filtered signal SF1 of the vertical window sub-histogram and the resulting signal SR of the horizontal window histogram are then analyzed. For this, the distribution spaces of the values of these signals are partitioned, and thresholds are dynamically fixed on these signals from the partitions created. More precisely, these partitions are classified into four groups by a K-Mean algorithm, which dynamically sets three thresholds. Thus, in the example of FIG. 6, the values of the filtered signal SF1 are distributed in four intervals according to their positions with respect to three thresholds S1, S2 and S3. These thresholds make it possible to classify the minima of the signals to keep only the global minima: one keeps only minima lower than the lowest threshold and one merges two minima if they are not separated of a maximum higher than the median threshold. This makes it possible to merge two minima that are in the same inter-element zone.

The global minima obtained by analysis of the SF1 filtered signal delimit the stages between them and make it possible to segment the facade image floor by floor. The global minima obtained by analyzing the resulting signal SR delimit the facade elements between them and make it possible to segment the floors into cells, each comprising only one facade element.

Alternatively, instead of calculating a single horizontal window histogram in step E2, step E4 computes a window histogram per floor, after step segmentation of the vertically filtered contour image, and step segmentation. of the image belonging to the background, from the global minima obtained by analysis of the filtered signal SF1. This makes it possible to refine the results, by inter-stage comparison of the limits of cells found by analysis of the different horizontal window histograms. This makes it possible to align the cells, which increases the visual quality of the segmentation, and to compare the segmentation of each stage to add or remove some cells according to the neighboring stages.

The fifth step E5 of the modeling method according to the invention is the detection of the facade elements in the cells obtained in the previous step. This step consists in precisely extracting the facade elements from the bottom of the facade. Indeed a cell is comparable to a facade element surrounded by background areas.

To accurately extract the area of each cell representing the facade element, a rectangular region growth process is used. The principle of this process is to enlarge a region having certain shape properties, here a rectangular shape, depending on a distance map calculated from the image. An example of an algorithm for growing rectangular regions to detect facade elements is described in the article by K. Schindler and J. Bauer cited above.

More precisely, the distance map is created in this step E5 by adding the values of the pixels of the membership image to the background IAF and the values of the pixels of the contour image IC. It should be noted that FIG. 4 represents for practical reasons an inverted IC contour image, the contour image IC being actually composed of white outlines on a black background. This distance map is then filtered by a Gaussian calculated according to the distribution of this map, that is to say according to the average and the variance of the map, to attenuate the noise and to minimize the values close to zero, that is to say the values of the pixels of the background areas, or the values close to one, that is to say the values of the pixels on facade elements. Finally, subtract the average value of the filtered map from all the values on this map to obtain a positive distance map for the regions representing facade elements and negative elsewhere. The map thus obtained is then supplied to the region growth process, which from a small rectangular starting area, positioned in the center of the cells as delimited in step E4, converges towards the rectangular zone maximizing the average value of the pixels of the corresponding region.

The sixth step E6 of the modeling method according to the invention is the classification of the facade elements and background textures of the facade image IO, from a catalog of facade elements and background textures. This catalog is for example contained in a database BD2, represented in FIG. 8, the modeling method according to the invention being implemented in a software manner in a servicing data server SERV, connected to a database BD1 of FIG. facade images of a city. In a variant, the modeling method according to the invention is implemented on several machines operating in parallel. The facade element catalog is updated as the SERV server parses the front images of the database

BD1. It contains images to reconstruct synthetic façade images as closely as possible to actual façade images. These images are of two types:

images of background textures of the facades, of small dimensions, which are repeated on the background of the computer-generated images to create high-resolution images,

and images corresponding to the elements of facades extracted from the cells during step E5.

The background texture images characterizing the stages of the facade image are created using the distance map created in step E5 and from which the unitary elements of facades were extracted. This map is searched to find the largest rectangular area for which the sum of the pixels component is minimum. A small uniform rectangular region corresponding to this area is then extracted into the initial color IO facade image, which is repeated to recreate a background image of the same size as the stages of the IO facade image. Another background texture image is similarly created to characterize the ground floor.

In order to allow a fast reconstruction of facade images from a remote terminal connected to a visualization data server, the catalog keeps only the images of facade elements and the most characteristic background texture images of the images analyzed in according to the invention. The constitution of the catalog is carried out at the same time as the analysis of the images of facades, in order to optimize the construction of the procedural facade models.

For this purpose, the images corresponding to the elements of façades extracted during step E5 are analyzed by color, texture and shape descriptors. In the same way, background texture images extracted from the facade images of the database BD1 are analyzed by descriptors of colors and textures. According to the result of the color change detection step E3, several background texture images per facade image are possibly described. The description of each element makes it possible to represent them in a large description space and to measure in this space the similarity between elements. The two types of images stored in the catalog require two description spaces, one for background texture images and the other for images corresponding to facade elements. Indeed, although the descriptors used to describe the background texture images and the descriptors used to describe the images corresponding to the extracted facade elements are partly identical, in particular for the description of the colors and textures, the latter images are analyzed. separately. They are also described by shape descriptors using a region approach, for example using the transformation Angular Radial Transform (ART).

The color descriptors are created by calculating color histograms of the images to be described. This approach is proposed by MJ Swain and DH Ballard in their article "Color indexing" published in 1991 in the magazine "International Journal of Computer Vision". It consists of representing the pixels of the images in the form of histograms of the values of their colors. In this embodiment of the invention, three histograms are created, one per color component of the pixels described in the CIE color space L * a ^* b ^* . These histograms are quantized to reduce the description space and the value of each quantization is stored as a descriptor.

Texture descriptors use an image texture analysis. This analysis is based on the work of Yong Man Ro, Kim Munchurl, Kyung Kang Ho, BS Manjunath, and Jinwoong Kim, described in the article "MPEG-7 Homogeneous Texture Descriptor" published in 2001 in "Electronics and Telecommunications Research" magazine. Institute (ETRI) journal ". Once all the background elements and textures of the facade images of the database BD1 indexed using these descriptors in the catalog of the database BD2, the two description spaces are segmented into classes of similar elements. by the algorithm of dynamic clouds. This algorithm is an automatic classification method that aims to partition space into a certain number of classes. It is described in particular in the following articles:

- P. Lloyd's "Least squares quantization in pulses-code modulation" (PCM), published in 1982 in the magazine "IEEE Transactions on Information Theory",

and J. B. MacQueen's "Some Methods for Classification and Analysis of Multivariate Observations" published in 1967 for the fifth "Berkeley Symposium on Mathematical Statistics and Probability". The choice of the number of classes to keep, and therefore the final number of background textures and facade elements in the catalog, is defined according to the accuracy of the desired realism and the size of the catalog that one wishes.

At the end of this segmentation, each image of background texture or facade element is associated with its class. This association is maintained for each facade image in the database BD1, which also stores the position of the elements or textures on the corresponding facades. Then we delete from the catalog all the images that are not defined by the dynamic clouds algorithm as a center of a class. This makes it possible to keep only a few facade elements, while ensuring that each element will have a visually close class representative who will replace it in the computer-generated images.

The facade image IO is associated in this embodiment with two background texture images, one for the ground floor and the other for the upper floors. Each of these texture images are classified, at the end of step E6, in a very similar class of background texture images represented by the image of the center of this class. Similarly, façade elements extracted in step E5 are classified into classes each represented by a very similar façade element. Depending on the desired realism or the type of facade analyzed, for example if several color changes of the facade are detected in step E3, the facade image IO is associated with more images of background textures: for example one associates with each floor of the IO facade image a background texture image.

Finally, the seventh step E7 of the modeling method according to the invention is the modeling of the facade image IO. This modeling is for example the procedural modeling proposed in the MPEG4 standard, in the "Animation Framework extension" part. It is carried out by filling modeling parameters defining in particular:

- the position and size of the ground floor and floors, - the number of floors,

- for each floor, the number and position of the cells

a class number associated with the texture image characterizing the stages of the IO facade image,

a class number associated with the texture image characterizing the ground floor of the IO facade image,

for each cell, their size and a class number associated with a facade element possibly present in this cell, as well as its position in the cell, its size, etc.

The server SERV of visualization data of a city is connected to client terminals by a communication network RES, represented in FIG. 8. When a user navigates virtually in the city using a browser on a terminal remote T connected to the server SERV by the network RES, the terminal T asks the server SERV, by a query REQ visualization data transmission to display corresponding to the location where the user is virtually in the city. The SERV server then sends the terminal T a REP response containing data facades to display. These data include the procedural models of the facades to be viewed, and all or part of the catalog of facade elements and background textures. The information signal conveying the data of the REP request is therefore much more compact than if it were necessary to transmit the real images of the facades, and allows a reconstruction of good resolution and similar to the original facades.

Claims

A method of analyzing an image (IO) containing unitary elements to be detected, aligned in two horizontal and vertical directions, comprising the steps of:

- Detection (ai) of contours and obtaining an outline image (IC), - Calculation (a3) of a histogram in a vertical or horizontal direction, representative of said contour image (IC), respectively called contour histogram vertical (HHY) or horizontal contour histogram (HVX), said method being characterized in that it comprises the additional steps of:

Calculation (a4) of a background image (IAF) corresponding to the distances between the colors of the pixels of said image (IO) containing unit elements to be detected and a mean background color corresponding to a uniformly distributed color in an area of said image (IO) containing unitary elements to be detected,

Calculating (a5) a histogram in a vertical or horizontal direction, representative of said background membership image (IAF), respectively called vertical background histogram (HFY) or horizontal background histogram (HFX),

- Merging (a6) of said vertical (HFY) or horizontal (HFX) vertical histogram with respectively said vertical contour histogram (HHY) or said horizontal contour histogram (HVX), in a histogram respectively called vertical window histogram or histogram of horizontal window.

2. An image analysis method (1O) according to claim 1, characterized in that in the melting step (a6) the vertical (HFY) or horizontal (HFX) vertical histogram is multiplied by 1 histogram of vertical (HHY) or horizontal (HVX) contours.

The method of analyzing an image (IO) according to claim 1 or 2, further comprising a step of detecting (E3) vertical or horizontal color change, for respectively separating the vertical window histogram or the horizontal window histogram in two sub-histograms each representative of a color of said image (IO) containing unitary elements to be detected.

An image analysis method (IO) according to any one of claims 1 to 3, wherein the histograms of vertical contours

(HHY) or horizontal (HVX) are obtained after respectively horizontal or vertical filtering of said contour image (IC).

The method of analyzing an image (IO) according to claims 3 and 4, wherein the step of detecting (E3) vertical color change comprises the steps of:

- Hough transformation (b1) of a luminance image (IL) of said image (IO) containing unit elements to be detected, - Calculation (b2) of a histogram in a vertical direction, representative of the resulting image ( ITH) of said transformation, called vertical Hough histogram (HTH),

Calculating (b3) a histogram called a ground floor histogram by linear combination of said vertical bottom histogram (HFY), said histogram of vertical Hough

(HTH), of said histogram of vertical contours (HHY), and a histogram (HVY) in a vertical direction representative of said vertically filtered contour image (IFV).

An image analysis method (IO) according to claim 5, wherein the step of calculating (b3) said ground floor histogram uses the following formula:

H _RDC (y) = 3 - H _n (y) + 2 • H _{ι γ} (y) + H _n (y) - H _m (y) OÙ - H _RDC (y) is the value of said ground level histogram -shipped in a variable y of line of said image (IO),

- H _FY (y) is the value of said vertical bottom histogram (HFY) in said variable y,

- H _vγ (y) is the value of said histogram (HVY) representative of said vertically filtered contour image (IFV), in a vertical direction, in said variable y,

H _m (y) is the value of said vertical Hough histogram (HTH) in said variable y,

and H _HY (y) is the value of said histogram of vertical contours (HHY) in said variable y.

7. A method of analyzing an image (IO) according to any one of claims 1 to 6, wherein one calculates both a horizontal window histogram and a vertical window histogram, characterized in that it comprises the additional steps of:

segmentation (E4) of said image (IO) containing unitary elements to be detected in cells each comprising one of said unitary elements of said image (IO), by analysis of said horizontal and vertical window histograms, detecting (E5) said unitary elements by using a rectangular region growth process using said background membership image (IAF) and said contour image (IC).

The image analysis method (10) of claim 7, wherein the cell segmentation step (E4) uses frequency filtering of at least one of said horizontal and vertical window histograms.

An image modeling method (IP) containing unitary elements to be detected, comprising the steps of:

transformation (E1) inverse perspective of said image (IP),

- analyzing (E2, E3, E4, E5) of said image thus transformed (IO) according to any one of claims 1 to 8,

classification (E6) of the elements of said image, detected at the end of said analysis (E2, E3, E4, E5), and background textures of said image, from a catalog of unitary elements and background textures,

modeling (E7) of said image (IP) from said catalog.

10. Device (SERV) comprising means adapted to implement one of the methods according to any one of claims 1 to 9.

An image modeling system (IP) containing unitary elements to be detected comprising:

image analysis means implementing the analysis method according to any one of claims 1 to 8,

and means for classifying unitary elements and background textures.

12. Representative visibility data signal, carrying an image analysis (IO) image model containing unit elements aligned in two horizontal and vertical directions, said analysis giving rise to an image of contours (IC) and histograms of horizontal (HVX) and vertical (HHY) contours, representative of said contour image in a respectively horizontal and vertical direction, said signal being characterized in that a background image (IAF) having been calculated from said structured image, and so-called vertical (HFY) and horizontal (HFX) histograms representative of said background image (IAF) ) in a respectively vertical and horizontal direction having been calculated, said analysis uses a merge of said vertical background histogram (HFY) with said vertical contour histogram (HHY) and said horizontal background histogram (HFX) with said histogram of horizontal contours ( HVX).

A computer program comprising instructions for implementing one of the methods according to any one of claims 1 to 9 when executed on a computer.