KR101264358B1 - Method and System for Automated Photomosaic Image Generation - Google Patents

Abstract

One embodiment of the present invention is directed to a method and system for automated photo mosaic image generation.
To this end, an embodiment of the present invention includes a data generation step of dividing a photo or tile used in a mosaic into a plurality of regions, extracting and storing color values of the divided regions; And a photo mosaic image generation step of generating a photo mosaic of an image input from the outside by using the color values of the stored regions.

Description

Method and System for Automated Photo Mosaic Image Generation {Method and System for Automated Photomosaic Image Generation}

One embodiment of the present invention is directed to a method and system for automated photo mosaic image generation.

Mosaic is a kind of decorative art that makes patterns or images by attaching small pieces or tiles of multicolor to basic sketches. In addition, the mosaic image can see small pieces at a close position, and can recognize a single large shape that the pieces are made at a far position. This mosaic work has been mostly done by artists in the past, and is now being carried out by computer graphics researchers as one of the unrealistic rendering techniques. Images made with this technique enhance the interest or artistry of the image.

In the past, such a mosaic had to collect a lot of data for a long time, but recently, with the introduction of a lot of low-cost digital devices, this problem is solved a lot, and the emergence of software that makes it easy for general users to use the mosaic processing Interest has increased. In recent years, non-realistic rendering techniques have been implemented by many graphic designers as well as by some researchers, and various mosaics have been introduced.

Photo mosaic using photo among these various mosaic methods is actively used in magazines, posters, installations, music videos, and TV commercials due to the recent development of digital imaging technology. It means making a big picture. However, since most of the mosaic image creation tasks are manually generated by professional graphic designers, it is difficult for general users to create photo mosaic images, and even though they are professional graphic designers, they are quite manual for creating mosaic images. There was a time-consuming problem. In addition, the development of software for automating the photo mosaic algorithm has been made, but there are still few commercial programs, and the performance is also not satisfactory.

One embodiment of the present invention provides an automated photo mosaic image generation method and system capable of generating an automated photo mosaic image through adaptive tiling and block matching.

In addition, an embodiment of the present invention generates a tile used for a mosaic through a photographic database, and generates a mosaic image having excellent qualitative quality through the process of image feature extraction, adaptive tiling, block matching and contrast adjustment An automated photo mosaic image generation method and system can be provided.

An automated photo mosaic image generation method according to an embodiment of the present invention comprises the steps of: generating a data for dividing a photo or tile used in a mosaic into a plurality of regions, extracting and storing color values of the divided regions; And a photo mosaic image generation step of generating a photo mosaic of an image input from the outside by using the color values of the stored regions.

The data generating step may include a data input step of receiving a photo or tile used for the mosaic; A division step of dividing the input photo or tile into a plurality of areas having a predetermined size; An average value calculation step of extracting color values of the divided regions and calculating an average value thereof; And a feature value storing step of storing the calculated average value as a feature value of the photo or tile.

The photo mosaic image generating step may include: an image feature extraction step of dividing an image input from the outside into a plurality of blocks having a predetermined size and extracting an average value of color values of the divided region as a feature value; An adaptive tiling step of determining similarity between any one reference block of the plurality of blocks and adjacent blocks adjacent to the reference block and merging similar blocks; A block matching step of extracting a similar picture or tile by determining a similarity between the divided or merged block with respect to the input image and the picture or tile in the data generation step; And a contrast adjustment step of replacing the contrast value of the extracted photo or tile with the contrast value of a block similar to the photo or tile. The color value is an RGB value.

In order to compare the similarity between adjacent blocks in the adaptive tile step, a difference of block feature values is determined by Equation 1,

[Equation 1]

Where SB is the reference block, B is the neighboring block, i is the level of the block, j is the block number in the level, SB (k) is the feature value of the reference block, and B _{i, j} (k) is the neighboring block. Characteristic value of the j block of the i level.

In the adaptive tile step, if the difference between the feature values between the blocks is less than or equal to a preset reference value, the blocks are merged.

The similarity (MatchBlock) in the block matching step is determined by Equation 2,

&Quot; (2) "

Here, SB (k) is the feature value of the k- th block divided or merged, DB _i (k) is the feature value of the k- th photo or tile in the data generation step.

In the block matching step, a picture or a tile in the data generation step having the minimum value determined by Equation 2 is extracted.

In addition, the automated photo mosaic image generation system according to another embodiment of the present invention, the data generation unit for dividing the photo or tile used in the mosaic into a plurality of areas, and extracts and stores the color values of the divided areas; And a photo mosaic image generator for generating a photo mosaic of an image input from the outside by using the color values of the stored regions.

The data generator may include a data input unit configured to receive a photo or tile used for the mosaic; A divider dividing the input photo or tile into a plurality of areas having a predetermined size; An average value calculator for extracting color values of the divided regions and calculating an average value thereof; And a feature value storage unit for storing the calculated average value as a feature value of the photo or tile.

The photo mosaic image generation unit may include: an image feature extraction unit configured to divide a plurality of blocks having a predetermined size with respect to an image input from the outside, and extract an average value of color values of the divided region as a feature value; An adaptive tiling unit configured to determine similarity between any one reference block among the plurality of blocks and adjacent blocks adjacent to the reference block and merge similar blocks; A block matching unit configured to extract a similar picture or tile by determining a similarity between the divided or merged block with respect to the input image and the picture or tile of the data generator; And a contrast adjustment unit for replacing the contrast value of the extracted photo or tile with the contrast value of a block similar to the photo or tile.

According to the method and system for generating an automated photo mosaic image according to an embodiment of the present invention, an automated photo mosaic image may be generated through adaptive tiling and block matching.

In addition, according to an automated photo mosaic image generation method and system according to an embodiment of the present invention, a tile used for a mosaic is generated through a photo database, and image feature extraction, adaptive tiling, block matching, and contrast value adjustment are performed. Through the process, a mosaic image having excellent qualitative quality can be generated.

1 is a flowchart illustrating an automated photo mosaic image generation method according to an embodiment of the present invention.
FIG. 2A is a flowchart illustrating a data generation step of FIG. 1.
FIG. 2B is a flowchart illustrating a process of generating a photo mosaic image of FIG. 1.
3A to 3H are views illustrating a process of forming the photo mosaic image generating step of FIG. 2B.
4 is a block diagram illustrating an automated photo mosaic image generation system according to another embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention.

1 is a flowchart illustrating an automated photo mosaic image generation method according to an embodiment of the present invention, FIG. 2A is a flowchart illustrating the data generation step of FIG. 1, and FIG. 2B illustrates the photo mosaic image generation step of FIG. 1. 3A to 3J are flowcharts illustrating a process of generating the photo mosaic image generation step of FIG. 2B.

1 to 2B, an automated photo mosaic image generation method according to an embodiment of the present invention includes a data generation step S100 and a photomosaic image generation step S200.

The data generating step (S100) divides the photo or tile used in the mosaic into a plurality of regions, extracts the color values of the divided regions and stores them as feature value data. In the data generating step S100, after dividing the tile into 4 × 4, the RGB average value of each region is calculated and stored as a feature value. That is, a total of 48 feature values represent the brightness value characteristics of the tile.

The data generation step S100 includes a data input step S110, a division step S120, an average value calculation step S130, and a feature value storage step S140.

The data input step S110 receives a photo or tile used for a mosaic. The data input step S110 receives photo or tile data used for a mosaic through a camera or a user interface.

The dividing step (S120) divides the photo or tile input in the data input step (S110) into a plurality of areas having a predetermined size, and the average value calculating step (S130) is the color of the areas divided in the dividing step (S120). Extract the value and compute its average.

Then, the feature value storing step (S140) stores the average value calculated in the average value calculating step (S130) as the feature value of the photo or tile. The stored feature values of the photo or tile are used to determine similarity with the image in the block matching step S230 of the photomosaic image generation step S200.

The photomosaic image generating step S200 generates a photo mosaic of an image input from the outside using color values of the areas stored in the data generating step S100. Here, the color value means RGB value.

The photomosaic image generation step S200 includes an image feature extraction step S210, an adaptive tiling step S220, a block matching step S230, and a contrast adjustment step S240.

The image feature extraction step S210 is a step of dividing the image input from the outside into a plurality of blocks having a predetermined size, and extracting an average value of color values of the divided region as a feature value. That is, the image feature extraction step S210 is a process of calculating the feature of each block after dividing the image input from the outside into small blocks. The calculated feature is utilized in the subsequent adaptive tiling step (S220) and block matching step (S230). More specifically, the image feature extraction step (S210) divides the input image into blocks having a predetermined size as shown in FIG. 3A, and for each block into 4 × 4 16 regions as shown in FIG. 3B. By dividing, the RGB average value is calculated for each area and used as a feature of each block. Thus, 48 feature values can be extracted for each block. The process of extracting the feature values is the same as the process of extracting the feature of the photo or tile in the segmentation step S120 and the feature value storage step S140 in the data generation step S100. In the present invention, the block size is set in units of 64 pixels as an example, but the block size is not limited thereto.

The adaptive tiling step (S220) is a step of merging similar blocks by determining similarity between any one reference block among a plurality of blocks and adjacent blocks adjacent to the reference block. On the other hand, in generating a photo mosaic image, the quality of the qualitative image may be improved when it is generated by including tiles of various sizes rather than the same tile size. Therefore, in the adaptive tiling step (S220), if the feature values are similar between the blocks generated in the image feature extraction step (S210), adjacent blocks are merged to expand the size of the block. Specifically, to explain the adaptive tiling step S220, it is assumed that blocks are divided as shown in FIG. 3C. Here, the block SB is a fixed block to be compared for merging with neighboring blocks, and the blocks B _{i and j} mean a j th block corresponding to Level i based on the block SB . The adaptive tiling step S220 determines that the contents of the blocks included in Level i are similar when the error between the fixed blocks SB and the feature values of all the blocks B _{i and j} corresponding to Level i is ± 10. To perform the merge. If any of the blocks of Level i is out of the error range, merging of the Levels is not performed and the adaptive tiling process for the fixed block SB is stopped. For example, comparing the block SB with blocks B _1,1 , B _1,2 , and B _1,3 in Level 1 means that tiles can be merged if the similarity is within ± 10. Similarity comparisons continue with B _2,1 , B _2,2 , B _2,3 , B _2,4 , and B _2,5 . If any one of the blocks of Level 2 is out of the error range, the merge with the Level 2 blocks is not performed, and the size of the block SB is determined to be twice the size of the x and y axes. As the level increases, the size of the merged block increases, but if the block is too large, the quality of the mosaic image is deteriorated, so the number of levels is limited.

The adaptive tiling step (S220) is used to calculate the difference ( Difference ) between the feature values of the block as shown in Equation 1 below to compare the similarity of the blocks.

[Equation 1]

Here, Difference is a similarity between blocks, SB is a reference block, B is an adjacent block, i is a level of the block, j is a block number in the level, SB (k) is a feature value of the reference block, B _{i , j} (k) is a feature value of the j block of the i level among the adjacent blocks.

3d is a diagram illustrating an expanded view of a block after the adaptive tiling step (S220). In FIG. 3D, a block marked with 1 denotes a fixed block, and a block marked with 0 represents an expanded block based on the fixed block. In this way, by utilizing the characteristics of the blocks of 1 and 0, in the adaptive tiling step (S220), the block in which 0 is displayed is a merged block. Will not. In addition, the speed can be improved by not performing the block matching step S230 in the next step.

The block matching step (S230) is a step of extracting a similar picture or tile by determining a similarity between the divided or merged block with respect to the input image and the picture or tile in the data generation step (S100). That is, the block matching step (S230) is a process of matching to find the photos or tiles matched in the data generation step (S100) for each of the divided blocks of the input image. In addition, in the block matching step S230, since the qualitative quality is deteriorated when the same picture is inserted in the adjacent block, the overlapping may be minimized to prevent the insertion of the same image in the adjacent block. Further, the block matching step (S230) can improve the performance speed by removing the route calculation by modifying the Euclidean distance, which calculates the distance between two points as a measure of the similarity between blocks. .

As such, the image matched through the block matching step S230 corresponds to an image having at least similarity values among similarity values as shown in Equation 2 below.

&Quot; (2) "

Here, MatchBlock is the similarity between the block and the picture or tile, SB (k) is the feature value of the k- th block divided or merged, DB _i (k) is the feature value of the k- th picture or tile in the data generation step.

Meanwhile, in generating a photo mosaic image, using overlapping and fixed tile sizes of the same tile may degrade qualitative quality. Therefore, in the block matching step (S230), if the corresponding tile is used in the neighboring blocks to minimize the redundancy of the tile, the overlap minimization process (not shown) of searching for the tile having the other best value as a matched image is used. Can be rough. Through this overlap minimization process, it is possible to limit the repetitive use by measuring the number of times the photo or tile is used in the data generation step (S100).

More specifically, in the block matching step S230, the process of block matching with respect to blocks starts at the upper left end and proceeds to the lower right direction. Therefore, at the time when block matching is performed on an arbitrary block, only the blocks on the left and the top are already completed. In the present invention, for the redundancy check, as shown in FIG. 3E, a search is performed on neighboring blocks included in a range of 45 to 180 degrees based on the matching target block, and the depth for searching is set to 3. If the matching target block is B _{i , j} and the neighboring blocks are B _{k , l} , the max (| ik |, | jl |) value, that is, the maximum value of the distance difference means the depth of the search. In this case, when the depth of the search is small, a similar photo is likely to appear. When the depth of the search is large, a large number of target blocks to be compared in the process of block matching require a lot of execution time. In addition, since the adaptive tiling step S220 is performed, when the neighboring blocks to be compared are previously merged and expanded blocks, tile information allocated to the fixed block SB representing the corresponding block is used.

Therefore, the result generated by applying the block matching step (S230) and the overlap minimization process is shown as in FIG. 3f. Here, the result before the redundancy is removed on the left side, and the quality of the qualitative mosaic image is improved on the right side as a result of minimizing the redundancy.

The contrast adjustment step S240 is a step of replacing the contrast value of the picture or tile extracted in the block matching step S230 with the contrast value of a block similar to the picture or tile. Meanwhile, as shown in the left side of FIG. 3G, when the photo mosaic image is generated using the image matched with the optimal value in the data generation step S100 through block matching, the color appears much different from the original image. . Therefore, in the present invention, qualitative quality is improved by adjusting the brightness of the original image block to be similar to the picture or tile matched in the data generation step S100 in the contrast adjusting step S240. To this end, the contrast adjustment step (S240) after changing the matched picture or tile and the original image block to the HSI (Hue Saturation Intensity) domain, the brightness value data of the matched picture or tile as the brightness value data of the original image block After replacement, the brightness value (i.e., the contrast value) of the matched tile is adjusted by converting to an RGB domain image. Therefore, as shown in the right side of FIG. 3F, it can be seen that the qualitative quality is improved in the boundary portion compared to the left side before the adjustment, and furthermore, a natural image can be made without overlapping the original image through this process. .

3H is a diagram illustrating a photo mosaic image generated through the image feature extraction step S210, the adaptive tiling step S220, the block matching step S230, and the contrast adjustment step S240. That is, as shown in FIG. 3H, the photo mosaic image generated through the image feature extraction step S210, the adaptive tiling step S220, the block matching step S230, and the contrast adjustment step S240 may include tiles of various sizes. By adjusting the contrast value, including the qualitative quality can be improved.

4 is a block diagram illustrating an automated photo mosaic image generation system according to another embodiment of the present invention. Here, the automated photo mosaic image generation system shown in FIG. 4 relates to a system for implementing the automated photo mosaic image generation method shown in FIGS. 1 to 3G.

As shown in FIG. 4, the automated photo mosaic image generating system according to another embodiment of the present invention may include a user interface unit 10, a data generator 100, a photo mosaic image generator 200, and a display unit. And 20.

The user interface unit 10 may be any terminal capable of generating a picture or a tile such as a camera, a mobile communication terminal, or the like. The user interface 10 is connected to the data generation unit 100 through a wired or wireless communication network to transmit data of a picture or tile for mosaic generation.

The data generator 100 divides a photo or tile used in a mosaic into a plurality of regions, and extracts and stores color values of the divided regions.

The data generator 100 includes a data input unit 110, a divider 120, an average value calculator 130, and a feature value storage 140. Here, the data input unit 110, the divider 120, the average value calculator 130, and the feature value storage 140 may include the data input step, the divide step, the average value calculate step, and the feature value shown in FIGS. 1 to 2A. As a configuration for performing the storing step, a detailed description of each configuration will be referred to with reference to FIGS. 1 to 2A.

The data input unit 110 receives a photo or tile used for a mosaic from the user interface unit 10.

The divider 120 divides the photo or tile input by the data input unit 110 into a plurality of regions having a predetermined size.

The average value calculator 130 extracts color values of the areas divided by the divider 120 and calculates an average value thereof.

The feature value storage unit 140 stores the average value calculated by the average value calculator 130 as the feature value of the photo or tile. The feature values of the stored pictures or tiles are used to determine the similarity with the image in the block matching unit 230 of the photo mosaic image generator 200.

Meanwhile, in the present invention, the data generation unit 100 is divided into functions for each function of the data input unit 110, the division unit 120, the average value operation unit 130, and the feature value storage unit 140. The functions are not limited and may be formed in one configuration according to the designer's intention.

The photo mosaic image generator 200 generates a photo mosaic of an image input from the outside by using color values of regions stored by the data generator 100.

The photo mosaic image generator 200 includes an image feature extractor 210, an adaptive tiler 220, a block matcher 230, and a contrast controller 240. Here, the image feature extraction unit 210, the adaptive tiling unit 220, the block matching unit 230 and the contrast adjustment unit 240 is an image feature extraction step, adaptive tiling step, shown in Figs. As a configuration for performing the block matching step and the contrast adjustment step, a detailed description of each configuration will be described with reference to FIGS. 1 to 3G.

The image feature extractor 210 divides the image input from the outside into a plurality of blocks having a predetermined size, and extracts an average value of color values of the divided region as a feature value. That is, the image feature extractor 210 divides the image input from the outside into small blocks and calculates the feature of each block. In addition, the image feature extraction unit 210 receives an image for generating a mosaic from the user interface unit 10.

The adaptive tiling unit 220 determines similarity between any one reference block among a plurality of blocks and adjacent blocks adjacent to the reference block and merges similar blocks. The similarity determination used in the adaptive tiling unit 220 may be determined by Equation 1 described above. Here, the adaptive tiling unit 220 merges blocks when the similarity between any one reference block among a plurality of blocks and adjacent blocks adjacent to the reference block is equal to or less than a preset reference value. Do not merge blocks.

The block matching unit 230 determines a similarity between the divided or merged block and the picture or tile of the data generator 100 with respect to the image input to the image feature extractor 210 and extracts a similar picture or tile. . The similarity determination used in the block matching unit 230 may be determined by Equation 2 described above. Here, the block matching unit 230 is a photo or tile of the data generator when the similarity between the split or merged block and the image or tile of the data generator with respect to the image input to the image feature extraction unit 210 has a minimum value. Will be extracted.

The contrast adjusting unit 240 replaces the contrast value of the photo or tile extracted by the block matching unit 230 with the contrast value of a block similar to the photo or tile. The contrast controller 240 changes the matched picture or tile and the original video block to the HSI (Hue Saturation Intensity) domain after the data generator 100 changes the brightness value data of the matched picture or tile to the original image block. By substituting the brightness value data and converting the image into an RGB domain image, the brightness quality (that is, the contrast value) of the matched tile is adjusted to improve the qualitative quality.

The display unit 20 receives the photo mosaic image generated through the photo mosaic image generating unit 200 and displays the external image. In addition, the display unit 20 may be a display device provided with a touch screen function. In this case, the display unit 20 may receive an input signal from a user and display an image corresponding thereto.

What has been described above is just one embodiment for implementing the method and system for automated photo mosaic image generation according to the present invention, and the present invention is not limited to the above embodiment, as claimed in the following claims. Without departing from the gist of the present invention, anyone of ordinary skill in the art will have the technical spirit of the present invention to the extent that various modifications can be made.

10: user interface unit 20: display unit
100: data generation unit 110: data input unit
120: division unit 130: average value calculation unit
140: feature value storage unit 200: photo mosaic image generation unit
210: image feature extractor 220: adaptive tiler
230: block matching unit 240: contrast adjustment unit

Claims (11)

A data generation step of dividing a photo or tile used in a mosaic into a plurality of areas, extracting and storing color values of the divided areas; And
An image feature extraction step of dividing an image input from the outside into a plurality of blocks having a predetermined size and extracting an average value of the color values of the divided region as a feature value, and the reference block of any one of the plurality of blocks; An adaptive tiling step of merging similar blocks by judging similarity between a reference block and adjacent adjacent blocks, and determining similarity between a block divided or merged with respect to the input image and a picture or tile in the data generation step. And a block matching step of extracting a similar picture or tile and a contrast adjustment step of replacing a contrast value of the extracted picture or tile with a contrast value of a block similar to the picture or tile. Generating a photo mosaic image;
Automated photo mosaic image generation method comprising a. The method of claim 1,
The data generation step,
A data input step of receiving a photo or tile used for the mosaic;
A division step of dividing the input photo or tile into a plurality of areas having a predetermined size;
An average value calculation step of extracting color values of the divided regions and calculating an average value thereof; And
And a feature value storing step of storing the calculated average value as a feature value of the photo or tile.
delete The method according to claim 1 or 2,
And wherein the color value is an RGB value. The method according to claim 1 or 2,
In order to compare the similarity between adjacent blocks in the adaptive tiling step, a difference between feature values is determined by Equation 1,
[Equation 1]

Where SB is a reference block, B is an adjacent block, i is a level of the block, j is a block number in the level, SB (k) is a feature value of the reference block, and Bi, j (k) is i of the adjacent blocks. And a feature value of the j block of the level. The method of claim 5,
In the adaptive tiling step, the blocks are merged when the similarity is less than or equal to a preset reference value, and the blocks are not merged when the similarity is greater than or equal to a preset reference value. The method according to claim 1 or 2,
The similarity ( matchBlock ) in the block matching step is determined by Equation 2,
&Quot; (2) "

Wherein SB (k) is the feature value of the k- th block divided or merged, and DBi (k) is the feature value of the k- th photo or tile in the data generation step. The method of claim 7, wherein
And in the block matching step, extract a photo or tile in the data generation step having the minimum value determined by Equation 2. A data generation unit for dividing a photo or tile used in a mosaic into a plurality of areas and extracting and storing color values of the divided areas; And
An image feature extraction unit for dividing an image input from the outside into a plurality of blocks having a predetermined size and extracting an average value of the color values of the divided region as a feature value; and a reference block of any one of the plurality of blocks; An adaptive tiling unit for merging similar blocks by judging similarity between the reference block and adjacent neighboring blocks, and determining similarity between a block divided or merged with respect to the input image and a picture or tile of the data generating unit A photo mosaic generator comprising a block matching unit for extracting a similar photo or tile and a contrast adjusting unit for replacing a contrast value of the extracted photo or tile with a contrast value of a block similar to the photo or tile;
Automated photo mosaic image generating system comprising a. 10. The method of claim 9,
Wherein the data generating unit comprises:
A data input unit for receiving a photo or tile used in the mosaic;
A divider dividing the input photo or tile into a plurality of areas having a predetermined size;
An average value calculator for extracting color values of the divided regions and calculating an average value thereof; And
And a feature value storage unit for storing the calculated average value as a feature value of the photo or tile.
delete

Images