CN108346129B

CN108346129B - Method for generating puzzle blocks with confusing blocks

Info

Publication number: CN108346129B
Application number: CN201810199413.5A
Authority: CN
Inventors: 傅川
Original assignee: Institute of Computing Technology of CAS
Current assignee: Institute of Computing Technology of CAS
Priority date: 2018-03-12
Filing date: 2018-03-12
Publication date: 2020-07-31
Anticipated expiration: 2038-03-12
Also published as: CN108346129A

Abstract

The invention provides a method for generating a jigsaw puzzle block with confusing blocks, which comprises the following steps: for each of the plurality of original pictures, performing picture style conversion on the original picture by using one of a plurality of style templates to obtain a corresponding synthesized picture; dividing each of the resulting plurality of composite pictures into a plurality of tiles having a predetermined size; and for the tiles with the same type, clustering based on the content similarity, and for each tile in all the tiles under the same category, taking the other tiles under the category as the confusing tiles of the tile. The invention introduces the confusion items into the jigsaw puzzle blocks so as to improve the observation power, the resolution power and the imagination of the user to a greater extent in the limited screen space of the electronic equipment.

Description

Method for generating puzzle blocks with confusing blocks

Technical Field

The invention relates to the field of information service and image processing, in particular to a generation technology of a jigsaw puzzle block.

Background

Jigsaw is a common mental tool used for training and developing the observation, resolution and imagination of children, and consists of a plurality of picture blocks which can be pieced together to form a complete picture. With the development of electronic technology, children can more conveniently develop intelligence on electronic equipment such as mobile phones, tablet computers and the like through jigsaw application. In the field of physical jigsaw, enlarging the size of a jigsaw picture or subdividing the jigsaw picture into more small pieces is a common approach to increasing the difficulty of jigsaw. However, since the screen size of the smart phone and the tablet computer is limited, the large-sized jigsaw puzzle picture is inconvenient to display. Furthermore, if the tiles are too small, operation is not facilitated and visual fatigue is easily generated.

Disclosure of Invention

In order to overcome the defects in the prior art, the invention provides the following technical scheme:

in one aspect, there is provided a method of generating a puzzle block having obfuscated tiles, comprising:

step 1) for each of a plurality of original pictures, carrying out picture style conversion on each of the original pictures by using one of a plurality of style templates to obtain a corresponding synthesized picture;

step 2) dividing each of the obtained multiple synthetic pictures into multiple image blocks with preset sizes, wherein the image blocks in the synthetic pictures obtained by using the same style templates are regarded as having the same style;

and 3) for the image blocks with the same type, clustering based on the content similarity to obtain the category to which the image block belongs, and regarding each image block in all the image blocks under the same category, regarding other image blocks under the category as the confusion image block of the image block.

In the method, in step 3), clustering the tiles having the same type based on the content similarity may include: extracting local feature points of each image block with the same type style, clustering the extracted local feature points, and enabling the center of each cluster to correspond to a visual word so as to construct a visual dictionary representing the image block; and clustering all the image blocks with the same type and represented by the visual dictionary to obtain the category of the image block.

In the method, in step 1), performing picture style conversion on the original picture by using the style template may include: and (3) converting the original picture into a synthetic picture with the same style as the style template by using a picture style conversion algorithm based on a convolutional neural network.

The method may further include, before step 1): the original picture is pre-processed to conform its length and width to predetermined dimensions.

The above method may further comprise: recording the corresponding relation between each composite picture and the picture blocks divided from the composite picture; and recording the category to which each tile belongs.

In another aspect, there is provided a method of providing a user with a puzzle tile containing confusing tiles, comprising:

providing a plurality of composite pictures to a user for selection by the user, the plurality of composite pictures being a plurality of composite pictures resulting from generating a puzzle piece having obfuscated tiles using the method described above;

according to the composite picture selected by the user, all the picture blocks divided from the composite picture are found from the picture blocks for the puzzle with the confusion picture blocks as a first group of picture blocks;

finding a aliased tile for each tile in the first set of tiles from the puzzle tiles having aliased tiles as a second set of tiles;

providing the first set of tiles and the second set of tiles to a user together.

Compared with the prior art, the invention has the advantages that:

1. the size of the jigsaw picture does not need to be enlarged or the size of the picture blocks does not need to be reduced, and the jigsaw puzzle picture is suitable for being applied to electronic equipment;

2. the confusing items are provided while the puzzle blocks are provided for the user, which is more beneficial to training and developing the observability, resolution and imagination of the user.

Drawings

Embodiments of the invention are further described below with reference to the accompanying drawings, in which:

FIG. 1 is a flow diagram of a method of generating a puzzle block with confusing tiles in accordance with one embodiment of the present invention;

FIG. 2 is a flow diagram of a method for classifying stylistic tiles based on content similarity, according to one embodiment of the present invention;

FIG. 3 is a flow diagram of a method of providing a user with puzzle tiles that include obfuscated tiles in accordance with one embodiment of the present invention;

FIG. 4 is a schematic diagram of a computer device for a method of generating puzzle tiles with obfuscated tiles in accordance with one embodiment of the invention;

fig. 5(a) is an example of an original picture;

FIG. 5(b) is an example of a style template;

fig. 5(c) is a composite picture having a style identical to that of fig. 5(b) obtained by picture-style converting the original picture in fig. 5(a) using the style template shown in fig. 5 (b).

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail by embodiments with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

It should be noted that some exemplary methods are depicted as flowcharts. Although a flowchart may describe the operations as being performed serially, it can be appreciated that many of the operations can be performed in parallel, concurrently, or with synchronization. In addition, the order of the operations may be rearranged. A process may terminate when an operation is completed, but may have additional steps not included in the figure or embodiment.

FIG. 4 shows a schematic diagram of a computer apparatus for a method of generating puzzle tiles with obfuscated tiles, comprising a processor unit and data storage mechanism for use in performing the method, the latter comprising: the system comprises a non-standard original picture library, a style template library, a synthesized picture block library, a style template record table, a synthesized picture corresponding relation table, a picture block corresponding relation table and an information temporary storage area.

The non-standard original picture library is used for storing original pictures with the length and the width not meeting the preset specification; the standard original picture library is used for storing original pictures with the length and width of the pictures meeting preset specifications; the style picture library is used for storing a plurality of style templates; the synthesized picture library is used for storing synthesized pictures obtained by performing style conversion on the original pictures by using the style templates; the synthesized picture pattern library is used for storing pattern blocks obtained after the synthesized picture is divided; and the information temporary storage area is used for storing system configuration parameters and temporary data generated in the processing process. In addition, the style template recording table is used to record an identification of the style template (for uniquely identifying the style template, a file name may be used for convenience) and a storage location thereof, as shown in table 1 below; the synthesized picture correspondence table is used to store an identifier (for uniquely identifying the synthesized picture, a file name may be used for convenience) of the synthesized picture and a storage location thereof, and a correspondence of the style template and the synthesized picture, as shown in table 2 below; the tile correspondence table is used to store an identification of a tile (for uniquely identifying the tile, a file name may be used for convenience), a category identification and a storage location, and a correspondence of the tile to the composite picture, as shown in table 3 below. Table 1: style template record table

Style template mark	Style template storage location

Table 2: synthetic picture corresponding relation table

Composite picture identification	Style template mark	Composite picture storage location

Table 3: graph block corresponding relation table

Composite picture identification	Pattern block identification	Category identification	Tile storage locations

FIG. 1 illustrates a method of generating a puzzle block with obfuscated tiles in accordance with one embodiment of the invention. In summary, the method comprises: step S11, for each original picture, using one of a plurality of different style templates to perform picture style conversion to obtain a corresponding composite picture; step S12, dividing each of the resultant composite pictures into a plurality of tiles having a predetermined size; and step S13, clustering the tiles with the same style based on the content similarity, and regarding all the tiles in the same category as the mutually confused tiles. The method will be described in detail below:

in step S11, for each original picture in the standard original picture library whose size meets the predetermined specification (as shown in fig. 5 (a)), one style template is selected from a plurality of style templates (or style pictures, template pictures, etc.) (as shown in fig. 5 (b)), and a convolutional neural network-based picture style conversion (or picture style migration) algorithm is applied to the original picture to obtain a synthesized picture corresponding to the original picture and having a style consistent with the style of the utilized style template, as shown in fig. 5 (c). Wherein the original pictures whose sizes meet the predetermined specification are stored in a standard original picture library as shown in fig. 4, and the style templates are stored in a style template library as shown in fig. 4. For the original pictures described above, the characteristics include style and content, as is well known to those skilled in the art. The style of a picture is different from the content of the picture, which refers to what is reflected in the picture, and the style of a picture generally refers to a representative unique appearance that the picture as a whole presents, such as a painting style, a sketch style, a cartoon style, and the like. With the development of the deep learning technology, the convolutional neural network can extract the high-level features of the picture on one hand to separate the style and the content of the picture, and can use the extracted style expression to render the content of other pictures on the other hand. Therefore, the application of the image style conversion algorithm based on the convolutional neural network to the original image by using the style template refers to rendering the content of the original image by using the style expression of the style template through the convolutional neural network, so that the style in the rendered original image is consistent with the style extracted from the style template.

After the original picture is subjected to picture style conversion by using the style template to obtain a composite picture, the composite picture is further stored in a composite picture library, and the correspondence between the utilized style template and the composite picture is further stored in a composite picture correspondence table as shown in table 2 so as to be used in the subsequent steps.

It should be noted that the size of the original picture needs to meet a predetermined specification, i.e. the corresponding synthesized picture obtained from the original picture (i.e. obtained by picture style conversion) can just be divided into a plurality of tiles with predetermined sizes, for example, assuming that the predetermined size of the tiles is 2cm × 2cm, if the size of the original picture is 8cm × 6cm, just 12 tiles can be divided from the corresponding synthesized picture, if the size of the original picture is 8cm × 8cm, just 16 tiles can be divided, however, if the size of the original picture is 8cm × 7cm, then tiles with sizes of all 2cm × 2cm cannot be divided, in which case, the original picture needs to be firstly cropped, for example, a picture with a size of 8cm × 6cm, so that just 12 tiles can be divided.

In step S12, each composite picture is divided into a plurality of tiles of a predetermined size, for example, 8cm × 6cm composite picture is divided into 12 tiles of 2cm × 2cm, thereby obtaining puzzle tiles suitable for use in an electronic device (the confusing tiles for each tile are described below with respect to how they are determined). As described above, the style template is consistent with the style of the composite picture obtained therewith, and therefore, all composite pictures obtained using the same style template are herein considered to be consistent in style.

After the composite picture is divided into corresponding picture blocks, the picture blocks can be stored in a picture block library of the composite picture, and the corresponding relationship between the composite picture and the corresponding picture blocks can be recorded in a picture block corresponding relationship table so as to be used in the subsequent steps. For example, the correspondence between the synthesized picture and the divided tiles may be recorded by using a tile correspondence table as shown in table 3.

In step S13, for tiles with consistent style, they are clustered based on content similarity, and for each tile in all tiles under the same category, the other tiles under the category are regarded as the confusing tiles for the tile. All the image blocks obtained by using the same style template can be found according to the synthetic image corresponding relation table and the image block corresponding relation table, and the image blocks have the same style as defined above (or the image blocks have the same style, that is, all the image blocks obtained from one style template belong to the same style). For each type of style tile, according to one embodiment of the present invention, clustering based on content similarity is performed by using the method shown in fig. 2, which includes:

in step S21, for each tile with this type of style, local feature points are extracted using SIFT (Scale-invariant feature transform) or SURF (Speeded Up Robust Features) operators, and each extracted local feature point is represented by a feature vector of the same dimension.

In step S22, local feature points of each tile having the style are clustered by using a K-means algorithm (K-means clustering), and the obtained center of each cluster corresponds to a visual word, so as to construct a visual dictionary representing the tile. Where each visual word is a numerical vector, the dimension of which may be, for example, 128 dimensions. The dimensionality of the visual dictionary is the number of visual words (e.g., 800), one for each dimension of the visual dictionary. In the visual dictionary representing a certain tile, the numerical value of each dimension may be the number of occurrences of the corresponding visual word in the tile, or a numerical value obtained by using a TF-IDF (Term Frequency-Inverse Document Frequency) method.

In step S23, all the tiles characterized by the visual dictionary and having the style are clustered by using a k-means algorithm to obtain a category to which each tile belongs, and the category to which each tile belongs is recorded. As described above, the category to which each tile belongs may be recorded into the tile correspondence table (i.e., table 3 above).

It should be noted that, in step S13, steps S21-S23 may be executed in parallel for the tiles in each type of style, and if the category of the tiles in each type of style is identified from the number 1, the category identifications of the tiles not having the same type (i.e., the tiles obtained by using different style templates) in the tile correspondence table (table 3 above) may conflict with each other, and then the synthesized picture correspondence table (table 2 above) may be required to be matched with table 3 to determine the tile confusion of the tiles.

After the categories of tiles are obtained, the tiles of the same category are treated as confusing tiles for each other (i.e., the confusing tiles of the tiles are determined, thereby generating the puzzle tiles with the confusing tiles) because the tiles have the same size, consistent style, and similar content, thereby making them more difficult for the human eye to distinguish. When the picture blocks for jigsaw are submitted to the user, other picture blocks in the same category can be provided to the user as confusion items, so that the difficulty of jigsaw is improved. It should be noted that after clustering tiles, there may be only one tile in some categories, and these tiles have no confusing terms for confusion.

While one embodiment of a method of generating puzzle blocks with obfuscated tiles, as shown in fig. 1, has been described above, according to another embodiment of the method, a preprocessing step may be included before step S11. That is, the original pictures in the non-standard original picture library whose sizes do not meet the predetermined specification are processed so that the lengths and widths thereof meet the predetermined specification (i.e., so that the corresponding composite picture of the original pictures can be just divided into a plurality of tiles having a predetermined size), and the processed original pictures whose sizes meet the predetermined specification are stored in the standard original picture library as shown in fig. 4.

Furthermore, those skilled in the art will also appreciate that although some specific algorithms or methods are used above to describe the method of generating puzzle tiles with confusing tiles, these algorithms or methods are not limited herein, e.g., histogram matching may be used to calculate similarity of tile content, density-based clustering may be used to classify tiles, etc.

FIG. 3 illustrates a method of providing a user with a puzzle tile containing confusing tiles in accordance with one embodiment of the present invention, including the steps of:

in step S31, the user is provided with the composite pictures available for the puzzle to be selected by the user. As described above, the composite picture is a composite picture obtained by performing picture-style conversion on the original picture when the puzzle piece having the confusing pieces is generated by the method described above, and these composite pictures can be found from the composite picture library.

In step S32, according to the composite picture selected by the user, all tiles segmented from the composite picture are found as a first group of tiles from the puzzle tiles having the confusing tiles generated according to the method described above. For example, all the tiles segmented from the composite picture can be found from the tile correspondence table according to the composite picture identifier submitted by the user, and the category to which each tile belongs can be found.

In step S33, the obfuscated tile of each tile in the first set of tiles is found as a second set of tiles from the generated puzzle tiles with obfuscated tiles. Specifically, the style template identifier corresponding to the synthesized picture can be found from the synthesized picture correspondence table, and all synthesized picture identifiers obtained by using the style template can be found from the synthesized picture correspondence table according to the style template identifier; then, finding out image blocks corresponding to all the synthesized pictures obtained by utilizing the style template in the image block corresponding relation table, and taking the image blocks as a second group of alternative image blocks; and finally, selecting the image blocks with the categories corresponding to the image blocks in the first group of image blocks from the second group of candidate image blocks as the second group of image blocks according to the image block corresponding relation table.

In step S34, the first and second sets of tiles are provided to the user together.

The above-described methods may be implemented by hardware, software, firmware, middleware, pseudocode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or pseudo code, the program code or code segments to perform the tasks may be stored in a computer readable medium such as a storage medium, and a processor may perform the tasks.

It should be appreciated that the software-implemented exemplary embodiment is typically encoded on some form of program storage medium or implemented over some type of transmission medium. The program storage medium may be any non-transitory storage medium such as a magnetic disk (e.g., a floppy disk or a hard drive) or an optical disk (e.g., a compact disk read only memory or "CD ROM"), and may be read only or random access. Similarly, the transmission medium may be twisted wire pairs, coaxial cable, optical fiber, or some other suitable transmission medium known to the art.

Although the present invention has been described by way of preferred embodiments, the present invention is not limited to the embodiments described herein, and various changes and modifications may be made without departing from the scope of the present invention.

Claims

1. A method of generating a puzzle block having confusing tiles, comprising:

2. The method of claim 1, in step 3), clustering tiles having a homogeneous style based on content similarity comprises:

extracting local feature points of each image block with the same type style, clustering the extracted local feature points, and enabling the center of each cluster to correspond to a visual word so as to construct a visual dictionary representing the image block;

and clustering all the image blocks which are characterized by the visual dictionary and have the same type to obtain the category to which the image blocks belong.

3. The method according to claim 1 or 2, wherein in step 1), the picture style conversion of the original picture using the style template comprises:

and (3) converting the original picture into a synthetic picture with the same style as the style template by using a picture style conversion algorithm based on a convolutional neural network.

4. The method according to claim 1 or 2, wherein step 1) is preceded by:

the original picture is pre-processed to conform its length and width to predetermined dimensions.

5. The method of claim 1 or 2, further comprising:

recording the corresponding relation between each composite picture and the picture blocks divided from the composite picture; and

the category to which each tile belongs is recorded.

6. A computer device comprising a processor and a memory, wherein the memory stores instructions executable by the processor, the instructions when executed by the processor causing the computer device to implement the method of any one of claims 1-5.

7. A method of providing a user with a puzzle tile containing confusing tiles, comprising:

providing a plurality of composite pictures to a user for selection by the user, the plurality of composite pictures being a plurality of composite pictures resulting from generating a puzzle picture block having confusing tiles using the method of any one of claims 1-5;

8. A computer device comprising a processor and a memory, wherein the memory stores instructions executable by the processor, the instructions when executed by the processor causing the computer device to implement the method of claim 7.