KR20200122179A - apparatus and method for generating interior designs based on images and text - Google Patents

Abstract

The present invention provides a design generation device, which comprises: a parsing unit parsing image data and text data; a learning unit learning a design generation model by using artificial intelligence based on the parsed image data and text data; and a design generation unit generating a new game content design or a design element for a specific item by using the design generation model learned by the learning unit.

Description

Apparatus and method for generating interior designs based on images and text}

The present invention relates to an image and text-based interior design extraction and design generation apparatus and method.

The pace of change in design fashion is very fast. In particular, as a global spa brand emerged and social media became more active, the pace of change became faster, and trendy items tended to converge to the same design worldwide.

Accordingly, in order to keep up with the rapid pace of change in fashion and supply according to global design, designers need to collect repetitive data and assist in design work, and Fashion MD needs to collect and analyze the latest trends and emerging brand information separately. In addition, designers must investigate rapidly changing fashion trends and design accordingly. However, if you are in a hurry, you may have to make three designs within 7 days, but in fact, you have to draw 200-300 to complete a new design.

Also, in the case of fashion MD, it is expensive to update fashion trends and create new designs, such as browsing collections to research trends and brands, and making order lists and lookbooks to major fashion cities such as New York/Paris. It took time and was inefficient.

In addition, although tools such as computers are currently used to create new designs and create screens, human manual work is inevitably required. For example, in developing a game, the content of the game, such as a mobile game or a console game, is also important, but the graphics are just as important.

This is because the more sophisticated and clear the graphics, the higher the satisfaction of game users. However, in the past, people had to work on graphics one by one, so it took a long time and cost a lot of labor.

Likewise, virtual graphics technology is used a lot in movies these days. In addition to science fiction movies where virtual graphic technology is basically used, special effects such as backgrounds and buildings are often used in general movies. In the case of graphics required for such movies, conventionally, tools such as computer programs are used, but such tools are only a function that makes human manual work a little more convenient, and in the end, people have to work and create them.

As an embodiment of the present invention, the present invention includes a parsing unit for parsing image data and text data; A learning unit for learning a design generation model using artificial intelligence based on the parsed image data and text data; And a design generation unit that generates a new interior design or design element of a specific item by using the design generation model learned by the learning unit.

According to various embodiments of the present disclosure, a new design may be created in a more efficient time/cost manner according to the rapidly changing fashion of the design.

1 is an overall configuration diagram according to an embodiment of the present invention.
2A is a block diagram of an apparatus for generating a design according to an embodiment of the present invention.
2B to 2J are diagrams schematically illustrating a design generation process of the design generation apparatus according to an embodiment of the present invention.
3 is a configuration diagram of an extraction unit according to an embodiment of the present invention.
4 is a block diagram of a design generator according to an embodiment of the present invention.
5 is a flowchart of a method of generating a design by learning a design generation model according to an embodiment of the present invention.
6 is a simplified flowchart of a design change method according to an embodiment of the present invention.
7A is a simplified flowchart of a design synthesis method according to an embodiment of the present invention.
7B is a simplified flowchart of design element creation according to an embodiment of the present invention.
8 is an example of using a design generation method according to an embodiment of the present invention.
9 is an example of using a design synthesis method according to an embodiment of the present invention.
10 is a diagram illustrating a configuration of hardware in which a program for executing a method according to an embodiment of the present invention is stored.

Hereinafter, the present disclosure will be described in detail with reference to the drawings. In this case, the same components in each drawing are indicated by the same reference numerals as possible. In addition, detailed descriptions of functions and/or configurations already known are omitted. In the following, a portion necessary for understanding an operation according to various embodiments will be mainly described, and descriptions of elements that may obscure the subject matter of the description will be omitted.

In this document, expressions such as "A or B", "at least one of A or/and B", or "one or more of A or/and B" may include all possible combinations of items listed together. . For example, “A or B”, “at least one of A and B”, or “at least one of A or B” includes (1) at least one A, (2) at least one B, Or (3) it may refer to all cases including both at least one A and at least one B.

Expressions such as "first", "second", "first", or "second" used in various embodiments may modify various elements regardless of their order and/or importance, and the corresponding elements Not limited. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may be renamed to a first component.

In addition, some elements in the drawings may be exaggerated, omitted, or schematically illustrated. The size of each component does not fully reflect the actual size, and therefore, the contents described herein are not limited by the relative size or spacing of the components drawn in each drawing.

1 is an overall configuration diagram according to an embodiment of the present invention.

The design generating device 200 parses an image exposed in real time on the Internet, such as a website, an SNS, a blog, and text data related to the image. Images can include any image that can be extracted on the Internet. In addition, the image may include each image used when playing as a video. Specifically, for example, the image is an image that includes all items related to fashion, an image that includes all items related to accessories, and images related to game content (the background of the game, people, clothes, items, buildings, maps, etc. Images), images related to movie content (all images necessary for movies such as movie background, people, special effects, buildings, maps, etc.), brand logo images, images related to interiors (furniture images, furniture arrangement, wallpaper, fabric, etc.) ), various tableware images, etc. may be included. Also, the image includes all visually identifiable images, such as landscape photos and drawings. In addition, the image may include an image of an animal such as a bird or a dog.

Also, when parsing an image in a website such as a shopping mall, the design generating device 200 may also parse the sales amount and sales amount data corresponding to the image. Further, the text related to the image may be, for example, text such as'T-shirt','T', or'Top' that is described in the category or product name of the image when a t-shirt is sold in a shopping mall. In addition, text data may include information on product specifications of the product, such as material and material. In addition, the product names included in the text data may include accessories such as'earrings' and'necklaces', interior terms such as'wallpaper','fabrics', and'fabrics', and furniture terms.

Also, the text data may include content design terms. The content may include all terms related to games, movies, and cartoons. Also, the text data may include all terms related to an image to be parsed.

In addition, the design generating device 200 may parse and receive image data uploaded to an SNS such as Instagram or Facebook for a certain period in real time. You can know the latest trends through images uploaded to social media such as Instagram or Facebook. In addition, the design generating device 200 may parse game or movie information in real time on the Internet in order to grasp the preference of a specific game or a specific movie. For example, the design generating device 200 may parse and receive the number of sales, the number of audiences, or the number of uploads on the Internet for a certain period from the present on the Internet.

In addition, image data and text data stored in the design generating device 200 may be parsed.

The design generation device 200, which is exposed on the Internet such as a website, SNS, blog, etc. in real time, or that parses the image in the design generation device 200 and text data related to the image, is a specific device based on the parsed image and text. Create a new design or design element for the item. Here, the design element may include at least one of all elements that can constitute a design, such as a pattern, shape, and color. Design elements may be elements constituting, for example, clothes, shoes, and the like, and design elements may be elements constituting, for example, interior related items (all items related to interiors such as furniture, wallpaper, and tiles). In addition, the design element may be an element constituting game content (a game background, a person, clothes, items, buildings, maps, etc.). Also, the design element may be an element constituting movie content (eg, a background of a movie, a person, a special effect, a building, a map, etc.). Also, the design element may be an element constituting a brand logo or the like. In addition, design elements can constitute all elements that can be created as photographic images. For example, a design element that can compose a landscape image, an animal image, a person image, etc. may be exemplified.

Specifically, the design generating device 200 extracts an item using the parsed image and text data. In extracting an item, in addition to the parsed image and text data, an item extraction model (objection detection) is used. For example, items can include various items such as shoes, bags, dresses, ties, shirts, hats, cardigans, coats, etc., all accessories, all interior related items, and movie content components (movie background, characters, special effects , Buildings, maps, etc.), game content components (game background, characters, character costumes, game items, buildings, maps, etc.), brand logos, and the like. E.g,

If there is a vector value corresponding to the shoe in the image, the shoe item can be extracted by recognizing the'shoes' part of the image.

In addition, text data related to the parsed image may also be used to extract items from the image. Also, a plurality of items may be extracted from one image. For example, if a man wearing a beanie, a turtleneck sweater and pea coat, jeans and sneakers is standing in one image, the items extracted from the image are'beanie','turtleneck sweater','peacoat', ' It could be jeans' and'sneakers'. Alternatively, for example, items such as “sofa”, “rug”, and “stand lighting” may be extracted from one living room image. Alternatively, for example, in the case of a game image, images such as a background of a game, a character, a character costume, an item, and a map may be extracted.

In addition, the design generating device 200 extracts design elements for each of the extracted items. As described above, the design elements may be elements included in all images, and the following will be described in more detail based on clothes. Although the costume is described as an example, it can be applied in various ways such as furniture, game content components, and movie content components. The elements of design include all the external elements that make up the item, such as color, sleeve length, fit, pattern, and print. In addition, elements of design are not limited to specific items, and include elements within the image itself or the image itself included in the image. For example, when the image is a landscape picture, the element of the design may be a color included in the landscape, a pattern, or a shape included in the image, for example, a shape such as a flower or a tree.

For example, referring to FIG. 2B, design elements of a sleeveless and knee-length white flared dress with a flower-shaped print as a whole include elements such as floral print, sleeveless, white, knee-length, and flare dress. . In addition, the design generating device 200 uses not only elements of design for each item, but also elements such as overall mood, shape, and color in the image from which the item is not extracted from the collected images. You can also extract patterns, colors, and shapes. In addition, although not shown, for example, when a chair is included in the image, the design elements of the chair include the design of the shape of the chair, that is, the shape of the legs of the chair, the shape of the back, and the shape of the seating part. The material of the seat can also be included as a design element.

The design generation device 200 may extract a higher design or design element from among the extracted design or design elements. Specifically, a higher design or design element may be determined based on a case in which the frequency of exposure of a specific design or design element on an image exposed to an SNS or a website is greater than or equal to a reference value. In addition, a higher design or design element may be determined based on a case in which the sales amount and sales amount of an item including a specific design or design element in a specific website is greater than or equal to a reference value. In addition, high-level design based on the preference for the image displayed on the Internet, such as a specific website or SNS (determined based on all data that can estimate the preference for the image, such as the number of clicks, feedback, and shares) Or you can judge design elements. In addition, for the design element itself, design elements above a certain reference value may be extracted by considering all of the exposure frequency, sales volume, and preference. For example, referring to FIG. 2B, if design elements in the extracted designs are “white”, “sleeveless”, “flower pattern” and “flare dress”, each element may be a higher design element. In addition, when determining whether a specific design or design element is an upper design or a higher design element, it is determined whether it is a design or design element included in an item worn by a celebrity such as a celebrity. Can be judged. In addition, for example, when the upper design is furniture, the design elements may include all elements that form the furniture, such as “wood”, “leather”, “with or without armrests”. In this way, the upper design may be any design (game content, movie content, brand logo, etc.) that may be included in the image, and in this case, the same upper design extraction process may be applied.

The design generating device 200 extracts an element of the upper design or the upper design and stores it in the storage unit. The design generating device 200 extracts and updates elements of a higher design in real time according to a rapid change in design, and categorizes and stores the higher design elements for each number of items. Also, when there are no items (general images, etc.), only design elements can be saved.

In addition, the design generating device 200 includes image data and text data parsed using artificial intelligence, design elements/design elements for the extracted specific items, design elements for specific items extracted from the design extraction unit. Design generation models can be created and trained by reflecting the upper-level design elements. In addition, the design generation device 200 learns a design generation model by additionally considering a generated design or design element for a specific item, and also evaluates the generated design or design element for a specific item. The design generation model is trained by additionally considering the result. Evaluation of the design or design element will be described later. In addition, the design generation device 200 may learn a design generation model by considering design elements of an image without a specific item, and evaluate the generated design and design elements, and additionally consider the result of the design generation model. To learn.

Thereafter, the design generation device 200 generates a new design (upper design or design element) using a design generation model that is learned in real time by the learning unit. That is, quality judgment is made such as extracting a specific generated design and design element as a higher design based on the above-described judgment criteria among the generated designs or design elements, and a higher design or design element, and a higher design or design element generation design generation model The design generation model is trained by using etc.

Generative adversarial networks (GANs) that allow category-specific and random generation to create new designs, and VAE (Variational Autoencoder)+ that create new designs in a form that allows random transformation of specific designs and high similarity in characteristics of specific designs. GAN, a genetic algorithm + GAN, design that recognizes specific design elements as human genetic traits (look, etc.) to create various crosses, and based on feedback on this, a new transformation with higher marketability can be repeatedly transferred between generations A style transfer technology that transforms the style while maintaining the appearance of the existing design by extracting conditional GAN for change and inspiration for new design from data such as various images may be used.

Machine learning is largely divided into three categories: the first is supervised learning, the second is reinforcement learning, and the last is unsupervised learning.

Here, GAN belongs to unsupervised learning, and the Generator (G) that generates an image and the Discriminator (D) that evaluates the image generated by the Generator face each other to create a new image as a model that improves each other's performance. do. Specifically, D makes an effort to judge only the original data as true, and G creates fake data so that D cannot be determined as false, so that the performance of the two models is improved together, so that D finally tries to determine the real data and the fake data. It becomes impossible to distinguish.

As an example of creating a design, there is a method of creating a new design element by changing a vector value of a design element. The change of the vector value may be input as a set value and may be made by the learned design generation model. Specifically, for example, referring to FIG. 2D, if the floral dress is fashionable, the vector values of the existing floral patterns in the floral dress extracted from the collected image are changed to create a new floral dress design. can do. In addition, for example, for a design for a background screen of a game, a vector value may be changed and a design extracted from another image may be changed and applied. In addition, for example, as the background screen of the game, a design extracted from the background screen of the previous game or a design extracted from an image such as a landscape picture is applied according to the vector value changed by the user, thereby creating a new game wallpaper. I can.

In addition, as another embodiment of creating a new design element, there is a method of creating a new design for a specific item by synthesizing different elements among design elements. Specifically, a new design can be created by merging different elements among the elements of each trendy design. For example, referring to FIG. 2E, a design of an ultra violet index long one-piece design with puff sleeves can be created by synthesizing “puff sleeve” of knit, “ultra violet color” and “long dress” of a shirt among design elements. have. On the other hand, in this case, each design element is not simply synthesized as it is, but elements are also changed and newly created by the design generation model to create a design. In addition, these different elements may be automatically and randomly merged, or may be merged according to input settings (for the design elements to be merged). Different design elements are applicable even when items are different. For example, a pattern of clothes may be applied to furniture, and a shape of an accessory may be applied to an item of a game.

In addition, as another embodiment of creating a new design element, there is a method of generating a new design element by randomly changing and combining design elements.

For example, referring to FIG. 2F, a pattern for a flower is extracted from a flower photo, and a sunset color and mood are extracted from the photo at sunset, so that a sunset index pattern containing the flower pattern may be generated. For the sake of understanding, the design elements that are merged are described as two, but in reality dozens of thousands of elements can be randomly generated and merged.

In addition, the design or design element created in this way is reflected when the design generation model is trained again, and the design generation model can be trained, and the newly learned design generation model can be used when a new design or design element is created again. .

Subsequently, the design generating device 200 evaluates the generated new designs or design elements according to a certain standard. In this case, the design generating device 200 may rank new designs or design elements extracted in consideration of various factors. In order to rank new designs or design elements, the frequency of exposure of a specific design or design element on images that are generally exposed on social media or websites, sales volume and sales amount of items containing a specific design or design element, a specific website or It can be determined based on the preference for the image exposed on the Internet such as SNS (determined based on all data that can estimate the preference for the image, such as the number of clicks, feedback, and shares). The specific design or design element judged here is a design similar to a new design or design element that has been created, and a judgment on this is also included. In addition, when determining the frequency of exposure or when determining the ranking, weights are applied to designs or design elements exposed on celebrities (e.g., celebrities or social media followers) or celebrity SNS or images. In addition, if the preference, sales volume, or sales amount for a design or design element sold by a specific brand or a specific website among the elements of the same design or design is higher, weight may be given to the design or design element (hereinafter, “ Weight judgment”). The design generating device 200 ranks a new design or new design element created in this way.

Also, in this way, a design generation model can be trained by using information ranked on a new design or a new design element.

2A is a block diagram of a design generating device 200 according to an embodiment of the present invention.

The design generation device 200 includes an input unit 201, a parsing unit 202, an extraction unit 203, a storage unit 204, an upper design extraction unit 206, a learning unit 207, and a design generation unit 208. , A display unit 210, and a transmission unit 212.

Text or label may be received through the input unit 201 and reflected in the change of the vector value. Also, the input unit 201 may receive an output category from a user. For example, the user may set an output category as furniture, interior, game character, clothes, shoes, movie background, etc. through the input unit 201.

The parsing unit 202 parses an image and text data related to the image in real time from an external website, blog, or SNS server or from the storage unit 204 through a network. In addition, sales volume data and sales amount data related to a specific image are parsed from a specific website server or the design generating device itself. Further, the text related to the image may be, for example, text such as'T-shirt','T', or'Top' that is described in the category or product name of the image when a t-shirt is sold in a shopping mall. In addition, the parser 202 may parse game or movie information in real time on the Internet in order to determine the preference of a specific game or a specific movie. For example, the parser 202 may parse and receive the number of sales, the number of audiences, or the number of uploads on the Internet for a certain period from now on the Internet.

The extraction unit 203 extracts an item by using the parsed image and text data. In extracting an item, in addition to the parsed image and text data, an item extraction model (objection detection) is used. For example, items can include various items such as shoes, bags, dresses, ties, shirts, hats, cardigans, coats, furniture, and wallpaper.If there is a vector value learned to be recognized as a shoe in the image, 'It is possible to extract the shoe item by recognizing the part. In addition, items include all accessories, all interior related items, all interior related items, movie content components (movie background, people, special effects, buildings, maps, etc.), game content components (game background, characters, character costumes, etc.) , Game items, buildings, maps, etc.), brand logos, etc. may be included. Also, the item may include a brand logo.

In addition, the extraction unit 203 extracts design elements for each of the extracted items. For example, elements of a design include all the external elements that make up an item, such as color, sleeve length, fit, pattern, and print. Also, for example, elements of design include elements of general design, colors, shapes, and textures constituting the design. Also, the extraction unit 203 may extract a plurality of items from one image. Referring to FIG. 2C, for example, if the user sets the output to fashion, if there is a woman holding a cat in an image and wearing sunglasses, wearing hot pants and a black T-shirt, and wearing a pair of clothes, the extracted item is sunglasses, These are hot pants, T-shirts, and shoes.

In addition, the extraction unit 203 extracts design elements for each of the extracted items. The elements of design include all the external elements that make up the item, such as color, sleeve length, fit, pattern, print, and material. In addition, elements of design include colors, shapes, and textures that make up the design. For example, referring to FIG. 2B, design elements of a sleeveless and knee-length white flared dress with a flower-shaped print as a whole include elements such as floral print, sleeveless, white, knee-length, and flare dress. .

In addition, the extraction unit 203 uses not only design elements for each item, but also elements such as overall mood, shape, and color in the image from which the item is not extracted from the collected images. , Color, shape, etc. can also be extracted.

The storage unit 204 stores an item extraction model. The item extraction model may determine which item corresponds to a vector value in a corresponding image. In addition, the storage unit 204 categorizes and stores the extracted items. In addition, the storage unit 204 stores a design element extraction model. The design element extraction model extracts design elements, that is, the shape, color, and material of the design from the image. In addition, the storage unit 204 categorizes and stores the upper design for each item extracted by the upper design extraction unit 206.

The upper design extraction unit 206 determines which of the extracted designs or design elements is an upper design or design element. Specifically, a higher design or design element may be determined based on a case where the frequency of exposure of a specific design on an image exposed to an SNS or a website is greater than or equal to a reference value. In addition, a higher design or design element may be determined based on a case in which the sales amount and sales amount of an item including a specific design or design element in a specific website is greater than or equal to a reference value. In addition, high-level design based on the preference for the image displayed on the Internet, such as a specific website or SNS (determined based on all data that can estimate the preference for the image, such as the number of clicks, feedback, and shares) Or you can judge design elements. For example, referring to FIG. 2B, if the design elements in the extracted designs are “white”, “sleeveless”, “flower pattern” and “flare one piece”, each element may be a higher design. In addition, by determining whether or not a celebrity such as a celebrity wears when determining a higher design or design element, the design or design element may be determined by giving a predetermined weight. In addition, for example, when weighting a design or design element, in the case of a game, factors such as a sales ranking of a game and a popularity ranking of a corresponding character within the same game may be considered. In addition, for example, in the case of a movie, the number of viewers of the movie, the number of feedback on the SNS for the movie, or the degree of exposure may be considered.

The learning unit 207 selects upper design elements among image data and text data parsed using artificial intelligence, design/design elements for the extracted specific item, and design elements for specific items extracted from the design extraction unit. It can be applied to create and train a design generation model. In addition, the learning unit 207 learns the design generation model by additionally considering the generated design or design element for a specific item, and also evaluates the generated design or design element for a specific item, and the result Consider additionally to train the design generation model. The evaluation is as described above.

The design generation unit 208 generates a new design using a design generation model that is learned in real time by the learning unit. To create a new design in the design generation unit 208, generative adversarial networks (GANs) capable of category-specific and random generation, VAE+GAN, genetic algorithm + GAN, and conditional for design change are possible. GAN, and style transfer techniques can be used.

Specifically, the design generation unit 208 uses the following exemplary method to generate a new design or design element.

As an example of creating a design, there is a method of creating a new design element by changing a vector value of a trendy design element. The change of the vector value may be input as a set value, and may be automatically performed by the learned design generation model. Specifically, for example, referring to FIG. 2D, if the floral dress is fashionable, the vector values of the existing floral patterns in the floral dress extracted from the collected image are changed to create a new floral dress design. can do. However, this is not a concept of simply synthesizing each existing design or design element, but creating a new design by randomly changing and synthesizing each design or design element.

In addition, as another embodiment of creating a new design element, there is a method of creating a new design for a specific item by synthesizing different elements among design elements. Specifically, it is possible to create a new one by merging different elements among the elements of each trendy design. For example, referring to FIG. 2E, a design of an ultra violet color long one-piece with puff sleeves is synthesized by combining “puff sleeves” of knit, “ultra violet color”, and “long dress” of a shirt among trendy design elements. Can be created. In addition, these different elements may be automatically and randomly merged, or may be merged according to input settings (for design elements to be merged).

In addition, as another embodiment of creating a new design element, a new design for a specific item may be created by using different elements among design elements. For example, as shown in FIG. 2G, a new design may be generated. The design of the game character can be created using a design or design element.

In addition, as another embodiment of creating a new design element, a new design for a specific item may be generated using different elements among design elements. For example, as shown in FIG. 2H, The design of the map of the game can be created using design or design elements.

In addition, as another embodiment of creating a new design element, a new design for a specific item may be created using different elements among design elements. For example, as shown in FIG. A background image of a movie may be created using a design or design element.

In addition, as another embodiment of creating a new design element, a new design for a specific item may be generated using different elements among design elements. For example, as shown in FIG. Furniture designs can be created using designs or design elements.

In addition, as another embodiment of creating a new design element, there is a method of generating a new design element using a design generation model based on the design elements.

For example, referring to FIG. 2F, a new pattern may be generated by extracting a pattern for a flower from a flower photo, and extracting a sunset color and mood from the photo at sunset. The generated pattern can be applied as a fabric pattern or wallpaper pattern.

In addition, the design or design element created in this way is reflected when the design generation model is trained again, and the design generation model can be trained, and the newly learned design generation model can be used when a new design or design element is created again. . In addition, these learned design generation models may each include individual models according to types of input values and output values.

Subsequently, the design generation unit 208 evaluates the generated new designs or design elements according to certain criteria. Specifically, the design generation unit 208 ranks the extracted new designs or design elements in consideration of various factors.

In order to rank new designs or design elements, the frequency of exposure of a specific design or design element on an image that is generally exposed on social media or websites, the sales volume and sales amount of an item containing a specific design or design element, a specific website or It can be determined based on the preference for the image exposed on the Internet such as SNS (all data that can estimate the preference for the image, such as the number of clicks, feedback, and shares). In addition, when judging the frequency of exposure or determining the ranking, weights are given to designs or design elements that are exposed on SNS or images of celebrities or celebrities, and among the elements of the top design, designs sold by a specific brand or a specific website are given weight. If the preference, sales volume, or sales value is higher, weights can be given to elements of the design (hereinafter, “weight determination”). In addition, in the case of a game or a movie, the number of users of the game or the number of viewers of the movie may also be used in determining the weight.

The design generating device 200 may rank new designs or design elements thus generated. In addition, it is possible to re-learn the design generation model by using information ranked on a new design or a new design element. That is, the design generation model is learned so that the design can be generated by reflecting the design generation when the design is generated by itself by ranking designs based on the above-described various elements generated by the design generation model.

The display unit 210 may display a design generated by the design generation unit 208.

The transmission unit 212 may transmit the design to be manufactured and the quantity to be manufactured to the factory when manufacturing is to be performed on the generated design.

3 is a configuration diagram of an extraction unit 203 according to an embodiment of the present invention.

The extraction unit 203 includes an item extraction unit 300 and a design extraction unit 302.

The item extracting unit 300 extracts an item using the received image and text data. In extracting an item, in addition to the received image and text data, a stored item extraction model is used. Here, the items include various items such as shoes, bags, dresses, tie shirts, hats, cardigans, and coats, and if there is a vector value corresponding to the shoes in the image, the'shoes' portion of the image is recognized and the shoe item is extracted. I can. In addition, in extracting an item from the received image, it may be used as a reference as text related to the image. In addition, the item extracting unit 300 may extract a plurality of items from one image. For example, referring to FIG. 2C, if there is a woman in the image holding a cat and wearing sunglasses, wearing hot pants and a black T-shirt, and wearing pants, the extracted items are sunglasses, hot pants, T-shirts, and shoes. . Cats are not extracted as items here.

The design extraction unit 302 extracts a design and a design element of the extracted item. In addition, the design extraction unit 302 may extract design elements through parsed images, text, and the like. Design elements include all the external elements that make up the item, such as color, sleeve length, fit, pattern, and print. In addition, design elements may include colors, moods, materials, patterns, etc. in a general image even if they do not constitute an item.

4 is a block diagram of a design generation unit 208 according to an embodiment of the present invention.

The design generation unit 208 includes a design receiving unit 400, a design change unit 402, a design synthesis unit 404 and an evaluation unit 406.

The design generation unit 208 generates a new design using a design generation model that is learned in real time by the learning unit.

To create a new design, category-specific and random generation are possible (generative adversarial networks), VAE+GAN, which allows random transformation of a specific design, genetic algorithm + GAN, conditional GAN for design change, and style transfer technology. Can be used.

Machine learning is largely divided into three categories: the first is supervised learning, the second is reinforcement learning, and the last is unsupervised learning.

Here, GAN belongs to unsupervised learning, and the Generator (G) that generates an image and the Discriminator (D) that evaluates the image generated by the Generator face each other to create a new image as a model that improves each other's performance. do. Specifically, D makes an effort to judge only the original data as true, and G creates fake data so that D cannot be determined as false, so that the performance of the two models is improved together, so that D finally tries to determine the real data and the fake data. It becomes impossible to distinguish.

In addition, cGAN is different from conventional GAN in that it can generate desired things rather than randomly.

The design receiving unit 400 receives all elements to generate a new design. For example, a design generation model and design elements may be received.

The design change unit 402 generates a new design by changing the vector value of the design element using the element of the design extracted for each item. Specifically, if there is a floral shirt, a new floral shirt design can be generated by changing the vector values of the floral patterns in the floral shirt extracted from the collected image. In addition, the design change unit 402 may generate a new design element by changing vector values for the design element extracted from the item and the design element extracted from the image without the item using the design generation model. . The vector value change may be performed automatically or may be performed by receiving a set value.

Using the design elements extracted from the collected items, the design synthesis unit 404 generates a new design of the item by synthesizing different elements among the elements of the collected fashion design for each item. However, this is not a simple synthesis of designs, but a new design by randomly using various designs or design elements. In addition, the design change unit and the design synthesis unit may be mixed to create a new design. In addition, the design change unit and the design synthesis unit continuously receive feedback from the evaluation unit 406 and generate a design using the feedback.

Specifically, referring to FIG. 2E, when the output setting is a one-piece, the “puff sleeve” of the knit, the “ultra violet color”, and the “long one-piece” of the shirt are synthesized among fashionable design elements to have a puff sleeve and an ultra violet. An index long one-piece design can be created. It is also possible to combine different elements of the same item. For example, a blouse design can be created by merging the “cuff sleeves” of a blouse, the “flower-shaped collar” of the blouse, and the “mint color” of the blouse, which are elements of fashionable design.

Also, the design synthesis unit 404 may generate a new design element by synthesizing elements of a design extracted from an item and elements of a design extracted from an image that does not contain an item.

For example, referring to FIG. 2I, a new movie background image may be generated based on a design element of a general landscape photograph and a design element of a landmark image.

In addition, the design synthesis unit 404 may generate a new design for a specific item by synthesizing elements of a design extracted from an item and elements of a design extracted from an image that does not contain an item.

The evaluation unit 406 evaluates a design or design element newly created by the design change unit 402 or the design synthesis unit 404 on a predetermined basis. Specifically, the evaluation unit 406 may rank new designs or design elements extracted in consideration of various factors.

In order to rank new designs or design elements, the frequency of exposure of a specific design or design element on an image that is generally exposed on social media or websites, the sales volume and sales amount of an item containing a specific design or design element, a specific website or All of the preferences (clicks, feedbacks, shares, etc.) for the image (including design or design elements similar to new design or design elements) exposed on the Internet such as SNS, etc. Judgment based on data) can be determined. In addition, when judging the frequency of exposure or determining the ranking, weights are given to designs or design elements that are exposed on SNS or images of celebrities or celebrities, and among the elements of the top design, designs sold by a specific brand or a specific website are given weight. If the preference, sales volume, or sales value is higher, weights can be given to elements of the design (hereinafter, “weight determination”). In addition, for example, when weighting a design or design element, in the case of a game, factors such as a sales ranking of a game and a popularity ranking of a corresponding character within the same game may be considered. In addition, for example, in the case of a movie, the number of viewers of the movie, the number of feedback on the SNS for the movie, or the degree of exposure may be considered. The evaluation unit 406 may rank the new designs thus generated.

For example, if the created design is a short cardigan in burgundy color, various burgundy colors can be expressed for each product. That is, the same burgundy color, but slightly different tones of burgundy color may be extracted. Accordingly, in various types of burgundy short cardigans, in order to select a burgundy color, a more preferred color may be extracted even with the same burgundy color to derive a short cardigan of the final burgundy color. Here, in order to extract a color that is the same burgundy color but more preferred, it is based on factors that can determine the preference between colors, such as whether it is a color used by a specific brand with high preference or a color used in an image with high exposure. It can be judged as. This can be applied to furniture design, game content components, movie content components, logos, etc.

If the evaluation unit 406 ranks a specific design in this way, it is determined that the evaluation of the designs with the highest ranking among the ranked designs is good, and the design generation model is learned using the design generation model that generated the design. do. The design is created again using the learned design generation model. This process is repeated continuously, and the quality of creation of the design is raised.

5 is a flowchart of a method for extracting a fashion design according to an embodiment of the present invention.

Images exposed in real time from websites, SNS, and blogs received through the parsing unit 202 and texts and texts related to the images are parsed (S500). Also, the parsing unit 202 parses an image, text related to the image, and text stored in the design generating device.

The item extracting unit 300 extracts an item based on the collected image and text (S502).

Specifically, the item extracting unit 300 extracts an item using the parsed image and text data. In extracting an item, in addition to the parsed image and text data, a stored item extraction model is used. Here, the items include various items such as, for example, shoes, bags, dresses, tie shirts, hats, cardigans, coats, and if there is a vector value recognized as a shirt in the image, the'shirt' portion of the image is recognized and the shirt Items can be extracted. In addition, in extracting an item from the received image, it may be used as a reference as text related to the image. In addition, the item extracting unit 300 may extract a plurality of items from one image.

When an item is extracted from the item extracting unit 300, the design extracting unit 302 extracts an element of a design from the item (S504). In extracting an item, in addition to the parsed image and text data, an item extraction model (objection detection) is used. For example, items can include various items such as shoes, bags, dresses, ties, shirts, hats, cardigans, and coats. If there is a vector value corresponding to a shoe in the image, the'shoes' portion of the image is recognized and the shoe Items can be extracted. In addition, items include all accessories, all interior related items, all interior related items, movie content components (movie background, people, special effects, buildings, maps, etc.), game content components (game background, characters, character costumes, etc.) , Game items, buildings, maps, etc.), brand logos, etc. may be included.

Specifically, the design extraction unit 302 extracts a design element of the extracted item (S504). Design elements include all the external elements that make up the item, such as color, sleeve length, fit, pattern, and print. In addition, the design element may include a material, mood, and the like. In addition, the design extraction unit 302 uses not only elements of the design for each item, but also elements such as overall mood, shape, and color in the image from which the item is not extracted among the collected images. You can also extract patterns, colors, and shapes.

Subsequently, a design or design element extracted from the upper design extraction unit 206 and the design extraction unit 302 is determined to be a higher design (S506).

Specifically, a trendy design or design element may be determined based on a case in which the frequency of exposure of a specific design or design element on an image exposed to an SNS or a website is greater than or equal to a reference value. In addition, a higher design or design element may be determined based on a case in which the sales amount and sales amount of an item including a specific design or design element in a specific website is greater than or equal to a reference value. In addition, high-level design based on the preference for the image displayed on the Internet, such as a specific website or SNS (determined based on all data that can estimate the preference for the image, such as the number of clicks, feedback, and shares) Or you can judge design elements. In addition, when determining whether a specific design or design element is an upper design or design element, it is determined whether or not a celebrity such as a celebrity wears it, and the design or design element may be determined by giving more weight.

Subsequently, a design generation model for generating a new design or design element is trained using artificial intelligence (S508).

The learning unit 207 includes image data and text data parsed using artificial intelligence, design elements/design elements for the extracted specific item, and design elements for specific items extracted from the design extraction unit. A design generation model can be generated and trained by reflecting the upper design element and the design generation model that generated the upper design/design element. In addition, the learning unit 207 learns the design generation model by additionally considering the generated design or design element for a specific item, and also evaluates the result of evaluating the generated design for a specific item or design element. Consider additionally and train the design generation model. In addition, the learning unit 207 generates a design or design element that is not limited to a specific item, a design element/design element, an upper design element among design elements extracted from the design extraction unit, and a higher design/design element. A design generation model can be created and trained by reflecting a design generation model.

Subsequently, a new design is generated using the learned design generation model (S510).

Specifically, the design generation unit 208 uses the following exemplary method to generate a new design or design element.

As an example of creating a design, there is a method of creating a new design element by changing a vector value of a trendy design element. The change of the vector value may be input by the user as a set value, and may be made by the learned design generation model. Specifically, for example, referring to FIG. 2D, if the floral dress is fashionable, the vector values of the existing floral patterns in the floral dress extracted from the collected image are changed to create a new floral dress design. can do.

In addition, as another embodiment of creating a new design element, there is a method of creating a new design for a specific item by synthesizing different elements among design elements. Specifically, it is possible to create a new one by merging different elements among the elements of each trendy design. For example, referring to FIG. 2E, a design of an ultra violet color long one-piece with puff sleeves is synthesized by combining “puff sleeves” of knit, “ultra violet color”, and “long dress” of a shirt among trendy design elements. Can be created. In addition, these different elements may be automatically and randomly merged, or may be merged according to input settings (for design elements to be merged).

In addition, as another embodiment of creating a new design element, there is a method of generating a new design element by randomly changing and combining design elements.

For example, referring to FIG. 2F, a pattern for a flower is extracted from a flower photo, and a sunset color and mood are extracted from the photo at sunset, and a flower pattern is included to generate a sunset index pattern.

In addition, the design or design element created in this way is reflected when the design generation model is trained again, and the design generation model can be trained, and the newly learned design generation model can be used when a new design or design element is created again. .

Although not shown, the design generation unit generates a design and then evaluates and ranks the generated design. Specifically, the design generation unit 208 ranks the extracted new designs or design elements in consideration of various factors.

In order to rank new designs or design elements, in general, the frequency of exposure of a specific design on images exposed on social media or websites, sales volume and sales amount of items containing a specific design, and exposure on the Internet such as a specific website or SNS. It can be determined based on the preference for the image (all data that can estimate the preference for the image, such as the number of clicks, the number of feedback, and the number of shares). In addition, when judging the frequency of exposure or determining the ranking, weights are given to designs or design elements that are exposed on SNS or images of celebrities or celebrities, and among the elements of the top design, designs sold by a specific brand or a specific website are given weight. If the preference, sales volume, or sales value is higher, weights can be given to elements of the design (hereinafter, “weight determination”). In addition, in the case of a game or a movie, the number of users of the game or the number of viewers of the movie may also be used in determining the weight. The design generating device 200 may rank new designs or design elements thus generated. Also, in this way, a design generation model can be trained by using information on which a new design or a new design element is ranked.

6 is a flowchart of a design change method according to an embodiment of the present invention.

Subsequently, the design change unit 402 generates a new design or design element by changing the vector value of the design element using the learned design generation model (S602). The vector value may be changed according to the set value received through the input unit or may be automatically changed.

In addition, the design or design element created in this way is reflected when the design generation model is trained again, and the design generation model can be trained, and the newly learned design generation model can be used when a new design or design element is created again. .

The generated design or design element is evaluated and ranked (S604).

In order to rank new designs or design elements, in general, the frequency of exposure of a specific design on images exposed on social media or websites, sales volume and sales of items containing a specific design or design element, and the Internet such as a specific website or SNS. It can be determined based on the preference for the image exposed on the image (all data that can estimate the preference for the image, such as the number of clicks, the number of feedbacks, and the number of shares). In addition, when judging the frequency of exposure or determining the ranking, weights are given to designs or design elements that are exposed on SNS or images of celebrities or celebrities, and among the elements of the top design, designs sold by a specific brand or a specific website are given weight. If the preference, sales volume, or sales value is higher, weights can be given to elements of the design (hereinafter, “weight determination”). In addition, in the case of a game or a movie, the number of users of the game or the number of viewers of the movie may also be used in determining the weight. The design generating device 200 may rank new designs or design elements thus generated. Also, in this way, a design generation model can be trained by using information ranked on a new design or a new design element.

7A is a flowchart of a design generation method according to an embodiment of the present invention.

Using the design generation model, the design synthesis unit 404 generates a new design by using different elements among the elements of the collected fashion design for each item (S702).

The created new design may be limited to a design for a specific item. However, the items from which different elements are extracted may be different items.

The generated design may be reflected when the design generation model is trained again, and the design generation model may be trained, and the newly learned design generation model may be used when a new design or design element is generated again.

The generated design is evaluated and ranked (S704).

In order to rank new designs, in general, the frequency of exposure of a specific design on images exposed on social media or websites, sales volume and sales amount of items containing a specific design, and corresponding images exposed on the Internet, such as a specific website or SNS. It can be determined based on the preference (all data that can estimate the preference for the image, such as the number of clicks, the number of feedbacks, and the number of shares). In addition, when determining the frequency of exposure or when determining the ranking, weights are given to designs exposed on SNS or images of celebrities or celebrities, and among the elements of the design included in the generated design, sales by a specific brand or a specific website If the preference, sales volume, or sales amount for a design element is higher, a weight can be given to the element of the design (hereinafter, “weight determination”). In addition, in the case of a game or a movie, the number of users of the game or the number of viewers of the movie may also be used in determining the weight. The design generating device 200 may rank new designs thus generated. Also, in this way, a design generation model can be trained using the information ranked on the new design.

7B is a flowchart of a design generation method according to an embodiment of the present invention

A new design element is generated using the design generation model (S706).

A new design element is generated using not only the design element for a specific item, but also the design element extracted based on the image from which the item is not extracted (S706). New design elements can include patterns, colors, and fit.

The generated design elements are evaluated and ranked (S708).

In order to rank new design elements, the frequency of exposure of a specific design on images that are generally exposed on social media or websites, sales volume and sales amount of items containing specific design elements, and corresponding images exposed on the Internet, such as a specific website or SNS. It can be determined based on the preference for (all data that can estimate the preference for the image, such as the number of clicks, the number of feedbacks, and the number of shares). In addition, when determining the frequency of exposure or when determining the ranking, weights are given to design elements exposed on SNS or images of celebrities or celebrities. If the preference, sales volume, or sales amount is higher, weights can be given to elements of the design (hereinafter, “weight determination”). In addition, for example, when weighting a design or design element, in the case of a game, factors such as a sales ranking of a game and a popularity ranking of a corresponding character within the same game may be considered. In addition, for example, in the case of a movie, the number of viewers of the movie, the number of feedback on the SNS for the movie, or the degree of exposure may be considered. The design generating device 200 may rank new designs or design elements thus generated. Also, in this way, a design generation model can be trained by using information ranked on a new design or a new design element.

Referring to FIG. 8, a new design created through the design change unit 402 is shown. If a shirt with various prints is in fashion, a shirt with a new print is designed by changing the existing print elements.

Referring to FIG. 9, an example in which a new design is created by synthesizing different design elements is shown.

10 is a diagram illustrating a configuration of hardware in which a program for executing a method according to an embodiment of the present invention is stored.

The server 1001 corresponds to a design generating device according to an embodiment of the present invention.

The server 1001 includes a memory 1000, a processor 1002, an input unit 1004, a display unit 1006, and a communication unit 1008.

The memory 1000 corresponds to the storage unit 204 and stores an item extraction model. The item extraction model may determine which item corresponds to a vector value in a corresponding image.

Also, the memory 1000 categorizes and stores the extracted items. In addition, the memory 1000 categorizes and stores design elements for each item extracted by the fashion design extraction unit 206. Also, the memory 1000 may store a model for ranking the generated new designs.

The processor 1002 creates a new design by using the collected image and text. It corresponds to the extraction unit 203 and the design generation unit 208 of the design generation device.

The processor 1002 uses the collected image and text to extract items using the image and text using an item extraction model stored in the memory 1000, and then extracts a design or design element for each item. do. Alternatively, the processor 1002 may extract design and design elements by using images and texts from which the item is not extracted, even without extracting the item.

Further, the processor 1002 extracts a higher design or higher design element from the extracted design or design element. Specifically, a higher design or design element may be determined based on a case in which the frequency of exposure of a specific design or design element on an image exposed to an SNS or a website is greater than or equal to a reference value. In addition, a higher design or design element may be determined based on a case in which the sales amount and sales amount of an item including a specific design or design element in a specific website is greater than or equal to a reference value. In addition, high-level design based on the preference for the image displayed on the Internet, such as a specific website or SNS (determined based on all data that can estimate the preference for the image, such as the number of clicks, feedback, and shares) Or you can judge design elements. In addition, when determining whether a specific design or design element is a higher-level design or design element, it is possible to determine whether or not a celebrity such as a celebrity wears such a design with more weight.

Also, the processor 1002 trains a design generation model.

The processor 1002 is a higher level among image data and text data parsed using artificial intelligence, design elements/design elements for the extracted specific item, and design elements for a specific item extracted from the design extraction unit. Design generation models can be created and trained by reflecting design elements.

In addition, the processor 1002 learns the design generation model by additionally considering the generated design or design element for a specific item, and also evaluates the generated design or design element for a specific item and evaluates the result. Consider additionally and train the design generation model.

Subsequently, the processor 1002 generates a new design by using the upper designs and the design generation model.

As an example of creating a design, there is a method of creating a new design element by changing a vector value of a trendy design element. The change of the vector value may be input as a set value and may be made by the learned design generation model. Specifically, for example, referring to FIG. 2D, if the floral dress is fashionable, the vector values of the existing floral patterns in the floral dress extracted from the collected image are changed to create a new floral dress design. can do.

In addition, as another embodiment of creating a new design element, there is a method of creating a new design for a specific item by synthesizing different elements among design elements. Specifically, it is possible to create a new one by merging different elements among the elements of each trendy design. For example, referring to FIG. 2E, a design of an ultra violet color long one-piece with puff sleeves is synthesized by combining “puff sleeves” of knit, “ultra violet color”, and “long dress” of a shirt among trendy design elements. Can be created. In addition, these different elements may be automatically and randomly merged, or may be merged according to input settings (for design elements to be merged).

In addition, as another embodiment of creating a new design element, there is a method of generating a new design element by randomly changing and combining design elements.

Further, the processor 1002 evaluates and ranks the generated design or design element. In order to rank new designs, in general, the frequency of exposure of a specific design or design element on images exposed on social media or websites, sales volume and sales amount of items containing a specific design, specific websites or SNS, etc. It can be determined based on the preference for the image (determined based on all data that can estimate the preference for the image, such as the number of clicks, the number of feedbacks, and the number of shares). In addition, when judging the frequency of exposure or determining the ranking, weights are given to designs or design elements exposed on SNS or images of celebrities or celebrities, and among the elements of design, designs sold by a specific brand or a specific website are given weight. If the preference, sales volume, or sales amount is higher, weights can be given to elements of the design (hereinafter, “weight determination”). The design generating device 200 ranks the new designs or design elements generated in this way, and derives only designs having a predetermined ranking or higher as a final design.

These ranked results are reflected when training the design generation model.

The display unit 1006 may display the generated design.

The communication unit 1008 may receive image data or text data, or transmit a design to be manufactured and a quantity to be manufactured to a factory when manufacturing is to proceed with respect to the generated design.

In addition, an apparatus or system according to various embodiments may include at least one or more of the above-described elements, some of the above-described elements may be omitted, or additional other elements may be further included. In addition, the embodiments disclosed in this document are presented for explanation and understanding of the disclosed technical content and do not limit the scope of the present invention. Therefore, the scope of this document should be construed as including all changes or various other embodiments based on the technical idea of the present invention.

Claims (1)

As an automatic interior design generation device using AI,
A collection means for collecting an image and information related to the image through a network, including a plurality of categories as a design object of the automatic design generating device, and including an interior as one of the plurality of categories;
Extracting means for extracting a plurality of constituent elements or features characterizing the image in the image collected through the collection unit;
An analysis unit for analyzing a trend based on a plurality of constituent elements in the image extracted by the extraction unit and information collected through the collection unit;
Learning means for machine learning a design generation model using the collected images; And
Design generation means for generating a new interior design by using the design generation model learned by the learning means and reflecting the trend analysis result of the analysis unit based on the input of the interior category as the design target category selected by the user Including ;,
The design generating means is characterized in that in generating the new interior design, an image included in a category other than the interior category among the plurality of categories and information related to the image are used together.
Automatic interior design generation device using AI.

Images