KR102366595B1 - System, method and program for obtaining appearance descriptive data from image data - Google Patents

Abstract

The present invention relates to a method, apparatus and program for acquiring outline description data of image data.
According to an embodiment of the present invention, there is provided a method for obtaining outline description data of image data, comprising: calculating, by a server, image data into an object type recognition model to calculate object type information (S3100); calculating, by the server, the individual appearance characteristics for a plurality of appearance classification criteria by inputting the image data into an appearance characteristic recognition model (S3200); and generating, by the server, the outline description data by combining the object type information calculated with respect to the image data and a plurality of individual appearance characteristics (S3300).

Description

Appearance description data acquisition method, device and program of image data {SYSTEM, METHOD AND PROGRAM FOR OBTAINING APPEARANCE DESCRIPTIVE DATA FROM IMAGE DATA}

The present invention relates to a method, apparatus and program for acquiring outline description data of image data.

Recently, with the development of the Internet, social media network services are rapidly growing. As a result, as the amount of multimedia explosively increases, an effective image search system is required, and the importance of image annotation is increasing due to the need for efficient image search according to the explosively increasing web image.

Most of the image retrieval studies have mainly conducted a content-based image retrieval (CBIR) method that analyzes the content of an image. Content-based image retrieval uses visual features such as color, texture, and shape to analyze the content of an image. This method works well when the number of tags to be defined is small, but the performance deteriorates as the dataset grows and the types of tags diversify.

Text-based image retrieval (TBIR) is a method of searching for images corresponding to text by using text as a query. In this method, the visual content of an image is expressed by a manually tagged text descriptor, and is used to perform image search in a dataset management system. That is, in the existing image or video search method, a search is performed based on information directly tagged by a user. In addition, the method of acquiring user interest information through an image is also acquired based on information directly tagged by the user, so there is a problem in that the acquisition result is inaccurate if the user tags the keyword incorrectly on the image. In addition, since there may be differences in keywords defined for each user, there is a problem in that results provided according to keywords selected by a user inputting an image are different.

Laid-open Patent Publication No. 10-2018-0133200, 2018.12.13

The present invention is a method of generating image data in the form of a code string form by acquiring type information of a specific object, an external characteristic expressing the external appearance of the object, or an abstract characteristic recognizable to the object from image data. It is intended to provide a method, apparatus and program for acquiring appearance description data.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

The method for obtaining outline description data of image data according to an embodiment of the present invention is performed by a server, wherein image data is input into an object type recognition model to calculate type information on the type of a first object to do; inputting the image data into an appearance characteristic recognition model for the first object type, and calculating a plurality of individual appearance characteristics for a plurality of appearance classification criteria; verifying the first object type information based on a result of calculating the individual appearance characteristics; and generating appearance descriptive data by combining the first object type information and the plurality of individual appearance characteristics, wherein the appearance classification criterion is a specific classification criterion for describing the appearance of a specific object, and the object includes a plurality of individual appearance characteristics expressing various appearance characteristics within the same classification criterion of , all of a plurality of pieces of candidate type information having the first probability equal to or greater than a threshold value are calculated, and in the individual appearance characteristic calculation step, the image data is inputted into a specialized appearance characteristic recognition model corresponding to each of the plurality of candidate type information to make the Individual appearance characteristics for each of the plurality of candidate type information are calculated, and candidate type information corresponding to the candidate appearance characteristic recognition model having the smallest number or ratio of individual characteristic recognition modules for which individual appearance characteristics are not calculated is set as the final type information. characterized in that

In addition, the appearance characteristic recognition model includes a plurality of individual characteristic recognition modules corresponding to a plurality of exterior classification criteria set to be applied to specific object type information, and the verification step includes individual appearance among the plurality of individual characteristic recognition modules. When the number or ratio of individual characteristic recognition modules for which characteristics are not calculated is equal to or greater than a threshold value, it is determined that the external characteristic recognition model cannot calculate the plurality of individual external characteristics.

The method further includes calculating an abstract characteristic based on the plurality of individual appearance characteristics, and the appearance description data further includes the calculated abstract characteristic.

In addition, the calculating step of the abstract characteristic calculates the abstract characteristic by inputting the appearance description data to an abstract characteristic recognition model, and the calculating of the appearance description data includes: based on the calculated abstract characteristic, the appearance description data Can be updated.

In addition, in the calculating step of the abstract characteristic, a score for each of a plurality of abstract characteristics is set for each individual external characteristic, and the score for each abstract characteristic set in the plurality of individual external characteristics can be summed to calculate it. there is.

A server apparatus for obtaining external appearance description data of image data according to another embodiment of the present invention includes one or more computers, and performs the above-mentioned external appearance description data acquisition method.

delete

delete

delete

delete

delete

delete

delete

The program for obtaining outline description data of image data according to another embodiment of the present invention is combined with hardware to execute the above-mentioned method for obtaining outline description data, and is stored in a medium.

Other specific details of the invention are included in the detailed description and drawings.

According to the present invention, it is possible to obtain type information, individual appearance characteristics, or abstract characteristics of an object based on the same criteria for a plurality of image data, and generate appearance description data based on the obtained type information.

In addition, according to the present invention, by calculating object type information of specific image data and inputting the image data into a specialized appearance characteristic recognition model corresponding thereto, the combination of appearance classification criteria applied according to the type information is changed more efficiently individual appearance characteristics can be obtained.

In addition, according to the present invention, for specific image data, abstract characteristics are calculated based on the appearance description data (obtained object type information or individual appearance characteristics), and the calculated abstract characteristics are added to the appearance description data. It is possible to acquire abstract characteristics of image data and update appearance description data.

In addition, according to the present invention, it is possible to minimize a situation in which the appearance characteristic recognition model needs to be re-learned or additionally learned. For example, when a new appearance classification criterion is added to a specific object, only the new individual characteristic recognition module needs to be learned, and the existing individual characteristic recognition module can be used as it is.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a flowchart of a method for acquiring outline description data of image data according to an embodiment of the present invention.
2 is a block diagram of an appearance characteristic recognition model according to an embodiment of the present invention.
3 is an exemplary diagram for explaining a code system according to an embodiment of the present invention.
4 is a flowchart of a method for obtaining outline description data further comprising the step of calculating abstract characteristics according to an embodiment of the present invention.
5 is a block diagram of an abstract characteristic recognition model according to an embodiment of the present invention.
6 is an exemplary diagram for explaining the setting of a first individual emotional characteristic score for an individual external characteristic according to an embodiment of the present invention.
7 is a flowchart of a method for obtaining outline description data further comprising the step of calculating detailed type information according to an embodiment of the present invention.
8 is a flowchart of a method for obtaining outline description data further including a user search step according to an embodiment of the present invention.
9 is a block diagram of an appearance description data acquisition server according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the present embodiments allow the disclosure of the present invention to be complete, and those of ordinary skill in the art to which the present invention pertains. It is provided to fully understand the scope of the present invention to those skilled in the art, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

As used herein, the term 'computer' includes various devices capable of providing a result to a user by performing arithmetic processing. For example, computers include desktop PCs and notebooks (Note Books) as well as smart phones, tablet PCs, cellular phones, PCS phones (Personal Communication Service phones), synchronous/asynchronous A mobile terminal of International Mobile Telecommunication-2000 (IMT-2000), a Palm Personal Computer (PC), a Personal Digital Assistant (PDA), and the like may also be applicable. In addition, when a head mounted display (HMD) device includes a computing function, the HMD device may be a computer. In addition, the computer may correspond to the server 10 that receives a request from a client and performs information processing.

In this specification, a 'client' refers to any device including a communication function that users can use by installing a program (or application). That is, the client device may include one or more of a telecommunication device such as a smart phone, a tablet, a PDA, a laptop, a smart watch, a smart camera, and a remote controller, but is not limited thereto.

As used herein, the term 'object' refers to an article of a specific classification or category included in image data.

As used herein, 'image data' refers to a two-dimensional or three-dimensional static or dynamic image including a specific object. That is, 'image data' may be static image data that is one frame, or dynamic image data (ie, moving image data) in which a plurality of frames are continuous.

In the present specification, the 'appearance classification standard' refers to a classification criterion for an appearance expression necessary for describing or annotating the appearance of a specific object. That is, the 'appearance classification criterion' is a specific classification criterion for describing the appearance of a specific object, and includes a plurality of individual appearance characteristics expressing various appearance characteristics within the same classification criterion of the object. For example, when the object is clothing, the appearance classification criterion is a classification criterion for the appearance of the clothing, and may correspond to a pattern, a color, a fit, a length, and the like. That is, if the appearance classification standard for a specific object increases, the appearance of a specific article belonging to the object may be described in detail.

As used herein, the term 'individual appearance characteristics' refers to various characteristics included in specific appearance classification criteria. For example, when the appearance classification criterion is color, the individual appearance characteristics mean various individual colors.

In the present specification, an 'abstract characteristic' is an abstract characteristic recognized with respect to a specific object, and includes a first emotional characteristic or a second emotional characteristic.

In the present specification, the 'first emotional characteristic' refers to an emotional characteristic recognized based on the appearance of a specific object. For example, it may be an emotional or fashionable expression such as 'cute' or 'vintage' for the appearance of a specific object.

In the present specification, the 'second emotional characteristic' refers to an emotional characteristic recognized based on information given to a product of a specific object. For example, it may be an emotional expression such as 'cheap' or 'expensive' recognized for a price among product information of a specific object.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a flowchart of a method for obtaining outline description data of image data according to an embodiment of the present invention.

Referring to FIG. 1 , a method for obtaining outline description data of image data according to an embodiment of the present invention includes: inputting image data into an object type recognition model by a server, and calculating object type information (S3100); calculating, by the server, the individual appearance characteristics for a plurality of appearance classification criteria by inputting the image data into an appearance characteristic recognition model (S3200); and generating, by the server, the shape description data by combining the object type information calculated with respect to the image data and a plurality of individual shape characteristics (S3300). Hereinafter, a detailed description of each step will be described.

The server 10 inputs the image data into the object type recognition model 100 to calculate object type information (S3100). That is, object type information that is a specific classification or category of an object included in the image data is calculated.

In an embodiment, the object type information may be calculated based on a preset object type classification. For example, the upper type classification is set to clothing, accessories, shoes, interior, etc., and the subtype classification for clothing among the upper types is 'Shirt & Blouse', 'Jacket', 'Coat', 'Pants', 'Skirt' ', 'leggings & stockings', and 'one piece', when image data for clothing is input, the object type recognition model may calculate object type information from among the seven subtypes for clothing.

Also, in an embodiment, the calculating of the object type information ( S3100 ) may calculate a plurality of type information. That is, when it is difficult to calculate one specific object type information for the input image data, it is possible to calculate a plurality of type information with high probability.

In an embodiment, the step of calculating the object type information ( S3100 ) may include obtaining a first probability that an object of specific image data corresponds to each specific type information, and calculating type information in which the first probability is equal to or greater than a threshold value. there is.

For example, when the threshold value is 0.4, the first probability that the object of the input image data corresponds to 'shirt & blouse' is 0.8, 'jacket' is 0.1, and 'coat' is 0.1, 'shirt & blouse' ' can be calculated as object type information.

Also, in an embodiment, when there are a plurality of type information having the first probability equal to or greater than a threshold value, the plurality of type information may be calculated as candidate type information.

For example, if the threshold value is 0.4, the first probability corresponding to 'Shirt & Blouse' is 0.5, 'Jacket' is 0.4, and 'Coat' is 0.1, 'Shirt & Blouse' and 'Jacket' are selected. Can be calculated from candidate type information. The final type information setting for the candidate type information will be described later.

Next, the server 10 inputs the image data to the external appearance characteristic recognition model 200, and calculates individual appearance characteristics for a plurality of external appearance classification criteria (S3200).

In one embodiment, the appearance characteristic recognition model 200 includes a plurality of individual characteristic recognition modules 210 for determining different appearance classification criteria. That is, the appearance characteristic recognition model includes a plurality of individual characteristic recognition modules specialized for recognizing each appearance classification criterion. As the number of appearance classification criteria of a specific object increases, the server includes a plurality of individual characteristic recognition modules in the exterior characteristic recognition model. The individual characteristic recognition module calculates individual appearance characteristics included in a specific appearance classification criterion of image data.

Also, in one embodiment, the individual characteristic recognition module 210 is trained through a deep learning learning model by matching individual appearance characteristics of a specific appearance classification criterion with respect to a plurality of image data for learning. That is, the individual characteristic recognition module is built with a specific deep learning algorithm, and learning is performed by matching a specific one of a plurality of appearance classification criteria with image data for learning.

To this end, the server 10 may perform a process of training each individual characteristic recognition module 210 as follows.

First, the server acquires a plurality of learning image data for a specific object. For example, when the object is a specific clothing type (eg, a shirt), the server acquires images of several shirts.

The server acquires a definition of each appearance classification criterion and a plurality of individual appearance characteristics for each appearance classification criterion. That is, the server sets the initial number of individual characteristic recognition modules by setting a plurality of appearance classification criteria. Then, as the server sets a plurality of individual appearance characteristics in each appearance classification criterion, it sets a feature type for labeling the image data for learning for each appearance classification criterion.

In an embodiment, the server may receive a plurality of appearance classification criteria for analyzing a specific object appearance and a plurality of individual appearance characteristics within each external appearance classification criterion from an expert client for analyzing a specific object appearance. For example, when constructing an appearance characteristic recognition model for clothing, the server may receive an appearance classification standard and individual appearance characteristics included therein from a client of a designer who is a clothing expert.

Then, the server labels the training image data with a plurality of individual appearance characteristics of each appearance classification criterion. That is, the server receives and matches at least one individual appearance characteristic for each of the plurality of appearance classification criteria for each training image data.

After that, the server performs training based on the training data set matching the individual appearance characteristics of the specific appearance classification criteria labeled with the image data for training. That is, when the server trains the individual characteristic recognition module for the appearance classification criterion A, only the image data for training and the individual appearance characteristics of the appearance classification criterion A matched thereto are extracted from the learning dataset and input to the deep learning learning model. Through this, the server 10 builds each individual characteristic recognition module 210 capable of recognizing individual appearance characteristics of each exterior classification criterion.

In addition, in one embodiment, the individual appearance characteristic calculating step (S3200) comprises inputting image data into each individual characteristic recognition module in the exterior characteristic recognition model to calculate a plurality of individual appearance characteristics for the image data. will be. Through this, the server acquires all the individual appearance characteristics of each appearance classification criterion for the image data.

Also, in one embodiment, the calculating of the individual external characteristics ( S3200 ) may include inputting the image data into a specialized external characteristic recognition model corresponding to the calculated object type information to calculate a plurality of individual external characteristics. .

In an embodiment, the specialized appearance characteristic recognition model includes an individual characteristic recognition module of a plurality of appearance classification criteria preset to be applied according to specific object type information. That is, the type of the appearance classification criterion applied may be determined according to the object type information calculated with respect to specific image data.

Specifically, referring to FIG. 2 , specialized appearance characteristic recognition in which different combinations of appearance classification criteria (ie, combination of individual characteristic recognition modules) are set according to specific object type information (object 1, object 2, object 3) A model can be generated, and the image data is input to each of a plurality of individual characteristic recognition modules in the specialized appearance characteristic recognition model of the corresponding type information according to the object type information calculated for specific image data, and individual appearance characteristics are calculated can do. Also, it goes without saying that the combination of individual characteristic recognition modules in the specialized appearance characteristic recognition model of different object type information may be the same.

For example, the specialized external characteristic recognition model of 'jacket' type information is 'neckline', 'sleeve', 'top length' and 'color' individual characteristic recognition module When the specialized appearance characteristic recognition model of 'pants' type information includes individual characteristic recognition modules for 'waist', 'bottom length' and 'color', appearance characteristics According to the object type information (jacket/pants) of the image data input to the recognition model, the specialized appearance characteristic recognition model to which the image data is input (ie, a combination of applied appearance classification criteria) may be different.

In addition, in one embodiment, as shown in FIG. 2 , each specialized appearance characteristic recognition model for a plurality of object type information may be used by sharing a general-purpose individual characteristic recognition module. The general-purpose individual characteristic recognition module refers to an individual characteristic recognition module for general-purpose appearance classification criteria applicable regardless of the type of object.

For example, the appearance classification criteria such as 'color', 'pattern', and 'texture' can be applied regardless of the type of object (calculation of individual appearance characteristics), so it is a universal appearance classification standard may correspond to

Also, in an embodiment, when a plurality of pieces of candidate type information are calculated in step S3100, the image data may be respectively input to all of the specialized appearance characteristic recognition models corresponding to each of the candidate type information. The final type information setting performed based on this will be described later.

For example, when 'jacket' and 'pants' are calculated as candidate type information for specific image data, the image data is input to each of the 'jacket' specialized appearance characteristic recognition model and the 'pants' specialized appearance characteristic recognition model. Individual appearance characteristics can be calculated.

delete

delete

delete

Further, in another embodiment, when a plurality of candidate type information is calculated in step S3100 and the image data is respectively input to the individual characteristic recognition module of the specialized appearance characteristic recognition model corresponding to each of the candidate type information, individual appearance characteristics are obtained. It further includes; setting candidate type information having the fewest appearance classification criteria that cannot be calculated as final type information.

That is, as in the above-described embodiment, it is difficult to calculate one specific object type information with respect to the input image data, so a plurality of candidate type information with high probability is calculated, and the image data is used as a specialized appearance characteristic recognition model of each candidate type information. When input to , candidate type information with a high recognition rate (the number of individual characteristic recognition modules that cannot calculate individual appearance characteristics is small) can be set as the final type information.

For example, candidate type information of 'jacket' and 'pants' is calculated in step S3100 for the image data about the jacket, and the corresponding image data is applied to the 'jacket'-specialized appearance characteristic recognition model and 'pants'-specialized appearance characteristic recognition model. In the case of each input, the recognition rate of each specialized appearance characteristic recognition model will be higher in 'jacket' than in 'pants' unless there are special circumstances (the number of individual characteristic recognition modules that cannot calculate individual appearance characteristics is 'pants' specialized Since there will be more in the appearance characteristic recognition model), 'jacket' can be set as the final type information (ie, object type information).

Next, the server generates the outline description data by combining the object type information calculated with respect to the image data and a plurality of individual appearance characteristics (S3300). That is, if the appearance classification criteria for a specific object are divided in detail, the appearance description data specifically describes the appearance of the object through individual appearance characteristics.

For example, with respect to specific image data on a shirt, the object type information calculated in step S3100 is 'shirt & blouse', and in the plurality of appearance classification criteria calculated in step S3200 (neck shape, sleeve, top length, color) and pattern) for 'Round Neckline', 'Short Sleeve', 'Crop', 'Red Right' and 'Polka Dot' ', the outline descriptive data for the image data may be generated as "shirt & blouse, round neck, short sleeves, crop top, bright red, polka dots".

In one embodiment, the step of generating the outline description data (S3300) may include: extracting object type information for the image data and a code value corresponding to each of a plurality of individual appearance characteristics; and generating outline description data in the form of a code string by combining the plurality of code values. That is, as the server codes the individual appearance characteristics, it is possible to generate the outline description data as a code string, and through this, the processing of the outline description data can be efficiently performed.

For example, code values corresponding to object type information (shirt & blouse) and a plurality of individual appearance characteristics (round neck, short sleeves, crop top, bright red, polka dots) are 'Zb01', 'Bb01', ' In case of Bg03', Bi01', 'Oa01', or 'Id00', the outline description data for the image data may be generated as "Bb01, Bg03, Bi01, Id00, Oa01, Zb01" by combining the code values.

In addition, the structure of the code system of the appearance description data according to an embodiment of the present invention may be constructed in various ways according to each object type classification criterion, appearance classification criterion, and abstract characteristic classification criterion.

For example, the code system according to an embodiment of the present invention may be constructed as shown in FIG. 3 . Referring to FIG. 3 , capital letters mean information on higher categories. That is, the code corresponding to the object type information may include Z, the code corresponding to the abstract characteristic may include A, and the code corresponding to the plurality of appearance classification criteria may include B to Y.

In addition, in the case of a code (Z) corresponding to the object type information, a lowercase letter (a~y) is assigned according to the upper type (eg, clothing, accessories, shoes, interior, etc.), and each A number (00 to 99) may be assigned according to the object type information. For example, Zb may be 'clothing', Zc may be 'accessories', Zb01 may be 'shirt & blouse', and Zb02 may be a code value corresponding to object type information of 'jacket'.

In addition, the codes corresponding to the plurality of appearance classification criteria (B to Y) include general-purpose appearance classification standards (E, I, O, U) that can be applied regardless of object type and specialized appearance classification standards applied to specific object types (E, I, O, U). B, C, D, etc.), and the code of the specialized external classification criterion may correspond to the above-described upper-type code (Za, Zb, Zc, etc.).

For example, a plurality of specialized appearance classification criteria for 'clothes' include B, and a lowercase letter (a~y) is given according to each appearance classification criterion, and a number according to the individual appearance characteristics for the appearance classification criterion (00 to 99) may be assigned. Specifically, Ba stands for 'Top Silhouette', Bb stands for 'Neckline', Bi stands for 'Pants Silhouette', and Bb01 stands for 'Round Neckline'. )' and Bb02 may be code values corresponding to individual appearance characteristics for the neck shape (Bb) appearance classification standard of 'V Neckline'. Examples of code values corresponding to abstract characteristics will be described later.

In the above, an example of the code system structure according to an embodiment of the present invention has been described, but the code system of the present invention is not limited thereto and can be constructed in various ways.

Referring to FIG. 4 , the method for obtaining the appearance description data of image data according to an embodiment of the present invention further includes: calculating, by the server, an abstract characteristic based on a plurality of individual appearance characteristics calculated (S3400); do.

In an embodiment, the calculating of the abstract characteristics ( S3400 ) includes calculating the abstract characteristics by the server inputting the outline description data into the abstract characteristic recognition model 300 .

That is, according to the present invention, as abstract characteristics are calculated based on appearance descriptive data (object type information and individual appearance characteristics) calculated from image data, not image data itself such as an image, efficient processing of data and objective abstract characteristics calculation is possible.

Also, in one embodiment, when the abstract characteristics are calculated by inputting the object type information and the individual appearance description data combined with the individual appearance characteristics to the abstract characteristic recognition model, the appearance description data is obtained by adding the abstract characteristics to the appearance description data. The step of updating may be further included.

In an embodiment, the abstract characteristic includes a first emotional characteristic. The first emotional characteristic is an emotional characteristic recognized based on the appearance of a specific object, and includes a plurality of first individual emotional characteristics that are specific emotional characteristics.

In an embodiment, the definition and number of each first individual emotional characteristic included in the first emotional characteristic may be set by the server, and may be added or changed. The first emotional characteristic, which is an emotional characteristic for the external appearance of the object, may be defined differently according to times or regions, so that it can be variously changed according to this.

For example, each of the first individual emotional characteristics may be set to 'Cute', 'soft', 'Modern', etc. The contrasting first individual emotional characteristics may be set to further include 'Elegant', 'Wild', 'Classic', and the like. However, the first individual emotional characteristic of the present invention is not limited thereto and may be set in various ways.

Referring to FIG. 5 , in one embodiment, the abstract characteristic recognition model 300 includes a first emotional characteristic recognition model 310 that receives individual appearance characteristics and calculates first emotional characteristics.

In an embodiment, in the first emotional characteristic recognition model, as scores for each of a plurality of first individual emotional characteristics are set for each individual external characteristic, each first individual emotion set in the input plurality of individual external characteristics The first emotional characteristic may be calculated by summing the scores for the characteristics.

For example, as shown in FIG. 6 , a score for each of the first individual emotional characteristics may be set for each of a plurality of individual appearance characteristics included in each appearance classification criterion. In FIG. 6 , each score is indicated as 0 or 1, but it is not limited thereto, and the score may be set in various ways, such as a number between 0 and 1 or a negative number.

In an embodiment, a score sheet in which a score for each individual emotional characteristic is set for each individual appearance characteristic may generate not only one score sheet but also a plurality of different score sheets. For example, a score sheet may be different for each country or region, or a score sheet personalized for each user may be generated, and it goes without saying that the score sheet may be freely changed by the server.

Then, based on the plurality of input individual appearance characteristics and each first individual emotional characteristic score set in the individual external characteristic, each first individual emotional characteristic score is summed to calculate a first emotional characteristic.

For example, in the example of FIG. 6 , when individual appearance characteristics of 'V-neck' and 'red' calculated from specific image data are input, the summed score for each first individual emotional characteristic is 'Cute: 1' , 'elegant: 2', 'soft: 0', 'rough: 1', 'modern: 0', 'classic: 1', and based on these, the first emotional characteristic can be calculated.

In an embodiment, the first emotional characteristic may be calculated including a ratio of each of the first individual emotional characteristic scores to the total score. For example, in the case of the above example, since the overall score is 5, the first emotional characteristic is "cute: 0.2, elegant: 0.4, soft: 0, rough: 0.2, modern" to include the ratio of each first individual emotional characteristic score. Enemy: 0, classic: 0.2".

In another embodiment, the first emotional characteristic may be calculated including each first individual emotional characteristic score. For example, in the case of the above example, the first emotional characteristic is "cute: 1, elegant: 2, soft: 0, rough: 1, modern: 0, classic: 1, so as to include each first individual emotional characteristic score. " can be calculated as

In another embodiment, as the first emotional characteristic, only the first individual emotional characteristic in which each first individual emotional characteristic score is equal to or greater than a preset threshold value is calculated. For example, in the above example, when the threshold value is 2 (or ratio 0.4), only the first individual emotional characteristic of 'elegant' may be calculated as the first emotional characteristic. However, the calculation of the first emotional characteristic is not limited to the above-described example and may be calculated in various ways.

Also, in an embodiment, the code value corresponding to the first emotional characteristic may include information about a score summed for each of the first individual emotional characteristics.

Specifically, in the above example, the first emotional characteristic is calculated as "cute: 0.2, elegant: 0.4, soft: 0, rough: 0.2, modern: 0, classic: 0.2", and each first individual emotional characteristic When the code value corresponding to "Cute: Aa, Elegant: Ac, Soft: Ad, Rough: Af, Modern: Ai, Classic: Ap" is, the appearance descriptive data in the form of a code string for the first emotional characteristic is "Aa20, Ac40, Ad00, Af20, Ai00, Ap20". In addition, if the code values corresponding to the individual appearance characteristics 'red' and 'V-neck' are 'Oa02, 'Bb02', the external description data of the image data is "Aa20, Ac40, Ad00, Af20, Ai00, Ap20" , Bb02, Oa02". However, as described above, the code system of the present invention is not limited thereto and may be constructed in various ways.

In an embodiment, the abstract characteristic includes a second emotional characteristic. The second emotional characteristic is an emotional characteristic recognized based on information given to a product of a specific object, and includes a second individual emotional characteristic that is various emotional characteristics felt for different types of product information.

For example, the second individual emotional characteristics of 'cheap' and 'expensive' felt for product information of 'price' or the second individual emotional characteristic of 'fast' and 'slow' felt for product information of 'delivery time' may include

Referring to FIG. 5 , in one embodiment, the abstract characteristic recognition model 300 includes a second emotional characteristic recognition model 320 that receives product information on an object of image data and calculates second emotional characteristics. do.

Also, in one embodiment, the second emotional characteristic recognition model 320 includes a plurality of second individual emotional characteristic recognition modules for determining emotional characteristics for different product information. The second individual emotional characteristic recognition module calculates each second individual emotional characteristic for specific product information of the object of the image data.

Also, in an embodiment, the second emotional characteristic may be calculated in consideration of not only the product information but also various information such as individual appearance characteristics, object type information, or user information.

Referring to FIG. 7 , in the method for obtaining outline description data of image data according to an embodiment of the present invention, the server 10 inputs outline description data of the image data into a detailed type recognition model 400, It further includes; calculating the detailed type information (S3500).

In an embodiment, the detailed type information may be a sub-concept of the object type information calculated in step S3100. For example, when 'pants' object type information is calculated for specific image data in step S3100, the detailed type information may be 'skinny pants', 'slacks pants', 'exhaust pants', and the like.

The detailed type recognition model may be a model that calculates detailed object type information of the image data when appearance description data (object type information, individual appearance characteristics) generated with respect to specific image data is input.

In an embodiment, the detailed type recognition model may be to calculate detailed type information for the individual appearance characteristic combination of the input appearance description data as one or more detailed type information is matched to each of one or more individual appearance characteristic combinations there is. Also, according to an embodiment, it is also possible to calculate a plurality of detailed type information.

For example, detailed type information (a2, a sub-concept when the object type information is A, b1, a sub-concept, etc.) Also, it is possible to calculate the corresponding detailed type information (b1) according to the input appearance description data (object type information B, individual appearance characteristics a, b, c).

Also, in an embodiment, when the detailed type information is calculated, the appearance description data may further include the detailed object type information. That is, in an embodiment, a code value corresponding to the calculated detailed type information may be extracted and added to the existing appearance description data.

Referring to FIG. 8 , the method for obtaining outline description data of image data according to an embodiment of the present invention includes: receiving, by the server, a search keyword from a user client (S3600); and extracting, by the server, image data including external description data corresponding to the search keyword and transmitting the image data to the user client (S3700). Hereinafter, a detailed description of each step will be described.

That is, when a user inputs a specific search keyword to search image data, the server extracts object type information, individual appearance characteristics, or abstract characteristics corresponding to the search keyword, and extracts the extracted type information, individual appearance characteristics or abstract characteristics. The image data including the appearance description data of the characteristic is provided to the user as a search result.

For example, when the user inputs 'cute red round T-shirt' as a search keyword, 'shirt & blouse' as object type information from the search keyword, 'red', 'round neck', 'collar' as individual appearance characteristics (collar) ', 'cute' is extracted as an abstract characteristic, and image data including the appearance descriptive data of "red, round neck, no collar, cute" can be provided to the user.

Also, in an embodiment, when extracting image data corresponding to the search keyword, preference information for each of the user's individual appearance characteristics, the first individual emotional characteristics, or the second individual emotional characteristics may be further taken into consideration. . In this case, even when the same search keyword is input, the search result may be different depending on the user inputting the search keyword.

In addition, although not shown in the drawings, in the method for acquiring external appearance description data of image data according to an embodiment of the present invention, when a new external appearance classification criterion for a specific object is added, the server performs a new external appearance classification for image data for learning building a new learning data set by acquiring the individual appearance characteristics of the standard; and training, by the server, a new individual characteristic recognition module based on the new learning dataset, and adding it to the external characteristic recognition model.

That is, when a new appearance classification criterion for a specific object is added (for example, when a new criterion for dividing the appearance characteristics of clothing is added), the server does not change the existing individual characteristic recognition module and applies the new appearance classification criterion to the new appearance classification criterion. By additionally constructing only the individual characteristic recognition module for

First, the server acquires the individual appearance characteristics of the new appearance classification criteria for the image data for learning, and builds a new learning dataset. In one embodiment, when a new individual characteristic recognition module is constructed by using the same image data previously used for training other individual characteristic recognition modules, the server provides a new appearance classification standard for each training image data from an expert client input the individual appearance characteristics of In addition, in another embodiment, the server acquires new image data for training the individual characteristic recognition module for the new appearance classification criterion, and receives individual appearance characteristics of the new appearance classification criterion for this, respectively, and creates a new learning dataset build

Thereafter, the server trains a new individual characteristic recognition module based on the new learning dataset, and adds it to the appearance characteristic recognition model. Through this, the server adds a new individual characteristic recognition module together with a plurality of existing individual characteristic recognition modules to the existing appearance characteristic recognition model (a plurality of specialized exterior characteristic recognition models).

In addition, in one embodiment, the server inputs the image data obtained by the individual characteristic recognition module that has already been built in the individual characteristic recognition module to the new individual characteristic recognition module to add individual appearance characteristics to the new appearance classification criteria; may include more. That is, the server performs a process of updating the appearance description data of the previously acquired image data to reflect the new appearance classification criteria. To this end, the server inserts all image data into the new individual characteristic recognition module to calculate individual appearance characteristics.

In addition, in an embodiment of the present invention, when the image data is moving picture data including a plurality of frames, the object type information calculating step S3100 and the individual appearance characteristic calculating step S3200 are each in the moving picture data. It may be characterized in that it is performed for a frame of , and the appearance description data generation step (S3300) may be characterized in that the object type information for each frame and a plurality of individual appearance characteristics are sequentially arranged and generated.

A server apparatus for obtaining external appearance description data of image data according to another embodiment of the present invention includes one or more computers, and performs the above-mentioned external appearance description data acquisition method.

Referring to FIG. 9 , the server apparatus 10 for obtaining external appearance description data of image data according to an embodiment of the present invention includes an object type recognition model 100 , an external appearance characteristic recognition model 200 , and an external appearance description data generating unit ( 600) and the database 800, and performs the above-described method for acquiring external appearance description data.

In addition, in another embodiment, the server device 10 further adds one or more of the abstract characteristic recognition model 300 , the detailed type recognition model 400 , the style recognition model 500 , or the recommended image data generator 700 . may include

In addition, the above-described method for acquiring outline description data of image data according to an embodiment of the present invention may be implemented as a program (or application) and stored in a medium in order to be executed in combination with a computer, which is hardware.

The above-described program is C, C++, JAVA, machine language, etc. that a processor (CPU) of the computer can read through a device interface of the computer in order for the computer to read the program and execute the methods implemented as a program It may include code (Code) coded in the computer language of Such code may include functional code related to a function defining functions necessary for executing the methods, etc., and includes an execution procedure related control code necessary for the processor of the computer to execute the functions according to a predetermined procedure. can do. In addition, the code may further include additional information necessary for the processor of the computer to execute the functions or code related to memory reference for which location (address address) in the internal or external memory of the computer to be referenced. there is. In addition, when the processor of the computer needs to communicate with any other computer or server 10 in a remote location in order to execute the functions, the code is transmitted to any other computer remotely using the communication module of the computer. It may further include a communication-related code for how to communicate with the server 10 and the like, and what kind of information or media should be transmitted and received during communication.

The storage medium is not a medium that stores data for a short moment, such as a register, a cache, a memory, etc., but a medium that stores data semi-permanently and can be read by a device. Specifically, examples of the storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the program may be stored in various recording media on the various servers 10 accessible by the computer or in various recording media on the computer of the user. In addition, the medium may be distributed in a computer system connected to a network, and a computer-readable code may be stored in a distributed manner.

In the above, the embodiments of the present invention have been described with reference to the accompanying drawings, but those skilled in the art to which the present invention pertains know that the present invention may be embodied in other specific forms without changing the technical spirit or essential features thereof. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

Claims (10)

A method, performed by a server, comprising:
inputting the image data into the object type recognition model, and calculating type information on the type of the first object;
inputting the image data into an appearance characteristic recognition model for the first object type, and calculating a plurality of individual appearance characteristics for a plurality of appearance classification criteria;
verifying the first object type information based on a result of calculating the individual appearance characteristics; and
Including; generating appearance description data combining the first object type information and the plurality of individual appearance characteristics;
The appearance classification criterion is a specific classification criterion for describing the appearance of a specific object, and includes a plurality of individual appearance characteristics expressing various appearance characteristics within the same classification criterion of the object,
The server is
In the step of calculating the type information, a first probability that an object of specific image data corresponds to each specific type information is calculated, and a plurality of pieces of candidate type information having the first probability equal to or greater than a threshold value are all calculated;
In the individual appearance characteristic calculation step, the image data is input to a specialized appearance characteristic recognition model corresponding to each of the plurality of candidate type information to calculate individual appearance characteristics for each of the plurality of candidate type information;
characterized in that candidate type information corresponding to a candidate appearance characteristic recognition model having the smallest number or ratio of individual characteristic recognition modules for which individual appearance characteristics are not calculated is set as final type information;
A method of generating outline description data of image data. The method of claim 1,
The appearance characteristic recognition model includes a plurality of individual characteristic recognition modules corresponding to a plurality of appearance classification criteria set to be applied to specific object type information;
The verification step is
If the number or ratio of individual characteristic recognition modules for which individual appearance characteristics are not calculated among the plurality of individual characteristic recognition modules is greater than or equal to a threshold value, it is determined that the exterior characteristic recognition model cannot calculate the plurality of individual appearance characteristics,
A method of generating outline description data of image data. delete The method of claim 1,
Further comprising the step of calculating an abstract characteristic based on the plurality of individual appearance characteristics,
The appearance description data further includes the calculated abstract characteristics,
A method of generating outline description data of image data. 5. The method of claim 4,
The step of calculating the abstract characteristic is
By inputting the appearance description data into an abstract characteristic recognition model, an abstract characteristic is calculated,
The step of calculating the outline description data is
Based on the calculated abstract characteristics, updating the appearance description data,
A method of generating outline description data of image data. 6. The method of claim 5,
The step of calculating the abstract characteristic is
A score for each of a plurality of abstract characteristics is set for each individual appearance characteristic, and the sum of the scores for each abstract characteristic set in the plurality of individual appearance characteristics is calculated,
A method of generating outline description data of image data. delete delete delete A program for generating outline description data of image data, which is combined with a computer that is hardware and stored in a medium to execute the method of any one of claims 1, 2, 4, 5 and 6.

Images