KR102595479B1 - service providing apparatus, system, method and program for reading an advertisement based on a machine learning model and determining whether the read advertisement image complies with guidelines - Google Patents

Abstract

A service provision device, system, method, and program that reads advertisements based on a machine learning model and determines whether the read advertisement image complies with guidelines is disclosed. The service providing device includes: an advertisement feature extraction unit that receives a first image, which is an advertisement image exposed to the user from a user terminal, and receives a second image, which is an advertisement image of companies corresponding to the first image; An advertising entity that determines a first matching degree corresponding to each of the searched companies based on the first image and the second image, and determines a company whose first matching degree is greater than or equal to a preset standard matching degree as the advertising entity company. matching department; And receiving a third image, which is an image corresponding to the advertising guideline information of the advertising company, and determining an advertising point provided in the first image and a second matching degree corresponding to the advertising point provided in the third image. and an advertisement reading unit that determines an advertisement in which the second matching degree is greater than or equal to a preset standard matching degree as an appropriate advertisement that complies with advertising guidelines.

Description

{service providing apparatus, system, method and program for reading an advertisement based on a machine learning model and determining whether the read advertisement image complies with guidelines}

The present invention relates to a service provision device, system, method, and program that reads advertisements based on a machine learning model and determines whether the read advertisement image complies with guidelines.

Unless otherwise indicated herein, the material described in this section is not prior art to the claims of this application, and is not admitted to be prior art by inclusion in this section.

Recently, the importance of advertising has increased, and as a result, advertising methods have become more diverse than in the past. However, recently, the number of users of SNS platforms has increased explosively, and the types of SNS platforms have become more diverse, so the proportion of advertisements through SNS (Social Network Service) platforms among companies' advertising methods is also increasing.

Meanwhile, when companies advertise products through SNS platforms, each company has advertising guidelines regarding advertising images posted on SNS platforms, and the advertising production company ensures that advertisements posted on SNS platforms comply with the company's advertising guidelines. I visually check it at the level of a person in charge within the company.

However, in the case of advertisements exposed on SNS platforms, there are many cases of non-compliance with advertising guidelines, which is a problem in that it takes a lot of time and money to monitor and correct this.

The matters described in the above background technology are intended to aid understanding of the background of the invention and may include matters that are not disclosed prior art.

The purpose of the present invention is to receive information related to advertisements displayed to users through SNS platforms, and use the information related to the received advertisements to easily determine whether advertisements displayed on SNS platforms comply with the company's advertising guidelines. The purpose is to provide service provision devices, systems, methods, and programs that determine whether images used in advertisements comply with advertising guidelines.

In addition, the present invention provides a service providing device, system, and method for determining whether images used in advertisements comply with advertising guidelines, and which can easily determine through machine learning which company's advertisements the advertisements exposed to the user are. The purpose is to provide programs.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned can be clearly understood by those skilled in the art from the description below.

One aspect of the present invention to solve the above-described problem provides a service providing device that determines whether images used in advertisements comply with advertising guidelines.

The device is a service providing device that determines whether images used in advertisements comply with advertising guidelines, and receives a first image, which is an advertisement image exposed to the user, from a user terminal, and advertisements of companies corresponding to the first image. an advertisement feature extraction unit that receives a second image, which is an image; An advertising entity that determines a first matching degree corresponding to each of the searched companies based on the first image and the second image, and determines a company whose first matching degree is greater than or equal to a preset standard matching degree as the advertising entity company. matching department; And receiving a third image, which is an image corresponding to the advertising guideline information of the advertising company, and determining an advertising point provided in the first image and a second matching degree corresponding to the advertising point provided in the third image. and an advertisement reading unit that determines an advertisement in which the second matching degree is greater than or equal to a preset standard matching degree as an appropriate advertisement that complies with advertising guidelines.

In addition, the advertising feature extractor extracts the first image, the keyword, the color of the keyword, the product, and the color of the product, and extracts the extracted keyword, color of the keyword, product, and the color of the product from the first image. Inputting a first color vector, a first keyword vector, and a first product vector from colors into a previously learned first machine learning model, and obtaining a first color vector corresponding to the first image from the first machine learning model, Obtain a first keyword vector and a first product vector, extract the second image, the company's product, the company's keyword, the color of the company's product, and the color of the company's keyword in the second image, and extract the extracted company's product. The second color vector, the second keyword vector, and the second product vector are input to a pre-trained second machine learning model to obtain the second color vector, the second keyword vector, and the second product vector from the product, the company's keyword, the color of the company's product, and the color of the company's keyword. A second color vector, a second keyword vector, and a second product vector corresponding to the second image can be obtained from the machine learning model.

In addition, the advertising subject matching unit calculates color similarity based on the cosine similarity between the first color vector and the second color vector, and calculates color similarity based on the cosine similarity between the first keyword vector and the second keyword vector. Calculating keyword similarity, calculating product similarity based on cosine similarity between the first product vector and the second product vector, and determining the first matching degree based on the following equation, In the above equation, M1 means the first matching degree, C is the color similarity, K is the keyword similarity, O is the product similarity, R1 is the first correction coefficient, R2 is the second correction coefficient, and R3 is the third correction coefficient. It may mean a correction coefficient.

In addition, the advertisement reading unit extracts a pixel value from the first image, inputs it to the first machine learning model that has already been trained to obtain an advertisement point vector from the extracted pixel value, and Obtaining a first advertising point vector corresponding to the first image, extracting a pixel value from the third image, inputting it to a third machine learning model previously trained to obtain an advertising point vector from the extracted pixel value, and A second advertising point vector corresponding to the third image can be obtained from the third machine learning model.

Additionally, the advertisement reading unit may calculate the second matching degree based on cosine similarity between the first advertisement point vector and the second advertisement point vector.

According to an embodiment of the present invention, it is possible to quickly determine which company the advertisement provided to the user is by using the company database.

Additionally, cost and time can be saved by using machine learning models to determine whether advertisements displayed on SNS platforms comply with the company's advertising guidelines.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

The accompanying drawings, which are included as part of the detailed description to aid understanding of the embodiments, provide various embodiments and describe technical features of the various embodiments along with the detailed description.
1 is a schematic diagram of a system for providing a service that determines whether images used in advertisements comply with advertising guidelines according to an embodiment.
FIG. 2 is a block diagram illustrating functional modules of the service providing device according to FIG. 1.
FIG. 3 is a flowchart illustrating an embodiment in which the advertising subject matching unit according to FIG. 2 matches an advertisement exposed to a user with a company.
FIG. 4 is a flowchart showing the specific process of step S140 of FIG. 3.
FIG. 5 is a flowchart illustrating an embodiment in which the advertisement reading unit according to FIG. 2 determines whether an advertisement exposed to a user is an appropriate advertisement that complies with advertising guidelines.
FIG. 6 is a diagram illustrating an exemplary hardware configuration of the service providing device according to FIG. 1.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention. While describing each drawing, similar reference numerals are used for similar components.

Terms such as first, second, A, and B may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be named a second component, and similarly, the second component may also be named a first component without departing from the scope of the present invention. The term and/or includes any of a plurality of related stated items or a combination of a plurality of related stated items.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

The terms used in this application are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the technical field to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless explicitly defined in the present application, should not be interpreted in an ideal or excessively formal sense. No.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings.

1 is a schematic diagram of a system for providing a service that determines whether images used in advertisements comply with advertising guidelines according to an embodiment.

Referring to FIG. 1, a system for providing a service providing device that determines whether images used in advertising comply with advertising guidelines includes a network 1, a service providing device 100, a user terminal 200, and a corporate database. It may be configured to include (300).

In the illustrated embodiment, the network 1, the service providing device 100, the user terminal 200, and the corporate database 300 are electrically connected to each other, but the present invention is not limited thereto.

The service providing device 100 and the user terminal 200 may be electrically connected through a wireless communication network. For example, the wireless communication network may be Long Term Evolution (LTE), 5 Generation (5G), and WiFi.

Meanwhile, the service providing device 100 and the user terminal 200 may be interconnected through a wireless communication network, but may also be interconnected through a wired communication network. In addition, various data, etc. can be transmitted and received between the service providing device 100 and the user terminal 200 in a combined wired and wireless communication network. Here, examples of various data transmitted and received between the service providing device 100 and the user terminal 200 may follow embodiments of the present invention described later.

Specifically, the network 1 serves to transmit data for data transmission and reception between the service providing device 100 and the user terminal 200, and depending on the system implementation method, Ethernet, xDSL (ADSL, VDSL), You can also use wired communication methods such as HFC (Hybrid Fiber Coaxial Cable), FTTC (Fiber to The Curb), and FTTH (Fiber To The Home), as well as WLAN (Wireless LAN), Wi-Fi, and Wibro. , wireless communication methods such as WiMAX, HSDPA (High Speed Downlink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution Advanced), and 5G (5-Generation) may be used.

In addition, this network 1 includes, for example, a plurality of access networks (not shown) and a core network (not shown), and may be configured to include an external network, such as an Internet network (not shown). Here, the access network (not shown) is an access network that performs wired and wireless communication with the user terminal 200, for example, a plurality of base stations such as BS (Base Station), BTS (Base Transceiver Station), NodeB, eNodeB, etc., and BSC. It can be implemented with a base station controller such as (Base Station Controller) or RNC (Radio Network Controller). In addition, as described above, the digital signal processing unit and the wireless signal processing unit integrated in the base station are divided into a digital unit (hereinafter referred to as DU) and a radio unit (hereinafter referred to as RU), respectively. , it can be configured by installing multiple RUs (not shown) in multiple areas and connecting multiple RUs (not shown) with a centralized DU (not shown).

In addition, the core network (not shown) that constitutes the mobile network together with the access network (not shown) serves to connect the access network (not shown) with an external network, such as an Internet network (not shown).

As described above, this core network (not shown) is a network system that performs major functions for mobile communication services such as mobility control and switching between access networks (not shown), and is a network system that performs circuit switching or packet switching. switching) and manages and controls packet flow within the mobile network. In addition, the core network (not shown) manages mobility between frequencies and plays a role in linking traffic within the access network (not shown) and the core network (not shown) with other networks, such as the Internet network (not shown). It may be possible. This core network (not shown) further includes Serving GateWay (SGW), PDN GateWay (PGW), Mobile Switching Center (MSC), Home Location Register (HLR), Mobile Mobility Entity (MME), and Home Subscriber Server (HSS). It may be configured to include.

In addition, the Internet network (not shown) refers to a typical open communication network, that is, a public network, in which information is exchanged according to the TCP/IP protocol. It is connected to the user terminal 200 and provides information from the user terminal 200. Information can be provided to the network 1 through the core network (not shown) and the access network (not shown), and conversely, the information provided to the service providing device 100 can be provided to the core network (not shown) and the access network (not shown). It may also be provided to the user terminal 200 through . However, it is not limited to this, and the service providing device 100 may be implemented integrally with a core network (not shown).

In addition, in addition to the above-mentioned communication methods, it may include all other types of communication methods that are widely known or will be developed in the future.

The user terminal 200 is a terminal of a user using the service that determines whether the image used in the advertisement provided by the service providing device 100 complies with the advertising guidelines, and is an advertisement exposed to the user on the service providing device 100. A first image that is an image of can be provided.

In one embodiment, the first image may be the entire screen where advertisements encountered by the user through various SNS platforms are displayed. However, it is not limited to this.

In addition, the service providing device 100 provides a user interface (User Interface) through which the user can input an advertising image that the user has encountered, to the user terminal 200, and the user terminal 200 displays the advertising image input from the user. It can be transmitted to the service provision device 100.

The user terminal 200 may be, for example, a desktop computer, a laptop computer, a laptop, a smart phone, a tablet PC, or a mobile phone capable of communication. ), smart watch, smart glass, e-book reader, PMP (portable multimedia player), portable game console, navigation device, digital camera, DMB (digital multimedia broadcasting) It may be a player, a digital audio recorder, a digital audio player, a digital video recorder, a digital video player, a PDA (Personal Digital Assistant), etc.

In the illustrated embodiment, the user terminal 200 and the service providing device 100 are shown separately, but the present invention is not limited thereto, and the user terminal 200 and the service providing device 100 may be integrated.

The corporate database 300 is a server that pre-stores advertising images of companies advertising their products through SNS platforms, etc., and provides a user interface that allows users accessing the server to search for advertising images of companies. to provide. In one embodiment, the corporate advertising image may include the company name, the company's service name, the name of the product sold by the company, the company's logo type, the company's logo color information, the company's vision, the company's slogan information, etc. . The service providing device 100 may be communicatively connected to a plurality of corporate databases 300.

The service providing device 100 may be a server that provides the user terminal 200 with a service that determines whether images used in advertisements comply with advertising guidelines.

First, the service providing device 100 receives a first image, which is an image of an advertisement exposed to the user, from the user terminal 200.

Additionally, the service providing device 100 may search for companies corresponding to the first image received from the user terminal 200 in the company database 300 and receive advertising images for each of the companies in the company database 300. there is.

The service providing device 100 may receive a second image containing color information, keyword information, and product information of the companies from each advertising image.

Additionally, the service providing device 100 may determine a first matching degree corresponding to each of the searched companies based on the first image and the second image. A detailed description of the first matching degree will be described later. Additionally, the service providing device may determine a company whose first matching degree is higher than a preset standard matching degree as the advertising subject company.

Additionally, the service providing device 100 may receive a third image of the advertising company from the company database 300. In one embodiment, the third image may be an image corresponding to the advertising guideline information of the advertising company.

In addition, the service providing device 100 determines a second matching degree corresponding to the advertising point included in the first image and the advertising point included in the third image, and the second matching degree among the advertisements exposed to the user is preset. An advertisement that matches the standard or higher can be determined as an appropriate advertisement that complies with the advertising guidelines. A detailed description of the second matching degree will be provided later.

FIG. 2 is a block diagram illustrating functional modules of the service providing device according to FIG. 1, and FIG. 3 is a flowchart showing an embodiment in which the advertising subject matching unit according to FIG. 2 matches an advertisement exposed to a user with a company. .

Referring to Figures 2 and 3, the service providing device 100 includes an advertisement feature extraction unit 101 and an advertisement subject matching unit 102.

The advertising feature extraction unit 101 may collect the first image, which is an advertising image exposed to the user, from the user terminal 200 (S110). In one embodiment, the first image may be the entire screen where advertisements that the user has encountered through various SNS platforms are displayed. However, it is not limited to this.

The advertisement feature extraction unit 101 may analyze the first image, which is an advertisement image exposed to the user. The first image may include a keyword, the color of the keyword, the product, and the color of the product.

The advertising feature extraction unit 101 may analyze the coordinates of keywords and products in the keyword area and product area of the first image. For example, the advertising feature extraction unit 101 may analyze the coordinates of a product outline in the product area of the first image and the coordinates of sentences, phrases, and words in the keyword area of the first image.

Additionally, the advertising feature extraction unit 101 searches for companies corresponding to the first image in the company database 300 and receives them from the company database 300 (S120).

The advertising feature extraction unit 101 may analyze advertising images of each company received from the corporate database 300. The advertising image may be a second image including the company's product, product color, keyword, and keyword color (S130).

The advertising image may be, for example, a product that the company wants to advertise, an image taken of the product that the company wants to advertise, or an image containing keywords that the company considers important. At this time, keywords considered important by the company may be the company's slogan or company's vision. The above-mentioned advertising image can be reflected in the second image that the company wants to expose through the SNS platform in order to achieve advertising effects.

The advertising subject matching unit 102 may determine a first matching degree corresponding to each of the searched companies based on the first image and the second image (S140).

FIG. 4 is a flowchart showing the specific process of step S140 of FIG. 3.

Referring to FIG. 4, the advertising feature extraction unit 101 may obtain a first color vector, a first keyword vector, and a first product vector from the first image (S141).

In one embodiment, the first image is an image provided in an advertisement that the user encounters through an SNS platform, and the image may include a product that appears in an advertisement that the user encounters through an SNS platform, and a keyword listed in an advertisement that the user encounters through an SNS platform. there is.

The advertising feature extraction unit 101 extracts a keyword, the color of the keyword, the product, and the color of the product from the first image, and generates a color vector from the extracted keyword, the color of the keyword, the product, and the color of the product. Keyword vectors and product vectors are input into a pre-trained first machine learning model to obtain them, and the first color vector, first keyword vector, and first product vector corresponding to the first image are obtained from the first machine learning model.

In one embodiment, when an image containing a keyword and a product is input as an input value, the first machine learning model outputs an object box for the keyword and product included in the image, and outputs an object box for the keyword and product included in the object box. When an image containing an image is entered as an input value, information about the keyword and product in the area occupied by the image of the keyword and product portion within the image is output. When an image of the keyword and product portion is input as an input value, the keyword And it may include a Mask-rcnn (Mask region convolutional neural network) model that has been machine learned to output keyword vectors and product vectors corresponding to the image of the product part. Additionally, the first machine learning model may include a separate artificial neural network model that is machine learned to output a color vector corresponding to the color included in the image when an image containing color is input as an input value.

In one embodiment, the preset color vector may be a vector representing characteristics of colors constituting keywords and products, and may specifically be a vector representing characteristics of color, saturation, and brightness of keywords and products.

In addition, in one embodiment, the preset keyword vector is expressed by extracting at least one of words, phrases, and sentences containing important words from the text written in the image using a known optical character reader (OCR) method. It can be a vector.

Additionally, in one embodiment, the preset product vector may be a vector representing characteristics of the shape, size, outline, etc. of the product included in the advertisement.

The obtained first color vector, first keyword vector, and first product vector are matched with the first image and stored in the database of the service providing device 100.

Additionally, the advertising feature extraction unit 101 may obtain a second color vector, a second keyword vector, and a second product vector from the second image (S142).

In one embodiment, the second image is an image for an advertisement provided by each company to the SNS platform from companies searched in the company database 300 corresponding to the first image information received from the user terminal 200. You can.

In one embodiment, the second image may be an image taken of a product that the company wants to advertise, or an image containing keywords that the company considers important. At this time, keywords considered important by the company may be sentences or words included in the company's slogan or company's vision. Specifically, at least one of words, phrases, and sentences among the text included in the second image can be read and determined as a keyword using a known optical character reader (OCR) method.

The advertising feature extraction unit 101 extracts keywords and products from the second image and inputs them into a pre-trained second machine learning model to obtain color vectors, product vectors, and keyword vectors from the extracted keywords and products, From the second machine learning model, the second color vector, second keyword vector, and second product vector corresponding to the second image are obtained.

In one embodiment, when an image containing a keyword and a product is input as an input value, the second machine learning model outputs an object box for the keyword and product included in the image, and outputs an object box for the keyword and product included in the object box. When an image containing an image is entered as an input value, information about the keyword and product in the area occupied by the image of the keyword and product portion within the image is output. When an image of the keyword and product portion is input as an input value, the keyword And it may include a Mask-rcnn (Mask region convolutional neural network) model that has been machine learned to output keyword vectors and product vectors corresponding to the image of the product part. Additionally, the second machine learning model may include a separate artificial neural network model that is machine learned to output a color vector corresponding to the color included in the image when an image containing color is input as an input value.

The acquired second color vector, second keyword vector, and second product vector are stored in the database of the service providing device 100.

Referring again to FIG. 4, the advertising subject matching unit 102 calculates the color similarity corresponding to each of the advertisement images based on the first color vector and the second color vector (S143).

The advertising subject matching unit 102 may calculate color similarity based on the cosine similarity between the first color vector and the second color vector.

Color similarity can be calculated based on Equation 1 below.

In Equation 1, C refers to the color similarity between the first color vector and the second color vector, C1 refers to the first color vector, and C2 refers to the second color vector. The smaller the angle between the first color vector and the second color vector, the relatively greater the color similarity, and the larger the angle between the first color vector and the second color vector, the relatively smaller the color similarity.

In one embodiment, the advertiser matching unit 102 may determine color similarity based on the number of colors, saturation, and brightness of the second image that match the color, saturation, and brightness of the first image. The advertising subject matching unit 102 may determine the color similarity to be higher as the number of colors, saturation, and brightness of the second image that match the color, saturation, and brightness of the first image increases. The advertising subject matching unit 102 may determine the color similarity to be lower as the number of colors, saturation, and brightness of the second image that match the color, saturation, and brightness of the first image decreases.

Additionally, the advertising subject matching unit 102 calculates keyword similarity corresponding to each of the advertising images based on the first keyword vector and the second keyword vector (S144).

The advertising subject matching unit 102 may calculate keyword similarity based on cosine similarity between the first keyword vector and the second keyword vector.

Keyword similarity can be calculated based on Equation 2 below.

In Equation 2, K refers to the keyword similarity between the first and second keyword vectors, K1 refers to the first keyword vector, and K2 refers to the second keyword vector. The smaller the angle between the first keyword vector and the second keyword vector, the relatively greater the keyword similarity, and the larger the angle between the first keyword vector and the second keyword vector, the relatively smaller the keyword similarity.

In one embodiment, the advertising entity matching unit 102 determines keyword similarity based on the number of sentences, words, and phrases included in the keywords of the second image that match the sentences, words, and phrases included in the keywords of the first image. You can decide. The advertising subject matching unit 102 may determine the keyword similarity to be higher as the number of sentences, words, and phrases included in the keywords of the second image that match the sentences, words, and phrases included in the keywords of the first image increases. The advertising subject matching unit 102 may determine the keyword similarity to be smaller as the number of sentences, words, and phrases included in the keywords of the second image that match the sentences, words, and phrases included in the keywords of the first image decreases. .

Additionally, the advertising subject matching unit 102 calculates the product similarity corresponding to each of the advertising images based on the first product vector and the second product vector (S145).

The advertising subject matching unit 102 may calculate product similarity based on the cosine similarity between the first product vector and the second product vector.

Product similarity can be calculated based on Equation 3 below.

In Equation 1, O means the color similarity between the first product vector and the second product vector, O1 means the first product vector, and O2 means the second product vector. The smaller the angle between the first product vector and the second product vector, the relatively greater the product similarity, and the larger the angle between the first product vector and the second product vector, the relatively smaller the product similarity.

In one embodiment, the advertising entity matching unit 102 may determine product similarity based on the number of products appearing in the second image that match the products appearing in the first image. The advertising subject matching unit 102 may determine the product similarity to be higher as the number of products appearing in the second image that match the products appearing in the first image increases. The advertising entity matching unit 102 may determine the product similarity to be lower as the number of products appearing in the second image that match the products appearing in the first image decreases.

Additionally, the advertising subject matching unit 102 determines a first matching degree corresponding to each of the companies stored in the company database 300 based on color similarity, keyword similarity, and product similarity (S146).

The first matching degree can be calculated based on Equation 4 below.

In Equation 4, M1 refers to the first matching degree, C refers to color similarity, K refers to keyword similarity, O refers to product similarity, R1 refers to the first correction coefficient, R2 is the second correction coefficient, and R3 is the third correction coefficient. It means correction coefficient.

The first correction coefficient, second correction coefficient, and third correction coefficient vary depending on the ranking of color similarity, keyword similarity, and product similarity values. The highest ranking similarity with the largest value corresponds to a correction coefficient of 0.9, The lowest ranking similarity with the smallest value corresponds to a correction coefficient of 0.3, and the correction coefficient of the similarity with the middle value among the color similarity, keyword similarity, and product similarity values corresponds to 0.6.

Referring again to FIG. 3, among the companies, a company whose first matching degree is higher than the preset standard matching degree is determined as the advertising company (S150). The determined advertising company information can be provided to the user terminal 200.

Through this, users can find out which companies display advertisements with images similar to those of advertisements they have encountered through SNS platforms. Therefore, users can easily determine which company's advertisement the advertisement they encountered through the SNS platform belongs to. In one embodiment, the reference matching degree may be the highest matching degree, but is not necessarily limited thereto.

For example, if the user identifies a blue color, the word galaxy or Samsung, or a product called a cell phone in the image in the advertisement displayed on the SNS platform, the company database 300 determines whether the product in the company's advertisement is an electronic product, especially an electronic product. If it is a cell phone, product similarity will be highest, if blue colors occupy most of the image area, color similarity will be high, and if keywords such as Samsung, galaxy S, galaxy fold, etc. are included, keyword similarity will be highest. Based on the calculated product similarity, keyword similarity, and color similarity, it can be determined through a machine learning model that the image of the advertisement exposed on the SNS platform corresponds to a mobile phone advertisement of "Samsung Electronics."

FIG. 5 is a flowchart illustrating an embodiment in which the advertisement reading unit according to FIG. 2 determines whether the advertisement exposed to the user is an appropriate advertisement that complies with the advertising guidelines (S200).

Referring to Figures 2 and 5, the service providing device 100 according to an embodiment of the present invention further includes an advertisement reading unit 103.

The advertisement reading unit 103 receives a third image, which is an image corresponding to the advertising guideline information of the advertising company, from the corporate database 300 (S210).

The advertisement reading unit 103 determines a second matching degree corresponding to the advertisement point included in the first image and the advertisement point included in the third image (S220).

The advertisement reading unit 103 may determine the second matching degree using the advertisement point included in the first image and the advertisement point included in the third image.

In one embodiment, the third image may be an advertising image that meets the company's advertising guidelines. In addition, in one embodiment, the advertising point is the ratio of the keyword in the image, the position where the keyword appears in the image, the ratio of the product in the image, the position where the product is displayed in the image, and when the advertisement is exposed in a mobile environment. This could be whether the image size is optimized for mobile or not.

The advertisement reading unit 103 overlaps or maps the first image to the third image to determine a match score and stores it in the database of the service providing device 100.

Specifically, the advertisement reading unit 103 calculates a second matching degree corresponding to the advertisement guide line based on the first advertisement point vector of the first image and the second advertisement point vector of the third image. At this time, the second matching degree can be calculated based on the cosine similarity between the first advertising point vector and the second advertising point vector.

The second matching degree can be calculated based on Equation 5 below.

In Equation 5, M2 refers to the second matching degree between the first and second advertising point vectors, A1 refers to the first advertising point vector, and A2 refers to the second advertising point vector. The smaller the angle formed by the first advertising point vector and the second advertising point vector, the relatively larger the second matching degree, and the larger the angle formed by the first advertising point vector and the second advertising point vector, the relatively smaller the second matching degree. Lose.

Meanwhile, the advertising point vector is a vectorized value of the pixel value according to the resolution of the image. For example, an image has a resolution of 3x3, 9 pixels, and each pixel has a value of 1 or 0. At this time, if the pixel value corresponding to the advertising guideline is set to 1 and the pixel value not corresponding to the advertising guideline is set to 0, the 3 It can be defined as 0, 0, 0, 0, 0, 0, 0, 0'.

At this time, the advertisement reading unit 103 extracts a pixel value from the first image, inputs it to a previously learned first machine learning model to obtain an advertisement point vector from the extracted pixel value, and extracts a pixel value from the first image. 1 Obtain the first advertising point vector corresponding to the image.

In addition, the advertisement reading unit 103 extracts pixel values from the third image, inputs them into a previously learned third machine learning model to obtain an advertisement point vector from the extracted pixel values, and obtains an advertisement point vector from the third machine learning model. 3 Obtain the second advertising point vector corresponding to the image.

That is, by determining whether the corresponding pixel value of the first image and the corresponding pixel value of the corresponding third image match using the cosine similarity between the first advertising point vector and the second advertising point vector, the advertising point matches or You can tell if it is similar or not.

The advertisement reading unit 103 determines an advertisement whose second matching degree is greater than or equal to a preset standard matching degree among advertisements exposed to the user as an appropriate advertisement that complies with the advertising guidelines (S230).

Users can save cost and time by using the machine learning model described above to determine whether advertisements displayed on SNS platforms comply with the company's advertising guidelines.

FIG. 6 is a diagram illustrating the hardware configuration of the service providing device 100 according to FIG. 1.

Referring to FIG. 6, the service providing device 100 stores at least one processor 110 and instructions instructing the at least one processor 110 to perform at least one operation. It may include memory.

The at least one operation may include the components 101 to 103 of the service providing device 100 described above or other functions or operation methods.

Here, the at least one processor 110 may mean a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which methods according to embodiments of the present invention are performed. You can. Each of the memory 120 and the storage device 160 may be comprised of at least one of a volatile storage medium and a non-volatile storage medium.

For example, the memory 120 may be one of read only memory (ROM) and random access memory (RAM), and the storage device 160 may be flash memory. , a hard disk drive (HDD), a solid state drive (SSD), or various memory cards (eg, micro SD card).

Additionally, the device 100 may include a transceiver 130 that performs communication through a wireless network. Additionally, the device 100 may further include an input interface device 140, an output interface device 150, a storage device 160, etc. Each component included in the device 100 is connected by a bus 170 and can communicate with each other.

Examples of the device 100 include a desktop computer, a laptop computer, a laptop, a smart phone, a tablet PC, and a mobile phone capable of communicating. , smart watch, smart glass, e-book reader, PMP (portable multimedia player), portable game console, navigation device, digital camera, DMB (digital multimedia broadcasting) player. , a digital audio recorder, a digital audio player, a digital video recorder, a digital video player, a PDA (Personal Digital Assistant), etc.

Methods according to the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. Computer-readable media may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on a computer-readable medium may be specially designed and constructed for the present invention or may be known and usable by those skilled in the computer software art.

Examples of computer-readable media may include hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions may include machine language code such as that created by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The above-described hardware device may be configured to operate with at least one software module to perform the operations of the present invention, and vice versa.

Additionally, the above-described method or device may be implemented by combining all or part of its components or functions, or may be implemented separately.

Although the present invention has been described above with reference to preferred embodiments, those skilled in the art may make various modifications and changes to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below. You will understand that you can do it.

Claims (1)

A service provision device that determines whether images used in advertisements comply with advertising guidelines,
An advertisement feature extraction unit that receives a first image, which is a screen image of an advertisement exposed to the user through an SNS platform, from a user terminal, and receives a second image, which is an advertisement image of companies corresponding to the first image;
An advertising entity that determines a first matching degree corresponding to each of the searched companies based on the first image and the second image, and determines a company whose first matching degree is greater than or equal to a preset standard matching degree as the advertising entity company. matching department; and
Receiving a third image, which is an image reflecting advertising guideline information of the advertising company, and determining an advertising point provided in the first image and a second matching degree corresponding to the advertising point provided in the third image, An advertisement reading unit that determines an advertisement whose second matching degree is higher than a preset standard matching degree as an appropriate advertisement that complies with advertising guidelines,
The advertising feature extraction unit,
Extract the first image from the keyword, the color of the keyword, the product, and the color of the product, and extract the keyword, the color of the keyword, the product, and the color of the product from the extracted first image. Input a first color vector, a first keyword vector, and a first product vector into a previously learned first machine learning model to obtain a first color vector, a first color vector, and a first product vector corresponding to the first image from the first machine learning model. Obtain keyword vector and first product vector,
From the second image, extract the company's product, company keyword, color of the company's product, and color of the company's keyword within the second image, and extract the extracted company's product, company's keyword, color of the company's product, and company. A second color vector, a second keyword vector, and a second product vector are input to a pre-trained second machine learning model to obtain a second color vector, a second keyword vector, and a second product vector from the colors of the keywords, and a second machine learning model corresponding to the second image is input from the second machine learning model. Obtain the color vector, second keyword vector, and second product vector,
The advertising entity matching unit,
Calculate color similarity based on cosine similarity between the first color vector and the second color vector,
Calculate keyword similarity based on cosine similarity between the first keyword vector and the second keyword vector,
Calculating product similarity based on cosine similarity between the first product vector and the second product vector,
The first matching degree is determined based on the following equation,

In the above equation, M1 means the first matching degree, C is the color similarity, K is the keyword similarity, O is the product similarity, R1 is the first correction coefficient, R2 is the second correction coefficient, and R3 is the third correction coefficient. It means correction coefficient,
The advertisement reading unit,
Extract pixel values from the first image, assign 1 as the pixel value corresponding to the advertising guidelines, assign 0 to pixel values that do not correspond to the advertising guidelines, and obtain an advertising point vector from the extracted pixel values. Entering the first machine learning model that has been previously trained to do so, and obtaining a first advertising point vector corresponding to the first image from the first machine learning model,
Extract the pixel value from the third image, assign 1 to the pixel value corresponding to the advertising guideline, assign 0 to the pixel value not corresponding to the advertising guideline, and obtain an advertising point vector from the extracted pixel value. Inputting a pre-trained third machine learning model to obtain a second advertising point vector corresponding to the third image from the third machine learning model,
Device.