KR102429392B1 - Method, device and system of providing product information using image crawling - Google Patents

Abstract

According to an embodiment, a method for providing product information using an image crawling technique, performed by a device, may comprise the steps of: acquiring a first image through crawling; encoding the first image to generate a first input signal; inputting the first input signal to a first artificial neural network and acquiring a first output signal based on a result of the input of the first artificial neural network; identifying a type of first product identified through the first image as a first category, based on the first output signal; performing a search for an image of a product registered in a posting space of the first category, and checking whether the image of the first product is registered; and when the image of the first product is not registered, generating first content providing product information on the first product based on the first image, and registering the first content in the posting space of the first category. According to the present invention, the product information can be efficiently managed.

Description

Method, device and system for providing product information using image crawling technique {METHOD, DEVICE AND SYSTEM OF PROVIDING PRODUCT INFORMATION USING IMAGE CRAWLING}

The following embodiments relate to a technology for providing product information by using an image crawling technique.

E-commerce refers to a transaction of buying and selling products or services that is performed on a network such as the Internet using a computer, that is, in a virtual space using an electronic medium (system).

With the development of electronic communication technology, electronic commerce has become more active, and it has become more common to sell products or advertise products by posting images or information related to products on the website of shopping malls through the Internet, and the development of related technologies has become more frequent. is being actively carried out.

However, in order to sell products using an online shopping mall, product sellers create product information (name of product, image of product, manufacturer, type of product, etc.) It has to be uploaded, but there is a problem that it takes a long time to work because you have to manually find information one by one and upload information according to the registration format.

Therefore, there is a need for research on a method of finding and automatically registering product-related information to increase efficiency.

Korean Patent Publication No. 10-2021-0130057 Korean Patent Publication No. 10-2021-0052237 Korean Patent No. 10-1932713 Korean Patent No. 10-2120465

According to an embodiment, a first image is obtained through crawling, a first input signal is generated by encoding the first image, and the first input signal is input to a first artificial neural network, and a first output signal is obtained. to obtain, based on the first output signal, identify the type of the first product identified through the first image as the first category, and perform a search for the image of the product registered in the posting space of the first category Thus, it is checked whether the image of the first product is registered, and when it is confirmed that the image of the first product is not registered, the first content providing product information about the first product based on the first image is displayed. An object of the present invention is to provide a method, apparatus and system for providing product information using an image crawling technique, which generates and registers the first content in the posting space of the first category.

Objects of the present invention are not limited to the objects mentioned above, and other objects not mentioned will be clearly understood from the description below.

According to an embodiment, there is provided a method for providing product information using an image crawling technique performed by an apparatus, the method comprising: acquiring a first image through crawling; encoding the first image to generate a first input signal; inputting the first input signal to a first artificial neural network, and obtaining a first output signal based on a result of the input of the first artificial neural network; identifying the type of the first product identified through the first image as a first category based on the first output signal; checking whether the image of the first product is registered by performing a search for an image of the product registered in the posting space of the first category; and when it is confirmed that the image of the first product is not registered, a first content providing product information on the first product is generated based on the first image, and the first content is displayed in the posting space of the first category. A method for providing product information using an image crawling technique is provided, including registering a first content.

The method for providing product information using the image crawling technique may include: when it is confirmed that a second image is registered as the image of the first product, checking first content generated based on the second image; performing an update on the first content by adding the first image to first content generated based on the second image; and when the first content is updated, changing the first content registered in the posting space of the first category to the updated first content.

The updating of the first content may include: setting an area occupied by the first product in the first image as a first area, and calculating the first number of pixels, which is the number of pixels in the first area; setting an area occupied by the first product in the second image as a second area, and calculating a second number of pixels, which is the number of pixels in the second area; assigning a pixel score of the first image based on the first number of pixels, and assigning a pixel score of the second image based on the second number of pixels; identifying the first product on the first image, detecting it as a first object, and calculating the number of first additional objects, which is the number of objects other than the first object; identifying the first product on the second image, detecting it as a second object, and calculating a second additional object number that is the number of objects other than the second object; assigning an object score of the first image based on the first number of additional objects, and giving an object score of the second image based on the second number of additional objects; calculating a first angle that is a photographing angle of the first image by analyzing a direction of the first object when the first object is detected on the first image; calculating a first angle that is a photographing angle of the second image by analyzing a direction of the second object when the second object is detected on the second image; giving a shooting score of the first image based on the first angle, and giving a shooting score of the second image based on the second angle; calculating a total score of the first image as a value obtained by adding the pixel score of the first image, the object score of the first image, and the shooting score of the first image; calculating a total score of the second image by adding the pixel score of the second image, the object score of the second image, and the shooting score of the second image; comparing the total score of the first image and the total score of the second image, and selecting an image having a higher total score as a representative image of the first content; performing an update on the first content so that when the first image is selected as the representative image of the first content, the first image and the second image are displayed in order; and when the second image is selected as the representative image of the first content, updating the first content to be displayed in the order of the second image and the first image.

The apparatus according to an embodiment may be controlled by a computer program stored in the medium to execute the method of any one of the above-described methods in combination with hardware.

According to an embodiment, a first image is obtained through crawling, a first input signal is generated by encoding the first image, and the first input signal is input to a first artificial neural network, and a first output signal is obtained. to obtain, based on the first output signal, identify the type of the first product identified through the first image as the first category, and perform a search for the image of the product registered in the posting space of the first category Thus, it is checked whether the image of the first product is registered, and when it is confirmed that the image of the first product is not registered, the first content providing product information about the first product based on the first image is displayed. By creating and registering the first content in the posting space of the first category, it is possible to automatically create and update content providing information on the product, thereby effectively managing the product information.

On the other hand, the effects according to the embodiments are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those of ordinary skill in the art from the following description.

1 is a diagram schematically showing the configuration of a system according to an embodiment.
2 is a flowchart illustrating a process of newly registering product information using an image crawling technique according to an embodiment.
3 is a flowchart illustrating a process of updating content according to an exemplary embodiment.
4 is a flowchart illustrating a process of calculating a total score of the first image by analyzing the first image according to an exemplary embodiment.
5 is a flowchart illustrating a process of calculating a total score of a second image by analyzing a second image according to an exemplary embodiment.
6 is a flowchart illustrating a process of selecting a representative image and performing content update according to an exemplary embodiment.
7 is a flowchart for explaining a process of setting a content exposure zone based on characteristics of an exposure screen according to an exemplary embodiment;
8 is a flowchart illustrating a process of setting an order of exposure zones according to an exemplary embodiment.
9 is a flowchart illustrating a process of setting a size of an exposure zone according to an exemplary embodiment.
10 is a diagram for explaining a first artificial neural network according to an embodiment.
11 is a diagram for explaining a method of learning a first artificial neural network according to an embodiment.
12 is an exemplary diagram of a configuration of an apparatus according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, since various changes may be made to the embodiments, the scope of the patent application is not limited or limited by these embodiments. It should be understood that all modifications, equivalents and substitutes for the embodiments are included in the scope of the rights.

Specific structural or functional descriptions of the embodiments are disclosed for purposes of illustration only, and may be changed and implemented in various forms. Accordingly, the embodiments are not limited to a specific disclosure form, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical spirit.

Although terms such as first or second may be used to describe various elements, these terms should be interpreted only for the purpose of distinguishing one element from another. For example, a first component may be termed a second component, and similarly, a second component may also be termed a first component.

When a component is referred to as being “connected to” another component, it may be directly connected or connected to the other component, but it should be understood that another component may exist in between.

Terms used in the examples are used for the purpose of description only, and should not be construed as limiting. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present specification, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that it does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiment belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

In addition, in the description with reference to the accompanying drawings, the same components are assigned the same reference numerals regardless of the reference numerals, and the overlapping description thereof will be omitted. In the description of the embodiment, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the embodiment, the detailed description thereof will be omitted.

The embodiments may be implemented in various types of products, such as personal computers, laptop computers, tablet computers, smart phones, televisions, smart home appliances, intelligent cars, kiosks, wearable devices, and the like.

1 is a diagram schematically showing the configuration of a system according to an embodiment.

Referring to FIG. 1 , a system according to an embodiment may include a plurality of user terminals 100 and devices 200 capable of communicating with each other through a communication network.

First, the communication network may be configured regardless of its communication mode, such as wired and wireless, and may be implemented in various forms so that communication between a server and a server and communication between a server and a terminal are performed.

The plurality of user terminals 100 are terminals used by users who want to receive product information, and include a first user terminal 110 used by a first user, a second user terminal 120 used by a second user, The third user terminal 130 used by the third user may be included.

Each of the plurality of user terminals 100 may be a desktop computer, a notebook computer, a tablet, a smart phone, or the like. For example, as shown in FIG. 1 , the first user terminal 110 , the second user terminal 120 , the third user terminal 130 , etc. may be smartphones, and may be employed differently depending on the embodiment. have.

Each of the plurality of user terminals 100 may be configured to perform all or part of an arithmetic function, a storage/referencing function, an input/output function, and a control function of a typical computer. The plurality of user terminals 100 may be configured to communicate with the device 200 by wire or wireless.

Each of the plurality of user terminals 100 is connected to a website operated by a person or organization that provides a service using the device 200 , or is developed and distributed by a person or organization that provides a service using the device 200 . One application can be installed. Each of the plurality of user terminals 100 may be linked with the device 200 through a website or an application.

Hereinafter, for convenience of description, the operation of the first user terminal 110 will be mainly described, but other user terminals such as the second user terminal 120 may perform the operation of the first user terminal 110 instead. to be.

The device 200 may be a self-server owned by a person or organization that provides services using the device 200, a cloud server, or a peer-to-peer (p2p) set of distributed nodes. may be The device 200 may be configured to perform all or part of an arithmetic function, a storage/referencing function, an input/output function, and a control function of a typical computer. The device 200 may include at least one artificial neural network that performs an inference function. The device 200 may be configured to communicate with a plurality of user terminals 100 through wired/wireless communication.

The device 200 may operate a site providing product information. At this time, since a posting space is provided for each category of product in the site, when a plurality of user terminals 100 access the posting space of the category, the device 200 exposes the content registered in the posting space of the category. , when the exposed content is selected, product information constituting the content may be displayed. Here, in order to provide product information, the content may consist of text, images, videos, etc., and may include the name of the product, the image of the product, the manufacturer, the type of product, the identification number of the product, detailed information of the product, etc. can

Meanwhile, for convenience of explanation, only the first user terminal 110 , the second user terminal 120 , and the third user terminal 130 are shown in FIG. 1 among the plurality of user terminals 100 , but the number of terminals is not It may vary depending on the example. As long as the processing capacity of the apparatus 200 allows, the number of terminals is not particularly limited.

2 is a flowchart illustrating a process of newly registering product information using an image crawling technique according to an embodiment.

Referring to FIG. 2 , first, in step S201 , the device 200 may acquire a first image through crawling.

Here, crawling is a technology that collects information published on the Internet and includes it as an index of a search target. The device 200 classifies the collected information into text, image, video, etc. By recognizing one product image, the product image may be acquired as the first image.

For example, the device 200 may search for product pages of products sold in an online shopping mall through crawling, and obtain a product image registered in the product page as the first image.

In step S202 , the device 200 may analyze the first image and confirm the first image as an image generated by photographing the first product.

Specifically, after recognizing the outline of the first product in the first image, the device 200 may recognize the first product by analyzing the outline of the first product, and when the first product is recognized in the first image , the first image may be identified as an image generated by photographing the first product.

In step S203 , the device 200 may generate a first input signal by encoding the first image.

Specifically, the device 200 may generate the first input signal by encoding the pixels of the first image with color information. The color information may include, but is not limited to, RGB color information, brightness information, and saturation information. The device may convert the color information into a numerical value, and may encode the first image in the form of a data sheet including the value.

In step S204 , the device 200 may input the first input signal to the first artificial neural network.

The first artificial neural network according to an embodiment is composed of a feature extraction neural network and a classification neural network, and the feature extraction neural network sequentially stacks a convolutional layer and a pooling layer on an input signal. The convolution layer includes a convolution operation, a convolution filter, and an activation function. The calculation of the convolution filter is adjusted according to the matrix size of the target input, but a 9X9 matrix is generally used. The activation function generally uses, but is not limited to, a ReLU function, a sigmoid function, and a tanh function. The pooling layer is a layer that reduces the size of the input matrix, and uses a method of extracting representative values by tying pixels in a specific area. In general, the average value or the maximum value is often used for the calculation of the pooling layer, but is not limited thereto. The operation is performed using a square matrix, usually a 9x9 matrix. The convolutional layer and the pooling layer are repeated alternately until the corresponding input becomes small enough while maintaining the difference.

According to one embodiment, the classification neural network has a hidden layer and an output layer. Classification of the first artificial neural network for product type detection The neural network consists of five or less hidden layers, and may include a total of 50 or less hidden layer nodes. The activation function of the hidden layer uses a ReLU function, a sigmoid function, and a tanh function, but is not limited thereto. There is a total of one output layer node of the classification neural network, and an output value for the type of product can be output to the output layer node. A detailed description of the first artificial neural network will be described later with reference to FIG. 10 .

In step S205 , the device 200 may obtain a first output signal based on a result of the input of the first artificial neural network.

In step S206 , the device 200 may identify the type of the first product as the first category based on the first output signal.

For example, as a result of checking the output value of the first output signal, when the output value is 1, the device 200 identifies the type of product as a clothing category, and when the output value is 2, identifies the type of product as a bag category, , when the output value is 3, the type of product may be identified as a shoe category, and when the output value is 4, the type of product may be identified as a jewelry category.

In step S207 , when the type of the first product is identified as the first category, the device 200 may search for an image of the product registered in the posting space of the first category.

Specifically, in the posting space of the first category, content for products classified into the first category is registered, and an image of the product is registered in the content. In this case, the device 200 may check each content registered in the posting space of the first category, and perform a search for an image of a product registered in each content.

In step S208, the device 200 may check whether the image of the first product is registered in the posting space of the first category as a result of performing a search for the image of the product registered in the posting space of the first category. have.

If it is confirmed in step S208 that the image of the first product is not registered in the posting space of the first category, in step S209, the device 200 provides product information about the first product based on the first image. 1 You can create content. Here, the first content is content that provides product information about the first product, and the first content has a first image registered as an image of the first product, and the first content includes the first product in addition to the image of the first product. It may further include the name of the product, the identification number of the first product, detailed information of the first product, and the like. To this end, the device 200 obtains the name of the first product, the identification number of the first product, and detailed information of the first product from the information collected through crawling, and then displays the first image of the first product. While registering as an image, by further registering the name of the first product, the identification number of the first product, and detailed information of the first product, the image of the first product, the name of the first product, the identification number of the first product, the first It is possible to create the first content including the detailed information of the product.

In step S210 , the device 200 may register the first content in the posting space of the first category.

That is, when the first image is identified as the image of the first product, and the type of the first product is identified as the first category through the first image, the device 200 generates the first content based on the first image. Thereafter, the first content may be uploaded to the posting space of the first category by registering the first content in the posting space of the first category.

Meanwhile, if it is confirmed in step S208 that the image of the first product is registered in the posting space of the first category, step S301 may be performed, and a detailed description related thereto will be described with reference to FIG. 3 .

3 is a flowchart illustrating a process of updating content according to an exemplary embodiment.

Referring to FIG. 3 , first, in step S301 , the device 200 determines that the image of the first product is registered in the posting space of the first category, and the second image is registered as the image of the first product. can be confirmed as

That is, when the second image is already registered as the image of the first product in the posting space of the first category, the device 200 converts the image of the first product registered in the posting space of the first category to the second image. can be checked At this time, when the first image is confirmed as the image of the first product, the device 200 compares the first image with the image registered in the posting space of the first category, and extracts a second image that matches a preset ratio or more and the extracted second image may be identified as the image of the first product.

In step S302 , the device 200 may check the first content generated based on the second image and registered in the posting space of the first category.

Specifically, when the second image is identified as the image of the first product, the device 200 may identify the first content including the second image. Here, the first content is content that provides product information on the first product, and the first content has a second image registered as an image of the first product, and the first content includes, in addition to the second image, information of the first product. It may further include a name, an identification number of the first product, detailed information of the first product, and the like.

In step S303 , the device 200 may update the first content by adding the first image to the first content. That is, the device 200 may update the first content by adding the first image to the first content in which the second image is previously registered as the image of the first product. Through this, the first content may be changed from a state in which only the second image is registered as the image of the first product to a state in which the first image and the second image are registered as the image of the first product.

In step S304 , when the first content is updated, the device 200 may change the first content registered in the posting space of the first category to the updated first content. At this time, the first content changed by the update is a state in which the first image and the second image are registered together as the image of the first product, and the device 200 registers a plurality of images in the first content as the image of the first product Therefore, any one of the first image and the second image may be selected as the representative image of the first content. A detailed description related thereto will be described later with reference to FIGS. 4 to 6 .

4 is a flowchart illustrating a process of calculating a total score of the first image by analyzing the first image according to an exemplary embodiment.

Referring to FIG. 4 , first, in step S401 , the device 200 may identify a first product on a first image.

Specifically, after recognizing the outline of the first product on the first image, the device 200 may perform an operation of separating the first product and the background based on the recognized outline of the first product, through this, A first product may be identified on the first image.

In step S402 , the device 200 may set the area occupied by the first product in the first image as the first area. That is, the device 200 may separate the first product and the background on the first image, and set the area occupied by the separated first product as the first area.

In step S403 , the device 200 may detect the first product identified on the first image as the first object. That is, the device 200 may separate the first product and the background on the first image, and detect the separated first product as the first object.

In operation S404 , when the first area is set in the first image, the device 200 may calculate the first number of pixels, which is the number of pixels in the first area.

In step S405 , when the first object is detected on the first image, the device 200 may detect whether there are other objects other than the first object on the first image, and confirm the remaining objects except for the first object, The number of first additional objects that is the number of objects other than 1 object may be calculated.

In operation S406 , when the first object is detected on the first image, the device 200 may analyze the direction of the first object to calculate a first angle that is a photographing angle of the first image. In this case, the device 200 may analyze which direction the first object is facing with respect to the front, and may calculate the first angle through the direction in which the first object is facing.

For example, as a result of analyzing the direction of the first object, when the direction of the first object is the front, the device 200 calculates the first angle of 0 degrees, and the direction of the first object is shifted by 30 degrees to the left. In this case, the first angle may be calculated as -30 degrees, and when the direction of the first object is shifted by 30 degrees to the right, the first angle may be calculated as 30 degrees.

In addition, as a result of analyzing the direction of the first object, when the direction of the first object is raised by 30 degrees, the device 200 calculates the first angle of 30 degrees, and the direction of the first object is 30 degrees downward. If it is lowered, the first angle can be calculated as 30 degrees.

In operation S407 , when the first number of pixels is calculated, the device 200 may assign a pixel score of the first image based on the first number of pixels. In this case, the device 200 may assign the pixel score of the first image to a higher score as the first pixel number is higher. For example, the device 200 may assign a pixel score of the first image to 4 when the first number of pixels is 2.1 million pixels, and when the first number of pixels is 8.3 million pixels, the pixel score of the first image can be awarded as 8 points.

In step S408 , when the first number of additional objects is calculated, the device 200 may give an object score of the first image based on the first number of additional objects. In this case, the apparatus 200 may give the object score of the first image as a higher score as the number of first additional objects decreases. For example, when the first number of additional objects is 4, the device 200 may assign an object score of the first image to 4, and when the number of first additional objects is 1, the device 200 may assign an object score of the first image to 7 points can be given.

In step S409 , when the first angle is calculated, the device 200 may give a shooting score of the first image based on the first angle. In this case, as the first angle is closer to 0 degrees, the device 200 may assign a higher score to the shooting score of the first image. For example, when the first angle is 45 degrees, the device 200 may give the shooting score of the first image as 5 points, and when the first angle is 0 degrees, the device 200 may give the shooting score of the first image as 10 points. can be given

According to an embodiment, the highest score of each of the pixel score, the object score, and the shooting score may all be set to the same score, but the highest score may be set differently according to user settings. For example, the highest score of the pixel score, the object score, and the shooting score may all be set equal to 10 points, the highest score of the pixel score is 15 points, the highest score of the object score is 10 points, and the highest score of the shooting score is 10 points. may be set differently by 5 points.

In step S410 , the device 200 is assigned a pixel score of the first image based on the first number of pixels, an object score of the first image is assigned based on the first additional object number, and based on the first angle When the shooting score of the first image is given, a total score of the first image may be calculated as a value obtained by adding the pixel score of the first image, the object score of the first image, and the shooting score of the first image.

For example, when 8 points are given as the pixel score of the first image, 4 points are given as the object score of the first image, and 10 points are given as the shooting score of the first image, the device 200 is The total score can be calculated as 22 points.

5 is a flowchart illustrating a process of calculating a total score of a second image by analyzing a second image according to an exemplary embodiment.

Referring to FIG. 5 , first, in step S501 , the device 200 may identify a first product on a second image.

Specifically, after recognizing the outline of the first product on the second image, the device 200 may perform an operation of separating the first product and the background based on the recognized outline of the first product, through this, The first product may be identified on the second image.

In step S502 , the device 200 may set the area occupied by the first product in the second image as the second area. That is, the device 200 may separate the first product and the background on the second image, and set the area occupied by the separated first product as the second area.

In step S503, the device 200 may detect the first product identified on the second image as the second object. That is, the device 200 may separate the first product and the background on the second image, and detect the separated first product as the second object.

In operation S504 , when the second area is set in the second image, the device 200 may calculate the second number of pixels, which is the number of pixels in the second area.

In step S505, when the second object is detected on the second image, the device 200 detects whether there are other objects other than the second object on the second image, and can check the remaining objects except for the second object, The number of second additional objects that is the number of objects other than the second object may be calculated.

In step S506 , when the second object is detected on the second image, the device 200 may analyze the direction of the second object to calculate a second angle that is a photographing angle of the second image. In this case, the apparatus 200 may analyze which direction the second object is facing with respect to the front, and may calculate the second angle through the direction in which the second object is facing.

In step S507 , when the second number of pixels is calculated, the device 200 may assign a pixel score of the second image based on the second number of pixels. In this case, in the same manner as in the method of assigning the pixel score of the first image, the device 200 may assign the pixel score of the second image as a higher score as the second pixel number is higher.

In step S508 , when the second number of additional objects is calculated, the device 200 may give an object score of the second image based on the second number of additional objects. In this case, the apparatus 200 may give the object score of the second image as a higher score as the number of second additional objects decreases, in the same manner as in the method of assigning the object score of the first image.

In step S509 , when the second angle is calculated, the device 200 may give a shooting score of the second image based on the second angle. In this case, in the same manner as in the method of assigning the shooting score of the first image, the device 200 may give the shooting score of the second image as a higher score as the second angle is closer to 0 degrees.

In step S510 , the device 200 assigns a pixel score of the second image based on the second number of pixels, assigns an object score of the second image based on the second number of additional objects, and assigns an object score of the second image based on the second angle. When the shooting score of the second image is given, a total score of the second image may be calculated as a value obtained by adding the pixel score of the second image, the object score of the second image, and the shooting score of the second image.

6 is a flowchart illustrating a process of selecting a representative image and performing content update according to an exemplary embodiment.

Referring to FIG. 6 , first, in step S601 , the device 200 calculates the total score of the first image as a value obtained by adding the pixel score of the first image, the object score of the first image, and the shooting score of the first image. can do.

In step S602 , the device 200 may calculate a total score of the second image as a value obtained by adding the pixel score of the second image, the object score of the second image, and the shooting score of the second image.

In step S603 , the device 200 may compare the total score of the first image and the total score of the second image, and select an image having a higher total score as the representative image of the first content.

For example, when the total score of the first image is 22 and the total score of the second image is 15, the device 200 may select the first image as a representative image of the first content, and vice versa, the first image When the total score of is 15 points and the total score of the second image is 22 points, the device 200 may select the second image as the representative image of the first content.

In step S604 , the device 200 may check whether the first image is selected as the representative image.

If it is confirmed in step S604 that the first image is selected as the representative image, in step S605 , the device 200 may update the first content so that the first image and the second image are displayed in order. That is, the device 200 sets the exposure order of the first content so that when the first content is displayed, the first image, which is a representative image, is displayed first, and the second image is displayed after the first image, 1 You can update the content.

If it is confirmed in step S604 that the second image other than the first image is selected as the representative image, in step S606, the device 200 updates the first content so that the second image and the first image are displayed in order. can do. That is, the device 200 sets the exposure order of the first content so that when the first content is displayed, the second image, which is the representative image, is displayed first, and the first image is displayed after the second image, 1 You can update the content.

7 is a flowchart for explaining a process of setting a content exposure zone based on characteristics of an exposure screen according to an exemplary embodiment;

Referring to FIG. 7 , first, in step S701 , the device 200 may calculate the number of views of the first content by checking the number of times the inquiry request for the first content is received from the plurality of user terminals 100 . have.

Specifically, when a request for access to the posting space of the first category is received from the plurality of user terminals 100, the device 200 processes the plurality of user terminals 100 to access the posting space of the first category, When the plurality of user terminals 100 access the posting space of the first category, a list of contents registered in the first category may be controlled to be displayed on each screen of the plurality of user terminals 100 .

Then, when the first content is selected from the content list of the first category displayed on the screen of each of the plurality of user terminals 100 , the device 200 receives an inquiry request for the first content from the plurality of user terminals 100 . can do. Here, the first content is content that provides product information on the first product, and the inquiry request for the first content may be a request for inquiring detailed information about the first product.

When an inquiry request for the first content is received from the plurality of user terminals 100 , the device 200 may update inquiry information of the first content according to the inquiry request for the first content. Here, the inquiry information of the first content may include information on the number of views of the first content, which is the number of times that the first content is viewed. The device 200 may store and manage inquiry information for each content in a database.

In updating the inquiry information of the first content, the device 200 may calculate the number of views of the first content by checking the number of times the inquiry request for the first content is received from the plurality of user terminals 100 . .

For example, in the device 200, in a state in which the number of views of the first content is 0, the inquiry request for the first content is received from the first user terminal 110 and the second user terminal 120 within a preset period, respectively. , the number of times the inquiry request for the first content is received may be checked twice, and the number of inquiry of the first content may be changed from 0 to 2 and calculated.

In addition, when the inquiry request for the first content is received from the first user terminal 110 in a state where the number of views of the first content is 0, the device 200 changes the number of views of the first content from 0 to 1. In a state where the number of views of the first content is 1, when a query request for the first content is received from the second user terminal 120, the number of views of the first content is changed from 1 to 2 and calculated. can

The device 200 may calculate the number of views of each of the plurality of contents in the same manner as in the method of calculating the number of views of the first content, and calculate the number of views of the second content and the number of views of the third content, respectively. can

In step S702 , the device 200 may calculate the number of views of the second content by checking the number of times the inquiry request for the second content is received from the plurality of user terminals 100 . Here, the second content is content that provides product information on the second product, and the inquiry request for the second content may be a request for inquiring detailed information on the second product.

In step S703 , the device 200 may receive a request for access to the posting space of the first category from the first user terminal 110 .

In step S704 , the device 200 may check the screen size of the first user terminal 110 based on the identification information of the first user terminal 110 . Here, the identification information of the first user terminal 110 is information necessary to identify the first user terminal 110, and may include information for identifying the type and model of the terminal, such as a smart phone, a tablet, and a PC.

Specifically, when a request for access to the posting space of the first category is received from the first user terminal 110 , the device 200 obtains identification information of the first user terminal 110 from the first user terminal 110 . Also, based on the identification information of the first user terminal 110 , the type of the first user terminal 110 may be identified, and the screen size of the first user terminal 110 may be checked.

In step S705 , the device 200 may determine whether the screen size of the first user terminal 110 is smaller than a reference value. Here, the reference value may be set differently according to embodiments.

If it is confirmed in step S705 that the screen size of the first user terminal 110 is smaller than the reference value, in step S706 , the device 200 provides the first web page composed of the first zone to the first user terminal 110 . can Here, the first zone is a zone in which a representative image of the first content is displayed.

If it is determined in step S705 that the screen size of the first user terminal 110 is larger than the reference value, in step S707 , the device 200 displays a second web page comprising the first zone and the second zone to the first user terminal 110 . ) can be provided. Here, the first zone is a zone in which a representative image of the first content is displayed, and the second zone is a zone in which a representative image of the second content is displayed. In this case, the first content and the second content may be set as the most recently created and registered content among the plurality of content. For example, the device 200 may use the most recently created and registered content as the first content. set, and the registered content generated immediately before the registration of the first content may be set as the second content.

8 is a flowchart illustrating a process of setting an order of exposure zones according to an exemplary embodiment.

Referring to FIG. 8 , if it is confirmed that the screen size of the first user terminal 110 is smaller than the reference value in step S705 , in step S801 , the device 200 determines that the number of views of the first content is greater than the number of views of the second content. You can check whether there are many.

If it is determined in step S801 that the number of views of the first content is greater than the number of views of the second content, in step S802 , the device 200 provides the first web page composed of one zone to the first user terminal 110 . In this case, it is possible to control so that the first zone is displayed first on the screen of the first user terminal 110 and then the second zone is displayed subsequently.

The device 200 may control so that the first zone is displayed first on the screen of the first user terminal 110, and then, when a screen movement is requested through a user operation through clicking and swiping, the second zone is displayed subsequently, After the first zone is first displayed on the screen of the first user terminal 110 , when a preset time elapses, it is also possible to control the second zone to be displayed continuously.

If it is determined in step S801 that the number of views of the first content is less than the number of views of the second content, in step S803, the device 200 provides the first web page composed of one zone to the first user terminal 110 . In this case, it is possible to control so that the second zone is first displayed on the screen of the first user terminal 110 and then the first zone is displayed subsequently.

After the second zone is displayed on the screen of the first user terminal 110, the device 200 may control the first zone to be displayed subsequently when a screen movement is requested by a user operation through clicking and swiping, After the second zone is first displayed on the screen of the first user terminal 110 , when a preset time elapses, the control may be performed so that the first zone is displayed continuously.

9 is a flowchart illustrating a process of setting a size of an exposure zone according to an exemplary embodiment.

Referring to FIG. 9 , if it is determined that the screen size of the first user terminal 110 is larger than the reference value in step S705 , in step S901 , the device 200 determines the number of views of the first content and the number of views of the second content With the summed value, the total number of hits can be calculated.

For example, when the number of views of the first content is 75 and the number of views of the second content is 25, the device 200 may calculate the total number of views as 100.

In step S902 , the device 200 may calculate the first ratio as a value obtained by dividing the number of views of the first content by the total number of views.

For example, when the number of views of the first content is 75 and the total number of views is 100, the device 200 may calculate the first ratio as 75%.

In step S903 , the device 200 may calculate the second ratio as a value obtained by dividing the number of views of the second content by the total number of views.

For example, when the number of views of the second content is 25 and the total number of views is 100, the device 200 may calculate the second ratio as 25%.

In step S904 , the device 200 may set the size of the first region at the first ratio.

In step S905 , the device 200 may set the size of the second zone by the second ratio.

In step S906, when the size of the first zone is set through the first ratio and the size of the second zone is set through the second ratio, the device 200 displays a second web page composed of the first zone and the second zone It is provided to the first user terminal 110 to control the operation of the first user terminal 110 so that the representative image of the first content is displayed in the first area and the representative image of the second content is displayed in the second area. can That is, the second web page is set so that the representative image of the first content and the representative image of the second content are divided and displayed through the first region and the second region. Here, the representative image of the first content is an image having a higher total score among the first image and the second image selected in step S603. In addition, the representative image of the first content may include a link leading to detailed information on the first product, and the representative image of the second content may include a link leading to the detailed information on the second product.

For example, when a representative image of the first content displayed in the first zone is selected while the second web page composed of the first zone and the second zone is being displayed on the screen of the first user terminal 110, the second 1 When a web page for the first content on which detailed information of the product is displayed is connected, and a representative image of the second content displayed in the second zone is selected, a web page for the second content on which detailed information of the second product is displayed can be connected to

10 is a diagram for explaining a first artificial neural network according to an embodiment.

The first artificial neural network 1000 according to an embodiment may input a first input signal generated by encoding a first image, and output information indicating the type of the first product.

Encoding according to an embodiment may be performed by storing the color information for each pixel of the image in the form of a digitized data sheet, and the color information may include RGB color, brightness information, and saturation information of one pixel. , but not limited thereto.

According to an embodiment, the first artificial neural network 1000 is composed of a feature extraction neural network 1010 and a classification neural network 1020, and the feature extraction neural network 1010 performs a task of separating a product and a background from an image. Also, the classification neural network 1020 may perform a task of identifying which category the product in the image belongs to.

The method in which the feature extraction neural network 1010 distinguishes the product from the background is that the change in each value of color information from the data sheet of the first input signal encoding the image is 30% in 6 or more of 8 pixels including one pixel A bundle of pixels detected as having an abnormal change may be used as the boundary between the product and the background, but is not limited thereto.

The feature extraction neural network 1010 proceeds by sequentially stacking a convolutional layer and a pooling layer on the input signal. The convolution layer includes a convolution operation, a convolution filter, and an activation function. The calculation of the convolution filter is adjusted according to the matrix size of the target input, but a 9X9 matrix is generally used. The activation function generally uses, but is not limited to, a ReLU function, a sigmoid function, and a tanh function. The pooling layer is a layer that reduces the size of the input matrix, and uses a method of extracting representative values by tying pixels in a specific area. In general, the average value or the maximum value is often used for the calculation of the pooling layer, but is not limited thereto. The operation is performed using a square matrix, usually a 9x9 matrix. The convolutional layer and the pooling layer are repeated alternately until the corresponding input becomes small enough while maintaining the difference.

The classification neural network 1020 checks the shape of the product separated from the background through the feature extraction neural network 1010, and confirms whether the product in the image belongs to which category by checking whether it is similar to the representative product for each predefined category can figure out Information stored in the database of the device 200 may be utilized for product comparison.

The classification neural network 1020 prioritizes the task of identifying the category to which the product belongs, and may also grasp the shape and size of the product.

The classification neural network 1020 has a hidden layer and an output layer, and is composed of five or less hidden layers, including a total of 50 or less hidden layer nodes, and the activation function of the hidden layer is a ReLU function and a sigmoid function. and tanh functions, but is not limited thereto.

The classification neural network 1020 may include only one output layer node in total.

The output of the classification neural network 1020 is an output value for the type of product, and may indicate to which category the type of product belongs. For example, when the output value is 1, it indicates that the type of product is a clothing category, when the output value is 2, it indicates that the type of product is a bag category, and when the output value is 3, the type of product is a footwear category , and when the output value is 4, it may indicate that the type of product is a jewelry category.

According to an embodiment, the first artificial neural network 1000 may learn by receiving the first learning signal generated by the corrected correct answer input by the user when the user discovers a problem with the output according to the first artificial neural network 1000 . can The problem of output according to the first artificial neural network 1000 may mean a case in which output values indicating different categories of products are output.

The first learning signal according to an embodiment is created based on the error between the correct answer and the output value, and in some cases, SGD using delta, a batch method, or a method following a backpropagation algorithm may be used. The first artificial neural network 1000 performs learning by modifying the existing weights according to the first learning signal, and may use momentum in some cases. A cost function can be used to calculate the error, and a cross entropy function can be used as the cost function. Hereinafter, the learning contents of the first artificial neural network 1000 will be described with reference to FIG. 11 .

11 is a diagram for explaining a method of learning a first artificial neural network according to an embodiment.

According to an embodiment, the learning apparatus may train the first artificial neural network 1000 . The learning device may be a separate entity other than the device, but is not limited thereto.

According to an embodiment, the first artificial neural network 1000 includes an input layer to which training samples are input and an output layer for outputting training outputs, and is located between the first training outputs 1103 and the first labels 1104 . can be learned based on the difference in Here, the first labels 1104 may be defined based on a representative image for each type of product. The first artificial neural network 1000 is connected as a group of a plurality of nodes, and is defined by weights between the connected nodes and an activation function that activates the nodes.

The learning apparatus may train the first artificial neural network 1000 using a GD (Gradient Decent) technique or a Stochastic Gradient Descent (SGD) technique. The learning apparatus may use a loss function designed by the outputs and labels of the first artificial neural network 1000 .

The learning apparatus may calculate a training error using a predefined loss function. The loss function may be predefined with a label, an output, and a parameter as input variables, where the parameter may be set by weights in the first artificial neural network 1000 . For example, the loss function may be designed in a Mean Square Error (MSE) form, an entropy form, or the like, and various techniques or methods may be employed in an embodiment in which the loss function is designed.

The learning apparatus may find weights affecting the training error by using a backpropagation technique. Here, the weights are relationships between nodes in the first artificial neural network 1000 . The learning apparatus may use the SGD technique using labels and outputs to optimize the weights found through the backpropagation technique. For example, the learning apparatus may update the weights of the loss function defined based on the labels, outputs, and weights using the SGD technique.

According to an embodiment, the learning apparatus may acquire the labeled training product images 1101 from the database. The learning apparatus may obtain pre-labeled information on each of the product images 1101 set for each type of product, and the product images 1101 may be labeled according to the pre-classified product information.

According to an embodiment, the learning device may acquire 1000 labeled training product images 1101, and based on the labeled training product images 1101 to generate the first training product vectors 1102. can Various methods may be employed for extracting the first training product vectors 1102 .

According to an embodiment, the learning apparatus may obtain the first training outputs 1103 by applying the first training product vectors 1102 to the first artificial neural network 1000 . The learning apparatus may train the first artificial neural network 1000 based on the first training outputs 1103 and the first labels 1104 . The learning apparatus calculates training errors corresponding to the first training outputs 1103 , and optimizes the connection relationship of nodes in the first artificial neural network 1000 to minimize the training errors to form the first artificial neural network 1000 . can learn The device may identify the type of product in the image by using the first artificial neural network 1000 on which the learning has been completed.

12 is an exemplary diagram of a configuration of an apparatus according to an embodiment.

The device 200 according to an embodiment includes a processor 210 and a memory 220 . The processor 210 may include at least one of the devices described above with reference to FIGS. 1 to 11 , or perform at least one method described above with reference to FIGS. 1 to 11 . An individual or entity using the device 200 may provide a service related to some or all of the methods described above with reference to FIGS. 1 to 11 .

The memory 220 may store information related to the above-described methods or a program in which methods to be described below are implemented. The memory 220 may be a volatile memory or a non-volatile memory.

The processor 210 may execute a program and control the device 200 . The code of the program executed by the processor 210 may be stored in the memory 220 . The device 200 may be connected to an external device (eg, a personal computer or a network) through an input/output device (not shown), and may exchange data through wired/wireless communication.

The device 200 may be used to train an artificial neural network or to use a learned artificial neural network. The memory 220 may include a learning or learned artificial neural network. The processor 210 may learn or execute an artificial neural network algorithm stored in the memory 220 . The apparatus 200 for learning an artificial neural network and the apparatus 200 for using the learned artificial neural network may be the same or may be separate.

The embodiments described above may be implemented by a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the apparatus, methods and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA). array), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that may include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Software may comprise a computer program, code, instructions, or a combination of one or more of these, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

As described above, although the embodiments have been described with reference to the limited drawings, those skilled in the art may apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (3)

In the product information providing method using the image crawling technique, performed by the device,
acquiring a first image through crawling;
encoding the first image to generate a first input signal;
inputting the first input signal to a first artificial neural network, and obtaining a first output signal based on a result of the input of the first artificial neural network;
identifying the type of the first product identified through the first image as a first category based on the first output signal;
checking whether the image of the first product is registered by performing a search for an image of the product registered in the posting space of the first category;
If it is confirmed that the image of the first product is not registered, a first content providing product information about the first product is generated based on the first image, and the first content is created in the posting space of the first category. 1 step of registering content;
when it is confirmed that a second image is registered as the image of the first product, checking first content generated based on the second image;
performing an update on the first content by adding the first image to first content generated based on the second image; and
When the first content is updated, changing the first content registered in the posting space of the first category to the updated first content,
The step of updating the first content includes:
setting an area occupied by the first product in the first image as a first area, and calculating a first number of pixels, which is the number of pixels in the first area;
setting an area occupied by the first product in the second image as a second area, and calculating a second number of pixels, which is the number of pixels in the second area;
assigning a pixel score of the first image based on the first number of pixels, and assigning a pixel score of the second image based on the second number of pixels;
identifying the first product on the first image, detecting it as a first object, and calculating the number of first additional objects, which is the number of objects other than the first object;
identifying the first product on the second image, detecting it as a second object, and calculating a second additional object number that is the number of objects other than the second object;
assigning an object score of the first image based on the first number of additional objects, and giving an object score of the second image based on the second number of additional objects;
calculating a first angle that is a photographing angle of the first image by analyzing a direction of the first object when the first object is detected on the first image;
calculating a second angle that is a photographing angle of the second image by analyzing a direction of the second object when the second object is detected on the second image;
giving a shooting score of the first image based on the first angle, and giving a shooting score of the second image based on the second angle;
calculating a total score of the first image as a value obtained by adding the pixel score of the first image, the object score of the first image, and the shooting score of the first image;
calculating a total score of the second image by adding the pixel score of the second image, the object score of the second image, and the shooting score of the second image;
comparing the total score of the first image and the total score of the second image, and selecting an image having a higher total score as a representative image of the first content;
performing an update on the first content so that when the first image is selected as the representative image of the first content, the first image and the second image are displayed in order; and
When the second image is selected as the representative image of the first content, performing an update on the first content so that the second image and the first image are displayed in the order,
A method of providing product information using image crawling techniques. delete delete

Images