KR20210120203A - Method for generating metadata based on web page - Google Patents

Abstract

According to an embodiment of the present invention, a method for generating metadata based on a webpage, which is generated by a metadata generating application executed in a processor of a computing device, comprises: a step of extracting text included in the webpage; a step of extracting text based on letters in an image included in the webpage; a step of performing natural language processing for the extracted text to extract part of the extracted text as valid text; a step of calculating importance of the extracted valid text and determining part of the extracted valid text as a core keyword according to the calculated importance; and a step of generating the metadata based on the determined core keyword.

Description

How to generate metadata based on a web page

The present invention relates to a method of generating metadata based on a web page. More particularly, it relates to a method of generating metadata based on text and images in a web page.

Crawling, which collects data required for search on a web site, is one of the major issues. Crawling refers to a technology that collects texts provided by each page of countless web sites and includes them as an index for search targets.

When crawling a web page, if the entire content of the page is saved as the main body, duplicate menus, advertisements, unnecessary tags, etc. are stored together, and unnecessary search results may be provided when searching, so the utility as search data is low.

Therefore, a filtering algorithm has been developed to automatically extract only the body content of a page, but there is still a lack of any one filtering alone, and to solve this problem, various filters are used in combination.

Nevertheless, when a search term is entered and searched, a search result based on crawled data is provided in a part unrelated to the search target, so that satisfactory search efficiency cannot be obtained.

In addition, the conventional algorithm provides a management system that allows web site administrators to access tag attributes of multiple pages and edit metadata, which is data required for search, but the system allows web site administrators to access SEO. It is only useful when specifying appropriate metadata for (search engine optimization), but there is the inconvenience of having to write metadata one by one and edit it every time the page is updated.

In addition, for SEO, it is important to actually create metadata related to a web page, but if not only text but also images exist in a web page, a system that derives meaningful keywords by considering all the characters in the image is insufficient. am.

Therefore, there is a need to develop a technology capable of extracting all texts in an image of a web page and generating metadata based on all texts in the extracted web page.

Meanwhile, as interest in crawling has increased, a TextRank algorithm for deriving and summarizing key keywords of web page documents has been developed.

Here, the TextRank algorithm is an algorithm that performs a function of deriving and summarizing key keywords from a web page.

In detail, the TextRank algorithm builds a word graph or a sentence graph, and then uses PageRank, a graph ranking algorithm, to rank each keyword and/or key sentence. You can choose. And the TextRank algorithm may summarize the text set of a given web page using the selected keywords and/or key sentences.

In this case, PageRank, an algorithm proposed by Brin and Page (1998), is a method of assigning weights to web documents having hyperlinks according to their relative importance. A web page with a high PageRank can be interpreted as receiving a lot of links from other websites, that is, a lot of references from other websites.

It is TextRank that utilizes such a PageRank algorithm, and TextRank uses words and/or sentences in a web page document to determine the importance of a website with a high 　 PageRank 　 importance based on the fact that it receives links from many other sites. It is an algorithm that calculates the following ranking.

On the other hand, with the development of information and communication technology and increasing interest in image analysis, a Google cloud vision API model capable of detecting characters in an image has been developed.

Specifically, Vision API Models provides powerful machine learning models pre-trained via REST and RPC APIs, allowing you to label images and quickly classify them into millions of predefined categories.

Such a vision API model can detect an object in an image, printed or handwritten text, and the like, and extract useful data from it to assist the image-based metadata creation process.

KR 10-2017-0094829 A

The present invention has been devised to solve the above-described problems, and by performing crawling based on any one or more of text of a web page, text in images, and alternative text for images, properties of the web page An object of the present invention is to provide a web page-based metadata generation method that generates metadata that describes

Another object of the present invention is to provide a web page-based metadata generation method for generating metadata for a corresponding web page based on attributes of text crawled from the web page.

However, the technical problems to be achieved by the present invention and embodiments of the present invention are not limited to the technical problems as described above, and other technical problems may exist.

A method for generating metadata based on a web page according to an embodiment of the present invention is a method for generating metadata based on a web page by a metadata generating application running on a processor of a computing device, comprising extracting text included in the web page. step; extracting text based on the characters in the image included in the web page; extracting a portion of the extracted text as valid text by performing natural language processing on the extracted text; calculating the importance of the extracted valid text, and determining a part of the extracted valid text as a key keyword according to the calculated importance; and generating the metadata based on the determined key keyword.

In this case, the step of extracting text based on the characters in the image included in the web page includes the steps of extracting the image included in the web page, extracting the characters in the extracted image, and the extracted characters converting to text.

In addition, the step of extracting the valid text from the part of the extracted text includes removing stopwords from the extracted text and removing stopwords from the extracted text.

In addition, the step of determining a part of the extracted valid text as a key keyword includes the steps of separating the valid text into word units, calculating the importance of the separated words, in the order of increasing importance. Accordingly, it includes the step of determining the selected key word as the key keyword.

In addition, the step of determining a part of the extracted valid text as a key keyword includes the steps of dividing the valid text into sentence units, calculating the importance of the separated sentences, and in the order of the highest importance. Therefore, it includes the step of determining the selected key sentence as the key keyword.

In addition, the metadata includes a title metadata object included in a title section used as a title of the web page, a description metadata object of a description section used as a description for the web page, and a tag word of the web page. Contains the tagword metadata object of the used tagword section.

In addition, the generating of the metadata includes setting at least one core sentence among the core sentences as the title metadata object according to a position or size of a character in an image that is a basis for the core sentence.

In addition, the generating of the metadata includes setting at least a part of the core sentences as the description metadata entity.

In addition, the generating of the metadata includes the steps of extracting the k-th key words according to the order of importance of the key words, and setting the extracted key words as the tag word metadata object. include

In addition, the method for generating metadata based on a web page according to an embodiment of the present invention further includes providing a user editing interface for previously generated metadata.

In the method for generating metadata based on a web page according to an embodiment of the present invention, crawling is performed based on any one or more of text of a web page, text in an image, and alternative text for an image to By providing metadata describing the attribute, it is possible to provide metadata in consideration of not only plain text in a web page but also text about an image included in the web page.

In addition, the method for generating metadata based on a web page according to an embodiment of the present invention provides metadata that considers not only plain text in a web page but also text for an image included in the web page, thereby providing metadata accuracy and reliability can be further improved.

In addition, the method for generating metadata based on a web page according to an embodiment of the present invention can improve the quality of generated metadata by generating metadata for a corresponding web page based on the properties of text crawled from the web page. can have an effect.

In addition, the method for generating metadata based on a web page according to an embodiment of the present invention automatically generates metadata for a web page and provides an editing interface for the automatically generated metadata, so that frequent updates occur and individuals can It has the effect of easily defining and managing metadata that is difficult to manage.

However, the effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood from the description below.

1 is a conceptual diagram of a system for generating metadata based on a web page according to an embodiment of the present invention.
2 is an internal block diagram of a mobile type computing device according to an embodiment of the present invention.
3 is an internal block diagram of a desktop type computing device according to an embodiment of the present invention.
4 is a flowchart illustrating a method of generating metadata based on a web page according to an embodiment of the present invention.
5 is an example of a state of showing a web page including an image according to an embodiment of the present invention.
6 is a diagram for explaining a method of generating metadata based on a web page including an image according to an embodiment of the present invention.
7 is an example of providing an editing interface for metadata according to an embodiment of the present invention.
8 is an example of providing a web page search call screen based on metadata according to an embodiment of the present invention.
9 is an example of a state showing an attribute suitability interface according to an embodiment of the present invention.
10 is an example showing a preview interface according to an embodiment of the present invention.

Since the present invention can apply various transformations and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. Effects and features of the present invention, and a method of achieving them, will become apparent with reference to the embodiments described below in detail in conjunction with the drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various forms. In the following embodiments, terms such as first, second, etc. are used for the purpose of distinguishing one component from another, not in a limiting sense. Also, the singular expression includes the plural expression unless the context clearly dictates otherwise. In addition, terms such as include or have means that the features or components described in the specification are present, and do not preclude the possibility that one or more other features or components will be added. In addition, in the drawings, the size of the components may be exaggerated or reduced for convenience of description. For example, since the size and thickness of each component shown in the drawings are arbitrarily indicated for convenience of description, the present invention is not necessarily limited to the illustrated bar.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, and when described with reference to the drawings, the same or corresponding components are given the same reference numerals, and the overlapping description thereof will be omitted. .

1 is a conceptual diagram of a system for generating metadata based on a web page according to an embodiment of the present invention.

Referring to FIG. 1 , a system for generating metadata based on a web page according to an embodiment of the present invention may include a computing device 100 , a metadata server 400 , and a network 500 .

In an embodiment, the computing device 100 , the metadata server 400 , and the network 500 interact with each other to generate metadata describing properties of a web page based on images and text in the web page. Data generation applications can be implemented.

In detail, in an embodiment of the present invention, the metadata generating application (hereinafter referred to as the meta application) may access a web page and extract text in the corresponding web page.

In addition, in an embodiment, the meta application may perform natural language processing on the extracted text to derive valid text that is a meaningful text as a search index when searching the Internet.

In addition, the meta application may summarize the corresponding web page based on the derived valid text, and through this, may derive a key keyword for the web page.

In this case, the core keyword may be a text determined to have a higher importance on a web page than a predetermined criterion among valid texts obtained through natural language processing. A detailed description thereof will be provided later.

Also, in an embodiment, the meta application may generate metadata based on the derived core keyword and may provide an editing interface for the generated metadata.

Also, the meta application may provide a web page search call screen for a corresponding web page based on the generated metadata.

Here, the web page search call screen may be an information providing interface for the web page displayed on a screen output as a search result when a web page search based on metadata is performed on a search engine. A detailed description will be given below.

Meanwhile, the computing device 100 and the metadata server 400 of FIG. 1 may be connected through the network 500 .

Here, the network 500 means a connection structure capable of exchanging information between each node, such as the computing device 100 and the metadata server 400 , and an example of such a network 500 includes 3GPP (3rd Generation Partnership Project network, LTE (Long Term Evolution) network, WIMAX (World Interoperability for Microwave Access) network, Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network) ), PAN (Personal Area Network), Bluetooth (Bluetooth) network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc., but are not limited thereto.

- Computing device (100)

In an embodiment of the present invention, the computing device 100 may execute a meta application that generates metadata describing properties of a web page based on images and text in the web page.

Also, in an embodiment, the computing device 100 may include various types of computing devices 100 (eg, a mobile type or a desktop type) on which meta-applications are installed.

1. Mobile Type Computing Device

In an embodiment of the present invention, the mobile type computing device 200 may be a mobile device such as a smart phone or a tablet PC on which a meta application is installed.

For example, the mobile type computing device 200 may include a smart phone, a mobile phone, a digital broadcasting terminal, personal digital assistants (PDA), a portable multimedia player (PMP), a tablet PC, and the like. have.

2 is an internal block diagram of a mobile type computing device 200 according to an embodiment of the present invention.

Referring to FIG. 2 , a mobile type computing device 200 according to an exemplary implementation includes a memory 210 , a processor assembly 220 , a communication module 230 , an interface module 240 , an input system 250 , and a sensor. may include a system 260 and a display system 270 . These components may be configured to be included within the housing of the mobile type computing device 200 .

In detail, the memory 210 stores the meta application 211, and the meta application 211 may store any one or more of various application programs, data, and commands for providing a web page-based metadata generation service. .

For example, the memory 210 may include web page identification data, a text buffer, an image buffer, a location engine, a display engine, and the like for a web page-based metadata generation service.

That is, the memory 210 may store commands and data that may be used to provide a web page-based metadata generation service environment.

In addition, the memory 210 may include at least one or more non-transitory computer-readable storage media and a temporary computer-readable storage medium. For example, the memory 210 may be various storage devices such as ROM, EPROM, flash drive, hard drive, and the like, and a web storage that performs a storage function of the memory 210 on the Internet. may include.

The processor assembly 220 may include at least one processor capable of executing instructions of the meta application 211 stored in the memory 210 to perform various tasks for providing a web page-based metadata generation service. can

In an embodiment, the processor assembly 220 may control overall operations of components through the meta application 211 of the memory 210 to provide a web page-based metadata generation service.

The processor assembly 220 may include a central processing unit (CPU) and/or a graphics processor unit (GPU). In addition, the processor assembly 220, ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers (controllers) ), micro-controllers, microprocessors, and other electrical units for performing other functions.

Communication module 230 may include one or more devices for communicating with other computing devices (eg, metadata server 400 ). The communication module 230 may communicate through a wireless network.

In detail, the communication module 230 may communicate with a computing device storing a content source for implementing a web page-based metadata generation service environment, and may communicate with various user input components such as a controller receiving user input. have.

In an embodiment, the communication module 230 may transmit/receive various data related to a web page-based metadata generation service with the metadata server 400 and/or other computing device 100 .

Such a communication module 230, the technical standards or communication methods for mobile communication (eg, Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), 5G NR (New Radio), WIFI) Alternatively, data may be wirelessly transmitted/received with at least one of a base station, an external terminal, and an arbitrary server on a mobile communication network constructed through a communication device capable of performing a short-range communication method or the like.

The sensor system 260 may include various sensors such as an image sensor 261 , a position sensor (IMU) 263 , an audio sensor, a distance sensor, a proximity sensor, and a contact sensor.

The image sensor 261 may capture images and/or images of the physical space around the mobile type computing device 200 .

In an embodiment, the image sensor 261 may capture and acquire an image (eg, an image included in a web page, etc.) related to a web page-based metadata generating service.

In addition, the image sensor 261 may be disposed on the front or / and rear of the mobile type computing device 200 to acquire an image by photographing the disposed direction side, and face the outside of the mobile type computing device 200 . Physical space can be photographed through the placed camera.

The image sensor 261 may include an image sensor device and an image processing module. Specifically, the image sensor 261 may process a still image or a moving image obtained by an image sensor device (eg, CMOS or CCD).

Also, the image sensor 261 may process a still image or a moving image obtained through the image sensor device using an image processing module to extract necessary information, and transmit the extracted information to the processor.

The image sensor 261 may be a camera assembly including at least one camera. The camera assembly may include a general camera that captures a visible light band, and may further include a special camera such as an infrared camera or a stereo camera.

The IMU 263 may sense at least one of motion and acceleration of the mobile type computing device 200 . For example, it may consist of a combination of various position sensors such as an accelerometer, a gyroscope, and a magnetometer. In addition, by interworking with the location communication module 230 such as GPS of the communication module 230 , spatial information about the physical space around the mobile type computing device 200 may be recognized.

Also, the IMU 263 may detect information for detecting and tracking the user's gaze direction and head movement based on the detected position and direction.

Further, in some implementations, the meta-application 211 may use such an IMU 263 and an image sensor 261 to determine a user's location and orientation within the physical space or to recognize a feature or object within the physical space.

The audio sensor 265 may recognize a sound around the mobile type computing device 200 .

Specifically, the audio sensor 265 may include a microphone capable of detecting a voice input of a user of the mobile type computing device 200 .

In an embodiment, the audio sensor 265 may receive voice data required for a web page-based metadata generation service from a user.

The interface module 240 may communicatively connect the mobile type computing device 200 with one or more other devices. Specifically, the interface module 240 may include wired and/or wireless communication devices that are compatible with one or more different communication protocols.

The mobile type computing device 200 may be connected to various input/output devices through the interface module 240 .

For example, the interface module 240 may be connected to an audio output device such as a headset port or a speaker to output audio.

Although it has been described as an example that the audio output device is connected through the interface module 240 , an embodiment in which the audio output device is installed inside the mobile type computing device 200 may also be included.

This interface module 240, a wired / wireless headset port (port), an external charger port (port), a wired / wireless data port (port), a memory card (memory card) port, for connecting a device equipped with an identification module Ports, audio Input/Output (I/O) ports, video I/O (Input/Output) ports, earphone ports, power amplifiers, RF circuits, transceivers and other communication circuits It may be configured to include at least one of.

The input system 250 may detect a user's input (eg, a gesture, a voice command, an operation of a button, or other type of input) related to the web page-based metadata generation service.

Specifically, the input system 250 may include a button, a touch sensor, and an image sensor 261 that receives user motion input.

Also, the input system 250 may be connected to an external controller through the interface module 240 to receive a user's input.

The display system 270 may output various information related to a web page-based metadata generation service as a graphic image.

In an embodiment, the display system 270 may display text and/or images included in a web page as graphic images.

Such displays include a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display. , a three-dimensional display (3D display), may include at least one of the electronic ink display (e-ink display).

The above components may be disposed within the housing of this mobile type computing device 200 , and the user interface may include a touch sensor 273 on a display 271 configured to receive user touch input.

In detail, the display system 270 may include a display 271 that outputs an image and a touch sensor 273 that detects a user's touch input.

Exemplarily, the display 271 may be implemented as a touch screen by forming a layer structure with the touch sensor 273 or being integrally formed therewith. Such a touch screen may function as a user input unit providing an input interface between the mobile type computing device 200 and the user, and may provide an output interface between the mobile type computing device 200 and the user.

2. Desktop type computing device

3 is an internal block diagram of a desktop type computing device 300 according to an embodiment of the present invention.

In the description of the components of the desktop type computing device 300, the overlapping contents will be replaced with the description of the components of the mobile type computing device 200. Hereinafter, the differences with the mobile type computing device 200 will be described. explained in the center.

Referring to FIG. 3 , in another example, the desktop type computing device 300 is wired/wireless, such as a fixed desktop PC, a laptop computer, and a personal computer such as an ultrabook in which the meta application 311 is installed. The apparatus may further include a device installed with a program for executing a metadata generation service based on a web page based on communication.

In addition, the desktop type computing device 300 may include the user interface system 350 to receive a user input (eg, a touch input, a mouse input, a keyboard input, a gesture input, a motion input using a guide tool, etc.). have.

Illustratively, the desktop type computing device 300 uses the user interface system 350 with various communication protocols such as a mouse 351 , a keyboard 352 , a gesture input controller, an image sensor 361 (eg, a camera), and audio It may be connected to at least one device such as the sensor 365 to obtain a user input.

In addition, the desktop type computing device 300 may be connected to an external output device through the user interface system 350 , for example, the display device 370 , an audio output device, or the like.

Further, the desktop type computing device 300 according to the example implementation may include a memory 310 , a processor assembly 320 , a communication module 330 , a user interface system 350 , and an input system 340 . . These components may be configured to be included within the housing of the desktop type computing device 300 .

However, in the embodiment of the present invention, the components shown in FIGS. 2 and 3 are not essential for implementing the computing device 100 , so the computing device 100 described in the present specification includes the components listed above. It may have more or fewer components.

- Metadata server: 400

Meanwhile, in an embodiment of the present invention, the metadata server 400 may perform a series of processes for providing a web page-based metadata generation service.

In detail, in the embodiment, the metadata server 400 describes the properties of the web page by performing crawling based on any one or more of text of a web page, text in an image, and alternative text for an image. It is possible to provide a metadata generation service that generates metadata for

In more detail, referring further to FIG. 1 , the above metadata server 400 may include a metadata service providing server 410 , a metadata generating server 420 , and a database server 430 . In this case, according to an embodiment, each of the components may be implemented as a device separate from the metadata server 400 , or may be implemented by being included in the metadata server 400 . Hereinafter, each component is described as being included in the metadata server 400 and implemented, but is not limited thereto.

Here, the metadata service providing server 410 may provide an environment in which the meta applications 211 and 311 can operate in the computing device 100 .

In an embodiment, the metadata service providing server 410 is an application program for implementing the meta applications 211 and 311 that generate metadata describing properties of a web page based on images and text in the web page. , data and/or instructions.

Also, the metadata generating server 420 may generate metadata based on key keywords derived from a web page according to a predetermined criterion.

In an embodiment, the metadata generating server 420 may obtain a core keyword derived from a web page, and include the location, font form, and/or frequency of appearance in the web page of the obtained core keyword in the web page. based on the predetermined criteria, it is possible to generate and provide metadata for a corresponding web page.

In addition, the database server 430 may store and manage various application programs, applications, commands and/or data for implementing a web page-based metadata generation service.

In an embodiment, the database server 430 provides text information for each web page, image information (in an embodiment, alternative text tag information, etc.), valid text information, key keyword information, metadata information, and/or a web page search call screen. You can store and manage information, etc.

On the other hand, the metadata server 400 including the above components may be composed of at least one or more metadata service providing server 410, metadata generating server 420 and/or database server 430, It may include processors for data processing and memories for storing instructions for providing a web page-based metadata generation service.

In addition, according to embodiments, all or some functions of the operations performed by the metadata server 400 may be performed by the computing device 100 , and various embodiments are possible. In the following description, a series of operations performed by the above-described metadata server 400 will be described as being performed by the computing device 100 , but the present invention is not limited thereto.

- Method of generating metadata based on web pages

Hereinafter, a method for generating web page-based metadata according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings. In the following embodiment, the computing device 100 will be described with reference to the mobile type computing device 200 , but is not limited thereto.

First, in an embodiment of the present invention, the mobile type computing device 200 may execute the meta application 211 capable of generating metadata describing properties of a web page based on images and text in the web page. have.

4 is a flowchart illustrating a method of generating metadata based on a web page according to an embodiment of the present invention.

Referring to FIG. 4 , the meta application 211 executed in the mobile type computing device 100 as described above may access an online web page. (S101)

In detail, the web page may be an Internet page implemented as an online viewable document, text, image, and/or video and displayed by a browser through an Internet search. Such a web page may include a simple form or blank space, and may provide a unique URL (Uniform   Resource   Locator) for inducing access to the web page.

That is, in the embodiment of the present invention, the meta application 211 may access a web page for which metadata is to be generated, based on a unique URL provided for each web page.

Here, the metadata according to the embodiment is data that implies and represents the properties of a web page, and may be data information structured in order to efficiently use data of a web page.

In an embodiment, such metadata may function as a search index utilized during a search in a search engine.

Also, metadata according to an embodiment may be composed of metadata objects included in a plurality of metadata sections.

Here, the metadata section defines an information type of an attribute describing a web page, and may include a title section, a description section, and/or a tag word section for the web page in an embodiment. In addition, each of the metadata sections may include a metadata section metadata object.

In this case, the metadata object means a series of metadata elements that describe the same aspect of data of a web page. Metadata section

In an embodiment, the metadata object may be formed from elements constituting a web page. For example, the metadata object may include text elements included in web page data and text elements obtained by extracting characters displayed in images.

That is, in the embodiment, the metadata object may be generated based on text elements extracted as key keywords from texts in the web page and texts included in images in the web page.

In this embodiment, the metadata object may include a title section included in the title metadata section, a description section included in the description metadata section, and/or a tag word section included in the tag word metadata section, and the like.

Also, in an embodiment of the present invention, the meta-application 211 accessing the above web page may extract text in the accessed web page. (S103)

In detail, the meta application 211 may obtain text data included in a web page by receiving and/or extracting text data from the web page.

5 is an example of a state of showing a web page including an image according to an embodiment of the present invention.

However, referring to FIG. 5 , when a web page is generally implemented, there are many cases in which text processed in the form of an image is included.

Thus, as shown in FIG. 5 ( 1 ), the entire web page is implemented as an image, or as shown in FIG. 5 ( 2 ), at least a part of the web page is implemented as an image.

Therefore, in an embodiment of the present invention, the meta application 211 extracts text in a web page accessed to more accurately generate metadata for a web page, and when an image exists in the web page, It is possible to extract text within a web page by considering all characters.

In this case, the meta-application 211 may use a character extraction model and/or algorithm in a predetermined image in order to extract characters in the image.

For example, the meta application 211 may extract even the characters in the image of the web page by utilizing a character extraction model in the image, such as the Google cloud vision API (Google cloud vision API).

For reference, Google Cloud's Vision API provides powerful machine learning models pre-trained via REST and RPC APIs, allowing you to label images and quickly classify them into millions of predefined categories. Such a vision API model can detect an object in an image, printed or handwritten text, etc., and extract useful data from it to assist the image-based metadata creation process.

However, in the present embodiment, the meta-application 211 may use various conventionally known mathematical algorithms to extract characters in an image in addition to the Google Cloud vision API described above, and in the present invention, It does not limit or limit the algorithm itself.

In this case, the text extracted from the web page as above may be in the form of words and/or sentences.

As such, in the embodiment of the present invention, the meta application 211 extracts not only the plain text in the web page but also the characters in the image included in the web page and uses it as basic data for automatically generated metadata, so that the web page later It is possible to further improve the accuracy and reliability of the metadata generated for the data.

Meanwhile, in another embodiment, when an alternative text tag matching an image of a web page exists, the meta application 211 may extract words and/or sentences included in the corresponding alternative text tag as text.

Here, the alternative text tag may be information stored by matching a predetermined main word and/or sentence to the image.

For example, if the meta application 211 matches the first image of the web page and an alternative text tag 'This sentence is a key sentence for the first image' exists, it is preset as the alternative text tag. You can include the sentences in the text extracted from the web page.

As described above, in the embodiment, the meta application 211 extracts the text in the corresponding web page by using the alternative text tag, which is preset information for the main text on the image in the web page, thereby increasing the efficiency of generating metadata. can be increased

In addition, in an embodiment of the present invention, the meta-application 211 may perform Natural Language Processing (NLP) on the extracted text as described above, and may derive valid text through this. (S105)

Here, the natural language processing according to the embodiment may mean a process by a series of description sets for analyzing, extracting, and understanding meaningful information from text.

In addition, the valid text may mean a text meaningful as a search index when a web page is searched through the Internet.

In detail, the meta-application 211 is a known natural language processing technique (eg, Tabular Parsing algorithm, Hidden Markov Model (HMM) and/or Bi-LSTM-CRF (Bidirectional Long Short-Term Memory with Conditional Random) Fields), etc.), natural language processing may be performed on the text extracted from the web page, and valid text may be set.

However, in the present embodiment, the meta-application 211 may use various conventionally known mathematical algorithms in addition to the above-described techniques to perform natural language processing for analyzing and extracting meaningful information from text. It does not limit or limit the algorithm itself that performs natural language processing by analyzing and extracting meaningful information.

In this case, for example, the meta-application 211 may perform natural language processing to perform stopword/stopword processing on text.

Here, the stopword/disused sentence processing may be a process of excluding from the text extracted from the web page, words/sentences that are not used as search terms/sentences during Internet search.

For example, stop-word processing may be a process of excluding meaningless articles, prepositions, articles and/or conjunctions as search index words by performing part-of-speech tagging on text extracted from web pages. have.

In addition, the dead sentence processing is a text classification operation (eg, advertising text classification and/or menu text classification irrelevant to the web page) and/or phonemic notation (Lemmatization: based on context) based on the text extracted from the web page This may be a process of excluding sentences that have no meaning as a search index word by performing a technique for identifying words).

In an embodiment, when natural language processing is performed on text in the form of words and/or sentences extracted from a web page, the meta-application 211 excludes stopwords/sentences from the extracted text to be used as a search index when searching the Internet. Only meaningful text can be derived.

In addition, the meta application 211 may refine the text extracted from the web page by setting the derived text as a valid text.

In addition, in the embodiment of the present invention, the meta application 211 may summarize a web page based on the valid text obtained based on the above natural language processing, and may derive a key keyword through this. (S107)

In detail, the meta-application 211 is a search index derived from a web page, based on valid text, which is a meaningful text (that is, text in which stopwords/sentences are excluded among texts extracted from a web page), It can be summarized and key keywords can be derived.

Here, the core keyword according to the embodiment may be text determined to have high importance on a web page by a predetermined standard or higher (eg, similarity between texts is greater than or equal to a preset value) among valid texts obtained through natural language processing. have.

These core keywords are a key word whose importance on a web page is determined to be greater than or equal to a predetermined criterion among valid texts in word form and/or a core sentence whose importance on a web page is determined to be greater than or equal to a predetermined criterion among valid texts in a sentence form. may include.

In detail, the meta application 211 may perform web page summary based on valid text using a known text summary algorithm, and may derive key keywords in this process.

For example, the meta application 211 may perform a summary of a web page by using a text summary algorithm such as a TextRank algorithm.

In this case, the meta application 211 may analyze the similarity between valid texts of the web page based on the TextRank algorithm.

In addition, the meta application 211 may derive key keywords according to importance in the web page based on the analyzed similarity.

For example, as the number of other texts having a high similarity with the first text of the valid text increases, the meta-application 211 may assign a high importance to the corresponding texts and may select them as key keywords.

Here, the importance is a TF value (exposure frequency value) calculated through the text rank algorithm, a TR (TextRank) value, a TF-IDF value, a PMI (Point-wise Mutual Information) value, or each text derived through the values. (eg, a word or a sentence) may mean a score.

In addition, the core keyword may mean some text classified from the entire text based on the importance value for each text. For example, the core keyword may include texts extracted as much as a preset number of texts according to the size of the importance value among all texts in the web page.

Also, the meta application 211 may summarize the corresponding web page based on the derived core keyword.

For example, the meta application 211 may summarize the corresponding web page in such a way that only the derived core keywords are maintained and processed.

Or, for example, the meta-application 211 builds a word graph or a sentence graph using a textrank algorithm, and then builds a graph ranking algorithm, PageRank. It can be used to determine key words and/or key sentences from a given text.

For example, the meta-application 211 may classify valid texts by word, calculate importance based on similarity and exposure frequency for each word, and select a predetermined number of key words or words having a predetermined importance or higher. By limiting the order, some words of the valid text may be determined as key words.

In addition, the meta-application 221 classifies the valid text by sentence, calculates the importance of each classified sentence, and limits the number of key sentences or sentences with a predetermined importance or higher in the order of highest importance to part of the valid text. A sentence can be determined as a key sentence.

In addition, the meta application 211 may organize the web page into a summarized text document by combining the determined key words and/or key sentences.

That is, the meta-application 211 may derive a key keyword including a key word and/or a key sentence from the valid text of the web page based on the TextRank algorithm, and summarize the web page based on the derived core keyword. You can create a single text document.

However, in the present embodiment, the meta-application 211 may use various conventionally known mathematical algorithms in addition to the above-described techniques for deriving key keywords from text, and in the present invention, the algorithm itself for deriving key keywords from text is used. not limiting or limiting.

Also, in an embodiment of the present invention, the meta application 211 may generate metadata based on the derived core keyword. (S109)

To recall again, the metadata according to the embodiment is data implying and representing the properties of a web page, and may be data information structured to efficiently use data of a web page, and a search used in a search in a search engine. It can function as an index.

Also, in an embodiment, such metadata may include a title, description, and/or tag word metadata section for the web page, and the like.

Also, in an embodiment, the metadata may include at least one or more metadata entities classified into each metadata section.

In detail, the meta application 211 may generate metadata based on a key keyword (including a key word and/or a key sentence) that is a meaningful text as a search index for a web page according to a predetermined criterion.

In more detail, the meta-application 211 includes 1) a location in a web page (eg, top, middle, or bottom, etc.) of a key keyword derived from a web page, 2) a font type (eg, font size and/or thickness, etc.) 3) the importance of each key keyword and/or 4) the frequency of appearance in the web page may be set based on the predetermined criteria.

In addition, the meta application 211 may classify the core keyword into at least one category of a title metadata section, a description metadata section, or a tag word metadata section according to a set predetermined criterion, and It can be set as a metadata object.

In detail, the meta application 211 may generate a title metadata object included in the title section based on some of the key keywords.

For example, the meta application 211 may extract the nth number of key words in the order of importance among the key words, and generate a sentence including all of the extracted n key words to generate the metadata object of the title section. .

Also, the meta-application 211 may extract m th key sentences from among the key sentences in order of importance, and list the extracted m core sentences to generate a metadata object of the title section.

In another embodiment, the meta application 221 may determine the core sentence to be the metadata object of the title section based on the position and/or the shape of the characters of the image that is the basis of the core sentence.

6 is a diagram for explaining a method of generating metadata based on a web page including an image according to an embodiment of the present invention.

For example, referring to FIG. 6 , the meta application 211 may determine the key sentence for the key sentence as the metadata object of the title section when the character that is the basis of the core sentence occupies the largest area in the image. have.

In addition, the meta application 211 may determine the core sentence of the corresponding character as the metadata object of the title section when the character, which is the basis of the core sentence, is located at the top of the image.

In addition, the meta application 211 may calculate a score based on the position and size of the letters that are the basis of each key sentence, and determine the core sentence for the letter having the highest score as the metadata object of the title section.

For example, the meta-application 211, among the key sentences of the key keywords extracted on the web page shown in FIG. 6 , is the letter 'Moisturizing the strong sunlight falling on my skin' located at the 'top' of the web page. In the title section, we classify the core sentences made up of the key sentences and the key sentences for the 'sunshine rain line', which is the largest in size, in the title section, and combine the classified key sentences with 'Strong Sunshine Rain Moisturizing Sunshine Rain Line' in the title section. It can be created as a data object.

In this case, according to an embodiment, the meta application 211 may directly set the title section based on an alternative text tag that may be pre-stored by matching the image of the web page.

In detail, when there is an alternative text tag matching the image of the web page, the meta application 211 may set the alternative text tag as the title section regardless of the derived core keyword.

Also, in an embodiment, the meta application 211 may set a limit on the total number of characters of a title metadata object generated when a title section is set.

For example, the meta application 211 may set the limit of the total number of characters of the title metadata object to 'within 40 characters including spaces', and through this, may assist in composing a title section of an appropriate length.

In addition, when setting the title section, the meta application 211 may further include additional information for enhancing the completeness of a call screen provided during a later search in addition to the generated title metadata object.

In an embodiment, the meta application 211 may set the title section by further including a generated title metadata object, a corresponding website name, and special characters to help distinguish the title metadata object from the website name. have.

For example, the meta application 211 may configure the title section by listing 'title metadata object', 'special character', and 'website name' in order.

Also, the meta application 211 may generate a description metadata object included in the description section based on at least some of the key keywords.

For example, the meta-application 211 may extract x-th keywords from among the key words in the order of importance, and generate at least one sentence combining the extracted x key words as explanatory metadata objects.

Also, the meta-application 211 may extract y th key sentences from among the key sentences in the order of importance, and list the y extracted core sentences to generate the metadata object of the description section.

For example, the meta-application 211, among the key sentences of the key keywords extracted on the web page shown in FIG. , 'Hatsalbi Cleansing Water' and 'Hatsalbi Moisture Cream' can be set as metadata objects as a description section.

In this case, according to embodiments, the meta application 211 may be configured in various embodiments, such as all or part of a title section may also be set as a description section.

For example, the meta application 211 may configure the description section by including the sentence 'sunray rain line' set as the title section in the description section.

In addition, in an embodiment, the meta application 211 may directly set a key sentence extracted based on the similarity of a word frequently appearing in a web page document as a description section.

In this case, the meta application 211 may set a limit on the total number of characters of the description metadata object in the description section.

For example, the meta application 211 may set the limit of the total number of characters of the description metadata object to 'less than 80 characters including spaces', and through this, a description section of an appropriate length may be configured.

Meanwhile, the meta application 211 may select at least some of the key words included in the core keyword as the tag word metadata object included in the tag word section.

In an embodiment, the meta application 211 may extract up to k th key words in the order of high importance, and set the extracted key words as tag word metadata objects in the tag word section.

In addition, in another embodiment, the meta application 211 calculates the frequency of occurrence of the key word in the web page, and the calculated frequency is based on a predetermined criterion (eg, a preset number of times or more and/or the frequency of appearance of each of the entire key words) If it satisfies the top 1% of the number, etc.), the corresponding key word can be classified as a tag word metadata section and set as a tag word section.

For example, the meta-application 211 tags the keyword 'sunshine rain', which is a key word with the highest frequency of appearance in the web page, among the core words of the core keywords extracted on the web page shown in FIG. 6 , in the metadata section. It can be classified as a tag word section.

In this case, according to an embodiment, the meta application 211 may set at least a part of the tag word section as the title section.

For example, the meta-application 211 may include the number of appearances of the first tagged word object having the highest frequency of appearance in the web page among the tagged word sections, and the highest web page among words included in the core sentence of the title section. By comparing the number of appearances of the first word having the number of appearances within, when the number of appearances of the first tagged word object has a higher frequency than the number of appearances of the first word, the title section is divided into the first tagged word It can be set as an object.

In addition, in the embodiment, the meta application 211 that classifies the core keyword derived from the web page into each metadata section and sets it as a metadata object, based on each metadata section and the metadata object, provides a meta data for the corresponding web page. data can be generated.

That is, the meta-application 211 provides metadata for the web page based on the title, description, and/or tag word section divided into a title section, a description section, and/or a tag word section by classification according to a predetermined criterion. can create

As such, in the embodiment of the present invention, the meta application 211 sets the corresponding core keyword as a title, description and/or tag word according to the position, font type, and/or frequency of appearance of the core keyword in the web page, thereby making the web The performance of a crawling process for information provided by a page may be improved, and the quality of metadata generated based thereon may also be improved.

Also, in an embodiment of the present invention, the meta application 211 may provide an editing interface for the metadata generated as described above. (S111)

7 is an example of providing an editing interface for metadata according to an embodiment of the present invention.

In detail, referring to FIG. 7 , the meta application 211 may provide a metadata editing interface capable of editing a title, description, and/or tag word section included in the generated metadata.

In addition, in an embodiment, the meta application 211 may correct the title, description, and/or tag word section based on a user input to the provided editing interface.

For example, when the title section of the generated metadata is 'This sentence is the title section.', the meta application 211 changes the title section to 'The title section in which this sentence has been changed according to a user input based on the editing interface.' It can be changed to '.

As such, the meta application 211 provides an editing function for the automatically generated metadata, so that the quality and accuracy of the metadata can be improved based on the user's cognitive judgment, and the metadata is optimized for the user's needs. You can set metadata.

In addition, in the embodiment of the present invention, the meta application 211 may provide various interfaces to assist the metadata generation process when generating metadata as described above.

In detail, the meta application 211 may provide 1) an attribute suitability interface and 2) a preview interface based on a screen for generating metadata.

In more detail, referring to FIG. 9 , when generating metadata, the meta-application 211 indicates the property suitability by determining whether each metadata object included in each section is an object suitable for inclusion in the corresponding section. An attribute conformance interface may be provided.

In this case, the meta application 211 may determine the property suitability of the metadata object for each section by utilizing the classification criteria for each section.

For example, in the case of a title section, the meta application 211 may determine the property suitability of the title metadata object based on whether a key keyword is included and/or whether the same text exists in other web pages.

In this case, the meta application 211 may provide the attribute suitability through the attribute suitability interface according to a predetermined criterion (eg, step-by-step classification according to the attribute suitability value interval, etc.).

In detail, the attribute suitability interface indicating the attribute suitability as described above can display the attribute suitability of the metadata object belonging to each section as a predetermined graphic image (eg, bar graph, etc.), and describes the criterion for determining the attribute suitability. It may include interfaces.

For example, in the attribute suitability interface, when the attribute fitness is classified according to the attribute suitability value range as 'very low, low, normal, high, and very high levels', and the calculated attribute fitness value is judged to be 'normal' , it is possible to display the property fit as above through a graphic image implemented as a bar graph.

In addition, the attribute suitability interface may display a description of the criterion for which the attribute suitability is calculated (eg, a core keyword of a web page is not included, etc.) in text.

On the other hand, referring to FIG. 10 , in the embodiment of the present invention, when the metadata is changed through the above-described process, the meta application 211 shows before and after modification of a web page search call screen to be generated based on the corresponding metadata. A preview interface can be provided to check the

In detail, the preview interface creates a graphic image of the web page search call screen before and after the changed metadata is applied as a graphic image before the changed metadata is actually applied to the web page search call screen according to the user's request. can provide

That is, the meta application 211 may provide a search call screen before and after modification before actually applying the modified metadata to compare before and after the change, and check the screen to be finally provided and perform re-correction if necessary. It can assist in the effective review of metadata, for example.

Also, in an embodiment of the present invention, the meta application 211 may provide a web page search call screen based on the generated metadata. (S113)

Here, the web page search call screen may be an information providing interface for the web page displayed on a screen output as a search result when a web page search based on metadata is performed on a search engine.

8 is an example of providing a web page search call screen based on metadata according to an embodiment of the present invention.

In detail, referring to FIG. 8 , in the embodiment, the meta application 211 selects at least one or more metadata objects included in the title, description, and/or tag word metadata section of metadata, based on a predetermined criterion (eg, meta data). data section, etc.) to create a web page search call screen.

For example, the meta application 211 may arrange the title section of the metadata at the top of the web page search call screen. Also, the meta application 211 may arrange at least one description section included in the metadata in the center of the web page search call screen. Also, the meta application 211 may arrange at least one tag word section included in the metadata at the bottom of the web page search call screen.

Also, in the embodiment, the meta application 211 displays a web page search call screen in which each metadata object of metadata is arranged on a search result screen when a web page search based on the metadata is performed on a search engine. It can be printed and provided.

As such, the meta application 211 generates and provides a web page search call screen based on the generated metadata considering not only plain text in the web page, but also characters in the image included in the web page, thereby providing the web page. It is possible to display a search result that more accurately implies information about a web page, and to allow the web page to be searched through a search engine based on a more accurate search index for the web page.

As described above, in the method for generating metadata based on a web page according to an embodiment of the present invention, crawling is performed based on any one or more of the text of the web page, the text in the image, and the alternative text for the image, and the web page is By providing metadata describing the properties of , it is possible to provide metadata that considers not only plain text in a web page but also text about an image included in the web page.

In addition, the method for generating metadata based on a web page according to an embodiment of the present invention provides metadata that considers not only plain text in a web page but also text for an image included in the web page, thereby providing metadata accuracy and reliability can be further improved.

In addition, the method for generating metadata based on a web page according to an embodiment of the present invention can improve the quality of generated metadata by generating metadata for a corresponding web page based on the properties of text crawled from the web page. can have an effect.

In addition, the method for generating metadata based on a web page according to an embodiment of the present invention automatically generates metadata for a web page and provides an editing interface for the automatically generated metadata, so that frequent updates occur and individuals can It has the effect of easily defining and managing metadata that is difficult to manage.

In addition, the embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include hard disks, magnetic media such as floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floppy disks. medium), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. A hardware device may be converted into one or more software modules to perform processing in accordance with the present invention, and vice versa.

The specific implementations described in the present invention are only examples, and do not limit the scope of the present invention in any way. For brevity of the specification, descriptions of conventional electronic components, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connection or connection members of the lines between the components shown in the drawings exemplarily represent functional connections and/or physical or circuit connections, and in an actual device, various functional connections, physical connections that are replaceable or additional may be referred to as connections, or circuit connections. In addition, unless there is a specific reference such as “essential” or “importantly”, it may not be a necessary component for the application of the present invention.

In addition, although the detailed description of the present invention has been described with reference to a preferred embodiment of the present invention, those skilled in the art or those having ordinary knowledge in the art will appreciate the spirit of the present invention described in the claims to be described later. And it will be understood that various modifications and variations of the present invention can be made without departing from the technical scope. Accordingly, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be defined by the claims.

Claims (10)

A method for generating metadata based on a web page by a metadata generating application running on a processor of a computing device, comprising:
extracting text included in the web page;
extracting text based on the characters in the image included in the web page;
extracting a portion of the extracted text as valid text by performing natural language processing on the extracted text;
calculating the importance of the extracted valid text and determining a part of the extracted valid text as a key keyword according to the calculated importance; and
and generating the metadata based on the determined key keyword.
A method of generating metadata based on a web page. The method of claim 1,
The step of extracting text based on the characters in the image included in the web page,
extracting an image included in the web page, extracting characters from the extracted image, and converting the extracted characters into text
A method of generating metadata based on a web page. The method of claim 1,
The step of extracting a valid text from a part of the extracted text,
removing stopwords from the extracted text; and removing stopwords from the extracted text.
A method of generating metadata based on a web page. The method of claim 1,
The step of determining a part of the extracted valid text as a key keyword,
Separating the valid text into word units, calculating the importance of the separated words, and determining a key word selected according to the order of high importance as the key keyword
A method of generating metadata based on a web page. 5. The method of claim 4,
The step of determining a part of the extracted valid text as a key keyword,
Separating the valid text into sentence units, calculating the importance of the separated sentences, and determining a key sentence selected according to the order of increasing importance as the core keyword.
A method of generating metadata based on a web page. 6. The method of claim 5,
The metadata is
A title metadata object included in a title section used as a title of the web page, a description metadata object of a description section used as a description for the web page, and a tag of a tag word section used as a tag word of the web page which contains a metadata object
A method of generating metadata based on a web page. 7. The method of claim 6,
The step of generating the metadata includes:
setting at least one core sentence among the core sentences as the title metadata object according to the position or size of characters in the image that is the basis of the core sentence
A method of generating metadata based on a web page. 7. The method of claim 6,
The step of generating the metadata includes:
and setting at least a part of the key sentences as the description metadata object.
A method of generating metadata based on a web page. 7. The method of claim 6,
The step of generating the metadata includes:
extracting the k-th key words according to the order of importance of the key words; and setting the extracted key words as the tag word metadata entity.
A method of generating metadata based on a web page. The method of claim 1,
Further comprising the step of providing a user editing interface for the generated metadata
A method of generating metadata based on a web page.

Images