KR102120465B1 - Method of providing comprehensive product information - Google Patents

Abstract

The method for providing comprehensive product information is performed by a comprehensive product information providing server connected to a plurality of shopping mall sites. The method for providing comprehensive product information may include driving a first crawler to select and access one of the plurality of shopping malls; Extracting basic information of a product from the selected shopping mall; Storing the extracted basic information in a database; Driving a second crawler to access the detailed description page URL address of the product among the basic information of the product stored in the database; Extracting option information of a product from a hypertext markup language (HTML) source of a page corresponding to the product description page URL address; And matching and storing the extracted option information with the basic information of the product.

Description

How to provide general product information{METHOD OF PROVIDING COMPREHENSIVE PRODUCT INFORMATION}

The present invention relates to a technology for providing comprehensive product information, and more particularly, to a method for providing comprehensive product information by providing products (or product information) provided in various shopping malls in a single open market.

Personal online shopping malls are active markets through websites as of 2006. The market has grown to 15 trillion won (as of 2015) through steady growth. And online shopping malls started as websites are using open markets (Kupang, Weimef, 11th Street, etc.) to adapt to the market that is rapidly changing from the PC-based market to the mobile market.

However, online shopping malls have to go through a cumbersome upload process in order to sell products in the open market. Shopping malls must upload again to register products already uploaded to their site in the open market. That is, multiple upload processes are required to expand the sales channel. In order to adapt to the smartphone era, online shopping malls are inevitably using the mobile open market as a sales channel, but they cannot upload all products, and because of high fees (average 15%), they are a window to sell promotional and carryover products. I use it to the extent.

Korean Registered Patent No. 1,717,063 (announced on March 17, 2017) "Web crawling device and web crawling method thereby" Korean Patent Publication No. 2016-0046662 (published on April 29, 2016) "How to collect the lowest price information of the Internet shopping mall and how to update the product information using the collected lowest price information"

The present invention has been devised to solve the above-described problems of the open market (that is, cumbersome upload), and provides an open market without a product upload process, that is, a method for providing comprehensive product information.

In order to achieve the above object, a method for providing comprehensive product information according to embodiments of the present invention may be performed in a comprehensive product information providing server connected to a plurality of shopping mall sites. The method for providing comprehensive product information may include driving a first crawler to select and access one of the plurality of shopping malls; Extracting basic information of the product from the selected one shopping mall-the basic information includes identification information of the product used in the selected one shopping mall, name information of the product, price information of the product, and details of the product Description page Contains uniform resource locator (URL) address information, image URL address information of the product, category information of the product, and out-of-stock information of the product; Storing the extracted basic information in a database; Driving a second crawler to access the detailed description page URL address of the product among the basic information of the product stored in the database; Extracting option information of a product from a hypertext markup language (HTML) source of a page corresponding to the product description page URL address; And matching and storing the extracted option information with the basic information of the product. Here, the option information may include color information of the product, size information of the product, additional price information for each option of the product, and information about whether the product is sold out by option.

According to an embodiment, the step of extracting the basic information of the product may include: identifying categories on a main page of the selected shopping mall; And identifying the basic information of the product on a page corresponding to one selected from the categories.

According to an embodiment, extracting the option information of the product may include: extracting first option values of the first option; Grasping second option values of a second option dependent on the first option by inputting one of the first option values to the first option; And extracting additional price information and out-of-stock information by inputting second option values in the second option.

According to an embodiment, the step of accessing the selected one of the plurality of shopping malls may include determining whether there is a record of crawling the selected one of the plurality of shopping malls within a first cycle. ; When there is a record that the crawling operation is performed within a first cycle, terminating the crawling operation for the selected one of the plurality of shopping malls; And selecting one different from the selected one of the plurality of shopping malls.

According to an embodiment, the step of extracting the basic information of the product may include: periodically determining whether a working time for extracting the basic information exceeds a reference working time; And when the working time exceeds the reference working time, terminating the extraction of the basic information.

According to embodiments of the present invention, the method for providing comprehensive product information (or crawling method) collects product information from various shopping malls and builds a database, thereby allowing shopping malls (or sellers of shopping malls) to upload separately. Allows users to sell products in the open market market (or mobile open market market) without a process (ie, product information upload process), and also enables users to check and purchase products in shopping malls in real time. Can be.

In addition, the method for providing comprehensive product information prevents excessive traffic from the shopping mall site due to the crawling operation by limiting the crawling operation time, and determines whether or not a history of the crawling operation exists (or crawling operation within a first cycle). By determining whether it is performed), it is possible to prevent the indiscriminate crawl operation.

1 is a view showing a system for providing comprehensive product information according to embodiments of the present invention.
FIG. 2 is a view for explaining the operation of the synchronization robot included in the comprehensive product information providing system of FIG. 1.
3A is a diagram illustrating an example of product information provided through a shopping mall.
3B is a block diagram showing the structure of product information obtained through a shopping mall.
4 is a flowchart illustrating a method for providing comprehensive product information performed in the comprehensive product information providing system of FIG. 1.
5A is a diagram illustrating an example of product information provided through a shopping mall.
5B is a diagram showing an example of basic information of a product extracted from a shopping mall.
6 is a diagram showing an example of product information stored in a database.
7 is a view showing an example of a configuration for obtaining product information for each shopping mall.
8 is a diagram illustrating an example of a product detail page provided in a shopping mall.
9 is a diagram illustrating an example of an HTML source extracted from the page of FIG. 8.
FIG. 10A is a flowchart illustrating a method of extracting option information when the option information is included in the HTML source of FIG. 9.
FIG. 10B is a flowchart illustrating a method of extracting option information when the option information is not included in the HTML source of FIG. 9.
11 is a view showing an example of a crawling operation sequence of a synchronization robot.
12 is a block diagram illustrating an example of a system for providing comprehensive product information of FIG. 1.
13 is a flowchart illustrating a method for monitoring a synchronization robot in the system for providing comprehensive product information of FIG. 12.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The same reference numerals are used for the same components in the drawings, and duplicate descriptions for the same components are omitted.

The method for providing the comprehensive product information (or the crawling method and the comprehensive product information providing system) may collect product information registered on the web pages of shopping malls and make a database (or DB). The comprehensive product information providing method collects product information of shopping malls by using an information retrieval technique called crawling, repeatedly collects product information using regular expressions, and repeatedly processes the product information into data, An open market (or sales channel, comprehensive product information providing server) database can be built.

Therefore, product information of shopping malls is synchronized in real time and provided through an open market, and users (or buyers) can easily check products of several shopping malls at once and easily purchase products. The sellers of the products (or the operators of the shopping mall) receive notification information about the products sold through the open market, check the sale products through the manager page of the open market provided separately, and deliver the products. You can do Therefore, not only can sellers easily provide product information of the corresponding shopping mall to users through an open market, but users can also receive more various product information and obtain products that meet more user preferences.

Meanwhile, some terms used in the present invention may be defined as follows.

Database synchronization (or DB synchronization) may correspond to an act of databaseizing product information registered on a web page (or a shopping mall homepage).

The synchronization robot (or crawler) may correspond to a subject that automatically performs DB synchronization and automatically uploads it to an open market (or a sales channel, a server for providing comprehensive product information).

The open market (or sales channel, server for providing comprehensive product information) corresponds to an open market built using DB synchronization, or a platform where users (or buyers) can check product information of various shopping malls at once. And may be implemented as a mobile application.

A shopping mall (or online shopping mall) may correspond to a seller who has built a shopping mall using a web page, and a user (or a buyer, a user) may be a subject who purchases and checks products of online shopping malls using an open market. have.

The shopping mall manager page may be a web page where a user can check a product (or a sale product) purchased in the open market.

1 is a view showing a system for providing comprehensive product information according to embodiments of the present invention, and FIG. 2 is a view for explaining the operation of a synchronization robot included in the system for providing comprehensive product information in FIG. 1. 3A is a diagram illustrating an example of product information provided through a shopping mall, and FIG. 3B is a block diagram showing a structure of product information acquired through a shopping mall.

1 and 2, the comprehensive product information providing system 100 may include a plurality of online shopping mall sites 110, a comprehensive product information providing server 120 and a user terminal 130. The plurality of online shopping mall sites 110, the comprehensive product information providing server 120, and the user terminal 130 may be interconnected through a wired/wireless network.

The comprehensive product information providing server 120 may include a synchronization robot 121 and a database (DB) 122, collect product information using the synchronization robot 121, and build a DB 122. Product information included in the constructed DB may be provided to the user terminal 130.

The synchronization robot 121 checks each web page of several shopping malls (or a plurality of shopping malls), and when a new product is confirmed (ie, new product information is uploaded) or a change in product type is confirmed (ie, When the conventional product information is changed), the product information can be uploaded/stored or updated in the DB 122 of the open market (or the comprehensive product information providing server 120).

In this case, as shown in FIG. 3A, various product information of various shopping malls is provided through an open market, and a user can easily check and purchase the product. As described above, the sale product is checked on the shopping mall manager page, and through this, the seller can send the sale product to the user.

In embodiments, the synchronization robot 121 may obtain basic information of a product and product options (or option information) on a web page of a shopping mall.

The synchronization robot 121 may obtain a basic information of a product by performing a URL request (or a request for providing information based on the URL address) to the shopping mall. For example, the information provided in the shopping mall, as shown in FIG. 3B, is classified stepwise into category type information (IF1), product type, product-specific basic information (IF2), and product-specific option information (IF3). Can be. The basic information of the product may be obtained from a page provided when the “category” type of the web page of the shopping mall is clicked, as shown in FIG. 3B.

Meanwhile, the synchronization robot 121 may acquire first to eighth basic information through the HTML source of the corresponding page. The first to eighth basic information includes product identification information (or ID) of a shopping mall, product name information, product price information, product detail page URL information, product detail mobile page URL information, product image URL information, product It may include category information (for example, outerwear, jacket, etc.), information about the absence of products.

For reference, most shopping malls in the same field use similar URL sources, but product name information, out of stock information, and the like may be provided differently for each shopping mall. Therefore, the comprehensive product information providing server can perform code optimization for each shopping mall.

4 is a flowchart illustrating a method for providing comprehensive product information performed in the comprehensive product information providing system of FIG. 1. 5A is a diagram showing an example of product information provided through a shopping mall, FIG. 5B is a diagram showing an example of basic information of a product extracted from a shopping mall, and FIG. 6 is an example of product information stored in a database It is a drawing. 7 is a view for explaining the operation of the synchronization robot included in the comprehensive product information providing system of FIG. 1.

1 to 4, the method of FIG. 4 may be performed by the synchronization robot 121 (or the general product information providing server 120 ).

The method of FIG. 4 can sort the categories of the shopping mall (S410). For example, through analysis of URL information such as “http://0000.com/product/list.html?cate_n=41”, “cate_no=41” is “outer” and “cate_no=52” By defining it as an outer, you can sort the categories.

Thereafter, the method of FIG. 4 may select one category from a plurality of categories and extract an HTML source of a product list page included in the selected category (S420). For example, the method of FIG. 4 may extract HTML sources for all pages (eg, 1 page to last page) without limiting the number of pages.

The method of FIG. 4 may separate product information from the HTML source extracted through the regular expression (S430). Here, the regular expression is a formal language used to express a set of strings having specific rules, and may be a regular expression used for searching and replacing strings in general text editors and programming languages.

The method of FIG. 4 may create a regular expression in response to information to be collected (for example, basic information of a product) (S440). For example, the regular expression for product name information is “\<cp class=\"name\"\>(.*?)\<Vp\>\”, and the regular expression for product price information is “ /([0-9]{1.3},)?( [0-9]{1.3},)?[0-9]{3}won/”.

The method of FIG. 4 may acquire basic information of a product through a regular expression, and process the obtained basic information in a desired form (or a standard form) (S450).

Thereafter, the method of FIG. 4 may store and/or update the processed basic information in a database (S460).

4 to 6, product information provided in a shopping mall as shown in FIG. 5A may be extracted through a regular expression as shown in FIG. 5B, processed, and stored in a database as shown in FIG. 6. .

4 to 7, the synchronization robot 121 may set a category and a regular expression for each shopping mall, and obtain product information (or basic information of the product) based on the set category and the regular expression. . For example, in the case of the first shopping mall, the synchronization robot 121 may define the first category types and the first regular expression. For another example, in the case of the second shopping mall, the synchronization robot 121 may define second category types and a second regular expression.

That is, the synchronization robot 121 may define an optimized code for each shopping mall and drive a frame capable of continuously collecting/processing product information to database product information.

In embodiments, the synchronization robot 121 may synchronize (or crawl) product option information (or product option information) in addition to the basic information of the product. Meanwhile, the collection of the basic information of the product and the collection of the product option information may be performed by different synchronization robots, for example, by the basic information synchronization robot (or, the first synchronization robot, the first crawler). Basic information is collected, and product option information may be collected by a product option synchronization robot (or a second synchronization robot and a second crawler).

8 is a diagram showing an example of a product detail page provided in a shopping mall, and FIG. 9 is a diagram showing an example of an HTML source extracted from the page of FIG. 8. FIG. 10A is a flowchart illustrating a method of extracting option information when the HTML source of FIG. 9 includes option information, and FIG. 10B is a method of extracting option information when the option information is not included in the HTML source of FIG. 9 It is a flow chart showing.

Referring to FIG. 8, detailed information of a product may be displayed through a corresponding page. For example, the corresponding page may be a page provided in the shopping mall when the corresponding product is clicked in the shopping mall.

In this case, the product option synchronization robot (or the synchronization robot 121) may acquire the first to fifth option information through the HTML source of the corresponding page (ie, the product detail page). Here, the first option information is a first option of the product, and may be, for example, a color, a kind of product, and the like. The second option information is a second option of the product, and may be, for example, the size of the product. The third option information may be a third option of the product. The fourth option information may be additional price information for each product option, and the fifth option information may be out of stock information for each product option.

Meanwhile, option information of the product may not be included in the HTML source, and in this case, the product option synchronization robot may obtain option information through a URL request to the shopping mall.

The method of obtaining option information may be performed on a shopping mall in which basic information is collected. That is, among the collected basic information, the “product detail page URL” is called, and the product option synchronization robot can access the corresponding URL to collect option information.

For example, in the case of a commodity option, a unique number may be given for each number of options 1, 2, and 3. For example, for products by color and size, such as "P000RZE000A = Red/32, P000RZE000B = Red/30, P000ZEQ001A = Red/28, P002RZQ000A = Blue/32, P000TEW000C = Blue/30, P000UBP000A = Blue/28" Unique numbers can be assigned.

Additional price information and out of stock information (option_stock_data) and option type information (option_value_mapper) may be previously provided according to the unique numbers. For example, in the case of additional price information and out-of-stock information (option_stock_data), it indicates additional price and out-of-stock by unique number, and in the case of option type information (option_value_mapper), it indicates the type of option 1, option 2 and option 3 by unique number. Can be.

Unique number Option_stock_data Option_value_mapper P000RZE000A Stock_price=0, use_soldout=F Red, 32 P000RZE000B Stock_price=0, use_soldout=F Red, 30 P000ZEQ001A Stock_price=0, use_soldout=T Red, 28 P002RZQ000A Stock_price=1000, use_soldout=T Blue, 32 P000TEW000C Stock_price=1000, use_soldout=T Blue, 30 P000UBP000A Stock_price=1000, use_soldout=F Blue, 28

Table 1 may indicate option information for each unique number. Here, stock_price may indicate an additional price, and use_soldout may indicate the absence or absence of stock (however, T is out of stock).

On the other hand, as described above, when the option information is not visible in the HTML source, the product option synchronization robot may obtain option information using a URL request method.

For example, the HTML source of the first option check box in the page may be displayed as shown in (a) of FIG. 9.

Here, if the option value of the first option (ops1) is requested by substituting the URL for the second option (ops2), the second option may be output as shown in FIG. 9(b). In addition, when the corresponding product is out of stock, "out of stock" may be output as an option lamp.

In addition, if the request is made by substituting the URL for the detailed option information up to the option value of the second option (ops2), the additional price when the first option and the second option are selected as shown in FIG. Can be output.

That is, when the option information is included in the HTML source, the synchronization robot may acquire the option information according to the method of FIG. 10A, and when the option information is not included in the HTML source, the synchronization robot uses the method of FIG. 10B. Accordingly, option information can be obtained.

Specifically, the method of FIG. 10A may call the URL of the product detail description page (S110), extract the HTML source of the corresponding page (S1020), and determine the type and number of options included in the HTML source (S1030). . Thereafter, the method of FIG. 10A may extract additional price information and sold out information for each option (S1040), and store the extracted option information in the DB 122 (S1050).

Similarly, the method of FIG. 10B may call the URL of the detailed description page (S110), and extract the HTML source of the page (S1020). However, since the option information is not included in the HTML source, the value for each option of the first option (ops1) is extracted through a regular expression (S1060), and the option value of the first option (ops1) in the option information URL ( value) to check/acquire additional price information for each option and information on whether or not sold out (S1070). For example, the method of FIG. 10B extracts additional price information by substituting the value of the first option (ops1) and the second option (ops2) in the option information URL, and thereafter, the second option (ops2) information. By substituting the option-specific value of the first option (ops1) in the URL, the value of the second option (ops2) can be extracted, and information on whether or not it is out of stock can be checked/obtained. Thereafter, the method of FIG. 10B may store the extracted option information in the DB 122 (S1050).

Through the process of FIGS. 4 to 10B, the method for providing comprehensive product information (or the synchronization robot 121 and the comprehensive product information providing server 120) includes product information (ie, basic information as well as option information) in various shopping malls. By collecting, you can build a database. Accordingly, shopping malls (or sellers of shopping malls) can sell products in an open market market (or a mobile open market market) without a separate upload process (ie, a process of uploading product information), and a user can store a shopping mall in real time. It may be possible to check their products and make an integrated purchase.

11 is a view showing an example of a crawling operation sequence of a synchronization robot.

1 and 11, the comprehensive product information providing server 120 may include a plurality of synchronization robots 1110 and 1120 (or crawling robots and crawlers). For example, the comprehensive product information providing server 120 may include first and second synchronization robots 1110 and 1120.

The first and second synchronization robots 1110 and 1120 may crawl basic information and option information of the product.

The first synchronization robot 1110 sequentially performs crawling operations for the A shopping mall, the B shopping mall, the C shopping mall, and the D shopping mall, and when the crawling operation for the D shopping mall is completed, may perform the crawling operation for the A shopping mall again. have.

In one embodiment, the first synchronization robot 1110 sequentially obtains the basic information of the product by sequentially performing the crawling operations for the A shopping mall, the B shopping mall, the C shopping mall, and the D shopping mall, and when the crawling operation for the D shopping mall is completed. , Again, the C shopping mall for the A shopping mall, the B shopping mall, the C shopping mall, and the D shopping mall may obtain option information of the product. That is, when the crawling operation of the basic information and the option information of the product is continuously performed, excessive traffic to the corresponding shopping mall may occur, so the first synchronization robot 1110 collects all the basic information of the product from the shopping malls. After that, detailed information of the product may be sequentially obtained from shopping malls.

Similar to the first synchronization robot 1110, the second synchronization robot 1120 sequentially performs crawling operations for the E shopping mall, the F shopping mall, the G shopping mall, and the H shopping mall, and when the crawling operation for the H shopping mall is completed, It is possible to perform a crawl operation for the E shopping mall again. Also, after the second synchronization robot 1120 collects all the basic information of the products from the shopping malls, the detailed information of the products from the shopping malls can be sequentially obtained.

On the other hand, A to D shopping malls are allocated to the first synchronization robot 1110, and E to H shopping malls are allocated to the second synchronization robot 1120, but the first and second synchronization robots 1110 and 1120 are shown. It is not limited to this. For example, the first synchronization robot 1110 sequentially crawls from shopping mall A to shopping mall H, and the second synchronization robot 112 sequentially crawls from shopping mall E to shopping mall H, and from shopping mall A to shopping mall D. You can do it. Also, schedules for the first and second synchronization robots 1110 and 1120 may be dynamically allocated through calculation/prediction of the crossing workload.

12 is a block diagram illustrating an example of the system for providing comprehensive product information of FIG. 1, and FIG. 13 is a flowchart illustrating a method for monitoring a synchronization robot in the system for providing comprehensive product information of FIG. 12.

1 and 12, the comprehensive product information providing server 120 may further include a monitoring module 123 for monitoring the synchronization robot.

Referring to FIG. 13, the monitoring module 123 is automatically driven when a URL is received from the database 122 and interlocked with the synchronization robot 121, delivers the URL to the synchronization robot 121 (S1310), and the synchronization robot ( 121).

Thereafter, the monitoring module 123 may store PID information (or process information) and a start time of the synchronization robot 121 as variables and files (S1320).

The monitoring module 123 may determine whether the working time of the synchronization robot 121 has passed the first reference working time (S1330). Here, the first reference working time is a maximum working time given to the monitoring module 123 and may be preset in consideration of an increase in the load of the shopping mall site due to the crawling operation.

If the working time of the synchronization robot 121 has not passed the first reference working time, the monitoring module 123 may periodically determine whether the working time of the synchronization robot 121 has passed the first reference working time. .

Meanwhile, if the working time of the synchronization robot 121 has passed the first reference working time, the monitoring module 123 may forcibly terminate the synchronization robot 121 (S1340). For example, the monitoring module 123 may use the pre-stored PID information to generate a forced termination command for the synchronization robot 121 to end the crawling process matching the PID information. For example, the crawling operation for the first shopping mall may be terminated. Thereafter, a second shopping mall different from the first shopping mall is selected, and a crawling operation for the second shopping mall may be performed.

Meanwhile, the synchronization robot 121 may receive a URL from the monitoring module 123 (S1350), and determine whether the same crawl (for example, crawling for the same shopping mall site) is performed within the first cycle ( S1360). That is, it may be determined whether or not there is a history in which the corresponding crawling operation is performed within the first cycle.

If the same crawl record does not exist, the synchronization robot 121 performs the corresponding crawl (S1370) and periodically determines whether the second working time has passed (S1380).

Meanwhile, the synchronization robot 121 terminates the corresponding crawling operation when the same crawling record exists or when a second working time has elapsed, or when a forced termination command is received from the monitoring module 123 (S1390), It can be transmitted to the monitoring module 123 that the crawl operation has been completed. In this case, the monitoring module 123 may delete pre-stored files (ie, PID information and start time files) (S1400).

That is, the monitoring module 123 may prevent the synchronization robot 121 from falling into an infinite loop and continuously performing a crawling operation, thereby preventing a load (eg, excessive traffic) from being generated in the corresponding shopping mall site. In addition, by setting a work cycle for the crawling task (that is, by limiting repetitive crawling tasks within the work cycle), indiscriminate crawling by the synchronization robot 121 can be prevented.

The present invention can be applied to a crawling method, a crawling system, and an information providing system.

As described above, although described with reference to embodiments of the present invention, those of ordinary skill in the art variously vary the present invention without departing from the spirit and scope of the present invention as set forth in the claims below. It will be understood that modifications and changes can be made.

100: comprehensive product information provision system
110: online shopping mall site
111, 112, 113: 1st to 3rd shopping malls
120: comprehensive product information providing server
121: synchronization robot
122: DB
123: monitoring module
710: frame
1110: first synchronization robot
1120: second synchronization robot

Claims (5)

In a method for providing comprehensive product information performed by a comprehensive product information providing server connected to a plurality of shopping mall sites,
Driving a first crawler to select and access one of the plurality of shopping malls;
Extracting basic information of the product from the selected one shopping mall through the first crawler-the basic information includes identification information of the product used in the selected one shopping mall, name information of the product, and price of the product Information, a detailed description page of the product, including uniform resource locator (URL) address information, image URL address information of the product, category information of the product, and out-of-stock information of the product;
Storing the extracted basic information in a database;
After storing all the basic information of the products of each of the plurality of shopping malls in the database, a second crawler is driven to access the URL of the detailed description page of the product among the basic information of the products stored in the database step;
Extracting option information of a product from a hypertext markup language (HTML) source of a page corresponding to the product description page URL address through the second crawler; And
And matching and storing the extracted option information with the basic information of the product,
The option information includes color information of the product, size information of the product, additional price information for each option of the product, and information about whether the product is sold out by option,
Extracting the basic information of the product,
Identifying categories on the main page of the selected shopping mall; And
And identifying the basic information of the product on a page corresponding to one selected from the categories,
Extracting the option information of the product,
Extracting first option values of a first option from the HTML source corresponding to the product description page URL address;
Identifying the second option values of the second option dependent on the first option by performing a URL request by substituting one of the first option values of the first option in the product description page URL address-the URL Request is to request the provision of information based on the URL address-; And
And extracting additional price information and out of stock information by performing a URL request by substituting the second option values of the second option to the product description page URL address,
Step of accessing the selected one of the plurality of shopping malls,
Determining whether there is a record of performing a crawling operation for the selected one of the plurality of shopping malls within a first cycle through the first crawler; And
If there is a record of the crawl operation performed within a first cycle, the first crawler ending the crawl operation for the selected one of the plurality of shopping malls; And
And selecting a different one from the selected one of the plurality of shopping malls through the first crawler.
Extracting the basic information of the product,
Periodically determining whether a working time for the first crawler to extract the basic information exceeds a reference working time through a monitoring module-the reference working time increases the load of the selected one of the plurality of shopping malls Pre-established in consideration of; And
And when the working time exceeds the reference working time, terminating the extraction operation of the basic information of the first crawler through the monitoring module. delete delete delete delete

Images