KR102352041B1 - Web scraping system with hierarchical abstraction authentication function and method therefor - Google Patents

Abstract

The present invention relates to a web scraping system having a hierarchical abstraction authentication function. In accordance with the present invention, a certificate and a certificate management part are installed in a user terminal, and a scraping module is installed in an information collection server separately from the certificate and the certificate management part of the user terminal, and, as a result, the web scraping system is capable of easily updating the scraping module, and reducing errors as well as preventing the forgery of a scraping result to increase reliability. The web scraping system includes an information collection server connected with a plurality of institution servers through the Internet, and a user terminal connected with the information collection server, and the system has a structure in which a certificate and a certificate management part are installed in the user terminal, and a scraping module is installed in the information collection server separately from the certificate and the certificate management part of the user terminal.

Description

Web scraping system with hierarchical abstraction authentication function and method therefor

The present invention relates to a web scraping system having a hierarchical abstraction authentication function and a method therefor. More specifically, a certificate and a certificate management unit are installed in a user terminal, and a scraping module is installed in an information collection server. A web scraping system with a hierarchical abstraction authentication function that can easily update the scraping module by being installed separately from the It's about the way.

Web scraping is a technology that automatically connects to a networked Internet system, displays data on the screen, and extracts only necessary data. It can be viewed from time to time, used as data for comparative analysis, and new data can be created by processing stored data. In particular, it is an essential program for Internet banking and is actively operated by each financial institution. Users can click to obtain information such as reward programs such as mileage for hotels, airlines, car rentals, and gas stations, integrated e-mail inquiry, news, chat, and weather. You can use it wherever you are.

Scraping has become common worldwide since the late 1990s, and a representative example of this is Personal Finance Management Software (PFMS), which can integrate and manage individual accounts of several financial institutions. It has the advantage of saving time and money by not having to visit each website individually, but there are also disadvantages that it is concerned about invasion of privacy and that the programming needs to be revised every time the site is updated.

1 is a block diagram of a conventional web scraping system.

As shown in FIG. 1, the conventional web scraping system includes a user terminal 3 connected to a plurality of agency servers 11 to 1n and the Internet 2 through the Internet 2, and the user terminal 3 includes a certificate. 31 and the certificate management unit 32 are installed, and the scraping module 33 is installed together with the certificate 31 and the certificate management unit 32 .

The operation of the conventional web scraping system by the above configuration is as follows.

When the scraping module 33 of the user terminal 3 accesses the institution servers 11 to 1n through the Internet 2 and requests a specific screen address, the institution servers 11 to 1n send the HTTP response (HTML source) of the corresponding screen. ), but if it is a screen that requires member authentication, authentication is required.

When a digital signature phrase is issued from the authority server 11 to 1n to the certificate management unit 32 of the user terminal 3 for authentication, the certificate management unit 32 of the user terminal 3 performs the electronic signature and then the digital signature value is transmitted to the institution servers 11 to 1n, and the institution servers 11 to 1n respond to the authentication result to the certificate management unit 32 of the user terminal 3, thereby completing the authentication procedure.

After the authentication process is completed, when the scraping module 33 of the user terminal 3 sends the screen URL to be scraped to the institution servers 11 to 1n, since the authentication process is completed, the institution servers 11 to 1n) sends the HTTP response (HTML source) of the corresponding screen down to the scraping module 33 of the user terminal 3, and the user terminal 5 receives it, thereby performing the scraping extraction operation.

2 is another configuration diagram of a conventional web scraping system.

As shown in Fig. 2, the conventional web scraping system is connected to a plurality of agency servers 11 to 1n and an intermediary server 4 connected through the Internet 2, and the intermediary server 5. A user terminal 5 is included, and the intermediate server 4 has a certificate 41 and a certificate management unit 42 installed, and a scraping module 43 is installed along with the certificate 41 and the certificate management unit 42 . It consists of an installed structure.

The operation of the conventional web scraping system by the above configuration is as follows. ,

In response to the scraping request of the user terminal 5, the scraping module 43 of the server 4 accesses the institution servers 11 to 1n through the Internet 2 and requests a specific screen address, the institution servers 11 to 1n) sends the HTTP response of the corresponding screen down. At this time, if the screen requires member authentication, authentication is required.

For authentication, when a digital signature text is issued from the authority server 11 to 1n to the certificate management unit 42 of the server 4, the certificate management unit 42 of the server 4 makes a digital signature and then sends the digital signature value to the authority. It is transmitted to the servers 11 to 1n, and the institution servers 11 to 1n respond to the authentication result to the certificate management unit 42 of the server 4, thereby completing the authentication procedure.

After the authentication process is completed, when the scraping module 43 of the server 4 sends the screen URL to be scraped to the institution servers 11 to 1n, since the authentication process is completed, the institution servers 11 to 1n The HTTP response of the corresponding screen is sent down to the scraping module 43 of the server 4, and the user terminal 5 receives it, thereby performing the scraping extraction operation.

In addition, as a technology related to a conventional web scraping system, the web browser is installed in a client device, a user certificate that can access a target server having scraping target information is stored, and accepts a user's scraping request; a scraping engine for receiving a scraping request from the web browser; a relay server (WSGS) that generates information (messages) that satisfy the requirements of the target server for scraping; and receiving a scraping request and a message from the relay server, accessing the target server with an IP different from the IP of the relay server for scraping, and transferring the scraped data to the relay server, each having a plurality of security gateway (SG); A configuration including a scraping management server (SMS) that manages IPs and operation status of the plurality of security gateways is disclosed in Korean Patent Publication No. 10-2179792 (November 17, 2020) "Web browser-based scraping system and methods".

However, in the conventional web scraping system including the aforementioned Patent Registration No. 10-2179792, since the scraping module is installed together with the certificate and the certificate management unit, the scraping module management cost is generated when the scraping module is frequently updated. Also, in the case of a smartphone, there is a problem in that it becomes slow or an error occurs depending on the CPU or network conditions depending on the scraping execution environment.

In addition, the conventional web scraping system is not only weak in security because it has to transmit a customer's certificate and certificate password to an authority server, but also has a cumbersome problem in that it is necessary to obtain the user's consent for sending the certificate.

In addition, the conventional web scraping system has a problem in that reliability is lowered because the scraping result can be forged or forged in the user terminal.

Registered Patent Publication No. 10-2179792

An object of the present invention is to solve the problems of the prior art as described above. A certificate and a certificate management unit are installed in the user terminal, and the scraping module is installed separately from the certificate and certificate management unit of the user terminal in the information collection server. It is to provide a web scraping system with hierarchical abstraction authentication function that can easily update a module, reduce the occurrence of errors, and increase reliability by preventing forgery and falsification of the scraping result.

As a means for achieving the above object, the configuration of the system of the present invention includes an information collection server connected to a plurality of institution servers and the Internet through the Internet, and a user terminal connected to the information collection server, wherein the user terminal includes It is preferable that a certificate and a certificate management unit are installed, and a scraping module is installed in the information collection server separately from the certificate and certificate management unit of the user terminal.

The configuration of the system of the present invention is, the user terminal requests scraping from the information collection server and at the same time requests the electronic signature phrase, and requests the electronic signature phrase by visiting the institution server by the information collection server, and the organization server The electronic signature text is issued from the information collection server to the user terminal, and the electronic signature is requested while the electronic signature text is transmitted from the information collection server to the user terminal. and transmits the digital signature value from the information collection server to the organization server, responds to the authentication result as a session key from the organization server to the information collection server, and starts the scraping extraction operation from the information collection server to the organization server, It is preferable to transmit the scraping extraction result from the institution server to the information collection server and process the scraping extraction result in the information collection server.

As a means for achieving the above object, the configuration of the method of the present invention comprises the steps of requesting a digital signature phrase while requesting scraping from the user terminal to the information collection server, and visiting the institution server by the information collection server. The steps of requesting a signature phrase, the step of issuing the electronic signature phrase from the institution server to the information collection server, the step of requesting an electronic signature while transmitting the electronic signature phrase from the information collection server to the user terminal; Decrypting the encrypted private key to digitally sign the certificate with a password, transmitting the digital signature value from the user terminal to the information collection server, and transmitting the digital signature value from the information collection server to the organization server; A step of responding to the authentication result as a session key from the server to the information collection server, starting a scraping extraction operation from the information collection server to the organization server, and transmitting the scraping extraction result from the organization server to the information collection server; It is preferable to include the step of processing the scraping extraction result in the information collection server.

Another configuration of the method of the present invention as a means for achieving the above object includes the steps of making a loan review request to a financial institution server by a user terminal, and scraping health insurance payment details from the financial institution server to an information collection server The requesting step, the step of requesting the electronic signature text from the information collection server to the Health Insurance Corporation server, the step of issuing the electronic signature text from the Health Insurance Corporation server to the information collection server, and the electronic signature from the information collection server to the user terminal A step of requesting an electronic signature while transmitting the phrase, decrypting the encrypted private key in the certificate with the certificate password at the user terminal to make a digital signature, and transmitting the digital signature value from the user terminal to the information collection server; The steps of transmitting the digital signature value from the information collection server to the Health Insurance Corporation server, the steps of responding as a session key to the authentication result from the Health Insurance Corporation server to the information collection server, and the steps from the information collection server to the Health Insurance Corporation server for health insurance Initiating scraping extraction work on payment details, transmitting the health insurance payment details as a scraping extraction result from the Health Insurance Corporation server to the information collection server, and transmitting health insurance payment details from the information collection server to the financial institution server and transmitting the loan examination result from the financial institution server to the user terminal according to the health insurance payment details.

The configuration of the method of the present invention preferably further includes the step of adding an authentication confirmation code such as year, month, date and time, and an authentication phrase to the scraping extraction result in the information collection server.

In this invention, the user terminal has a certificate and certificate management unit installed, and the information collection server has the scraping module installed separately from the certificate and certificate management unit of the user terminal, so that the scraping module can be easily updated, and the occurrence of errors can be reduced. Not only that, it has the effect of increasing reliability by preventing forgery of the scraping result.

1 is a block diagram of a conventional web scraping system.
2 is another configuration diagram of a conventional web scraping system.
3 is a block diagram of a web scraping system having a hierarchical abstraction authentication function according to an embodiment of the present invention.
4 is a flowchart of a web scraping method having a hierarchical abstraction authentication function according to an embodiment of the present invention.
5 is a flowchart of a non-face-to-face loan examination method using a web scraping method having a hierarchical abstraction authentication function according to another embodiment of the present invention.

Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings in order to describe in detail enough that a person of ordinary skill in the art can easily carry out this invention. Other objects, features, and operational advantages of this invention, including the object, action, and effect, will become clearer by the description of the preferred embodiment.

For reference, the embodiments disclosed herein are only presented by selecting and presenting the most preferred embodiments in order to help those skilled in the art understand from among various possible examples, and the technical spirit of the present invention is not necessarily limited or limited only by the presented embodiments. No, various changes, additions, and modifications including equivalents or substitutes are possible without departing from the technical spirit of the present invention.

In addition, the terms or expressions used in the specification and claims of the present application are defined based on the principle that the inventor can appropriately define the concept of the term in order to best describe his invention. , should not be construed as being limited only to the ordinary or dictionary meaning, but should be interpreted as meaning and concept consistent with the technical idea of this invention. As an example, the singular expression includes a plural expression unless the context clearly indicates otherwise, and the expression regarding the direction is set based on the position expressed on the drawing for convenience of description, and is "connected" or "connected" The expression "to be" includes not only a direct connection or connection, but also a connection or connection through another element in the middle. In addition, the expression "~ server" or "~ part" includes a unit realized using hardware, a unit realized using software, a unit realized using both hardware or software, etc., and one unit is 1 It may be realized using two or more pieces of hardware or software, and two or more units may be realized using one piece of hardware or software.

3 is a block diagram of a web scraping system having a hierarchical abstraction authentication function according to an embodiment of the present invention.

As shown in FIG. 3, the configuration of the scraping system having a hierarchical abstraction authentication function according to an embodiment of the present invention collects information connected through a plurality of agency servers 11 to 1n and the Internet 2 a server 6 and a user terminal 7 connected to the information collection server 6, wherein a certificate 71 and a certificate management unit 72 are installed in the user terminal 7, the information The collection server 6 has a structure in which the scraping module 61 is installed separately from the certificate 71 and the certificate management unit 72 of the user terminal 7 .

4 is a flowchart of a web scraping method having a hierarchical abstraction authentication function according to an embodiment of the present invention.

As shown in FIG. 4 , the web scraping method having a hierarchical abstraction authentication function according to an embodiment of the present invention requests scraping from a user terminal to an information collection server and at the same time requesting a phrase for an electronic signature (S10) ), requesting the electronic signature phrase by visiting the institution server by the information collection server (S20), issuing the electronic signature phrase from the institution server to the information collection server (S30), and the user terminal from the information collection server Step (S40) of requesting an electronic signature while transmitting the phrase for digital signature to the user terminal, decrypting the encrypted private key in the certificate in the user terminal to digitally sign with the certificate password (S50), and the user terminal to the information collection server Transmitting the digital signature value (S60), transmitting the digital signature value from the information collection server to the institution server (S70), and responding to the authentication result as a session key from the institution server to the information collection server (S80) A step of starting the scraping extraction operation by sending the screen URL to be scraped from the information collection server to the organization server (S90), and the step of transmitting the scraping extraction result from the organization server to the information collection server (S100), and information collection It is made including the step of processing the scraping extraction result in the server (S110).

In the step (S110) of processing the scraping extraction result in the information collection server, the year, month, date and time (ex. March 15, 2021 15:30) in the scraping extraction result, an authentication confirmation code such as an authentication phrase unique to the information collection server The step of adding may be further included. That is, by adding an authentication confirmation code unique to the information collection server to the scraping extraction result, reliability can be imparted.

After the digital signature is requested while transmitting the electronic signature phrase from the information collection server to the user terminal (S40), if the electronic signature is not made in the user terminal for a preset time, the previous scraping execution request may be canceled. The length of the preset time may vary according to the user's reliability (or grade). That is, the higher the reliability (grade), the longer the preset time may be. The more the number of times the user uses the information collection server 6 or the greater the amount charged, the higher the reliability (grade) will be. Also, the length of the preset time may vary depending on the type of the institution server.

The operation of the web scraping system and method having the hierarchical abstraction authentication function according to an embodiment of the present invention according to the above configuration is as follows.

First, when a user requests scraping for a specific institution server (ex financial institution, National Tax Service, etc.) using the user terminal 7, the user terminal 7 requests the electronic signature phrase from the information collection server 7 (S10). The electronic signature phrase may correspond to a digital signature nonce.

For reference, in the process of requesting the electronic signature phrase, in some cases, the user does not directly request the electronic signature phrase using the user terminal 7, but enters personal information into the information collection server 6 and then logs in. It can also be carried out as a process, membership registration process, etc.

Alternatively, an input for the user terminal 7 to access a specific institution server (ex Health Insurance Corporation, etc.) among a plurality of institution servers (ex an input for issuing a certificate of qualification acquisition, etc.) to the information collection server 6 The process of requesting the electronic signature phrase from the information collection server 6 by the user terminal 7 may be carried out in the process of delivery as well.

According to an embodiment of the present invention, a specific input from the user terminal 7 (ex, input for checking the balance of a bank account, etc.) is performed with a plurality of institutional servers (ex. My account server, bank server, etc.) There may be relevance. In this case, the information collection server 6 may visit the Internet site of any one of the plurality of institutional servers to request the issuance of the electronic signature phrase.

Below, a method of selecting one of the plurality of institution servers will be described.

The information collection server 6 i) displays a plurality of institution servers on the user terminal 7 and induces the user to select any one institution server, or ii) the information collection server 6 among the plurality of institution servers. You can request the issuance of a phrase for an electronic signature from a specific institution server (ex bank company server) that is connected to . In addition, in some cases, it may be possible to request the issuance of the electronic signature phrase to all of the plurality of institution servers.

When the electronic signature phrase is requested from the user terminal 7, the information collection server 6 requests the electronic signature phrase by visiting the Internet site of the corresponding institution server (S20).

When the electronic signature phrase is requested from the information collection server 6, the corresponding institution server issues the electronic signature text to the information collection server 6 (S30).

When the electronic signature text is issued from the corresponding institution server, the information collection server 6 requests the electronic signature while transmitting the electronic signature text to the user terminal 7 (S40).

The digital signature phrase is a unique phrase generated for use in encryption, and the digital signature value is added to the encrypted data to disguise a repeated pattern. The electronic signature phrase may be arbitrary or sequential. Phrases for random digital signatures are generated by binding random numbers together at random, whereas phrases for sequential digital signatures are generated gradually. The electronic signature phrase is configured to have both a random segment and a sequential segment.

When the digital signature is requested from the information collection server 6, the certificate management unit 72 of the user terminal 7 decrypts the encrypted private key in the certificate and digitally signs it with the certificate password (S50), the information collection server ( 6) to transmit the digital signature value (S60).

The digital signature uses the public key method to provide a non-repudiation function after obtaining the hash value of the original. After obtaining a hash value from the original document and document content, it is encrypted with the private key and sent to the other party. The digital signature is the encryption of the hash value of the document with the private key. If the digital signature is decrypted with the public key, the hash value can be known, and the authenticity of the document can be determined by comparing it with the hash value of the enclosed and delivered document.

When the digital signature value is transmitted from the certificate management unit 72 of the user terminal 7, the information collection server 6 transmits the digital signature value to the corresponding institution server (S70).

When the digital signature value is transmitted from the information collection server 6, the corresponding institution server responds to the information collection server 6 with the authentication result as a session key (S80). The session key is maintained for a preset period of time (ex 10 minutes), and the information collection server 6 may perform a scraping extraction operation during the corresponding period of time. The retention time of the session key may be different depending on the type of the institution server 11 or the scraping extraction target.

For example, the session key retention time of an institution that holds sensitive personal information, such as a financial institution, may be shorter than the session key retention time of other institutions that have less sensitive information.

When the authentication result is answered from the corresponding institution server, the scraping module 61 of the information collection server 6 sends the screen URL to be scraped to the corresponding institution server to start the scraping extraction operation (S90). The scraping module 61 may perform a scraping extraction operation for information existing on a web page of the screen URL, and may also perform a scraping extraction operation for information present in other web pages linked to the web page of the screen URL. have. In some cases, it is possible to perform a scraping extraction operation on information existing in the other web pages only for the terminal 7 of the user who has paid an additional fee.

Next, the corresponding institution server completes the scraping extraction operation by transmitting the scraping extraction result together with the HTTP response of the corresponding screen to the scraping module 61 of the information collection server 6 (S100).

When the scraping extraction operation is completed, the information collection server 6 processes the scraping extraction result according to the purpose and use (S110). As such, since the scraping extraction result is stored in the information collection server 6 instead of the user terminal 7, it is impossible to forge or falsify the scraping extraction result in the user terminal 7, so the reliability of the scraping extraction result can be increased. Then, in the information collection server 6, by adding an authentication confirmation code such as the year, month, date and time, and the authentication phrase, such as "June 1, 2021 15:35:24 sec certified" to the scraping extraction result, for example, the scraping extraction result is can be given more credibility. For reference, the authentication confirmation code may include a QR code, etc., and when the authentication confirmation code is recognized by another device, it may indicate that it has been authenticated by the information collection server 6 to secure reliability.

In some cases, an authentication validity period (ex 5 years) may be set for each authentication confirmation code, and the length of the authentication validity period may vary based on an institution server or a scraping target. In addition, if the validity period of the authentication expires or is before a predetermined period of time (eg one month before the expiration), the information collection server 6 transmits it to the user terminal 7 and causes the user terminal 7 to perform scraping. You can pass a message to request

In addition, according to an embodiment, the information collection server 6 sends a message to only the terminal 7 of the user who has paid a predetermined amount of money to request the scraping execution before the expiration of the authentication validity period or a predetermined period before the expiration. By forwarding it, it is also possible to maintain the authentication to the authority server.

Next, a case in which the web scraping method with the hierarchical abstraction authentication function described above is applied to the non-face-to-face loan examination of a financial institution will be described. For reference, most of the web scraping methods with the measurement abstraction authentication function described above can be equally applied to non-face-to-face loan examination of financial institutions unless otherwise specified.

5 is a flowchart of a non-face-to-face loan examination method using a web scraping method having a hierarchical abstraction authentication function according to another embodiment of the present invention.

As shown in FIG. 5 , the non-face-to-face loan examination method using the web scraping method having the hierarchical abstraction authentication function according to another embodiment of the present invention is a method of requesting (loan) examination to a financial institution server by a user terminal. Step (S210), requesting scraping (for health insurance payment details) from the financial institution server to the information collection server (S220), and requesting the electronic signature text from the information collection server to the Health Insurance Corporation server (institutional server) step (S230), the step of issuing the electronic signature phrase from the Health Insurance Corporation server to the information collection server (S240), and the step of requesting the electronic signature while transmitting the electronic signature phrase from the information collection server to the user terminal (S250) and decrypting the encrypted private key in the certificate with the certificate password in the user terminal to digitally sign (S260), and transmitting the digital signature value from the user terminal to the information collection server (S270), and in the information collection server Transmitting the digital signature value to the Health Insurance Corporation server (S280), responding to the authentication result as a session key from the Health Insurance Corporation server to the information collection server (S290), and from the information collection server to the Health Insurance Corporation server Sending a screen URL about health insurance payment details to start scraping extraction (S300), and transmitting health insurance payment details as a scraping extraction result from the Health Insurance Corporation server to an information collection server (S310), and information collection Transmitting the health insurance payment details from the server to the financial institution server (S320), and transmitting the loan examination result from the financial institution server to the user terminal according to the health insurance payment details (S330).

In the step (S320) of transmitting the health insurance payment details from the information collection server to the financial institution server, the method may further include adding an authentication confirmation code such as year, month, date, and time to the scraping extraction result.

The operation of the web scraping system with the hierarchical abstraction authentication function according to an embodiment of the present invention by the above method configuration is as follows.

First, when the user makes a loan review request to the financial institution server using the user terminal 7 (S210), the financial institution server requests scraping of the user's health insurance payment history to the information collection server (S220).

When scraping of the user's health insurance payment details is requested from the financial institution server, the information collection server 6 requests the electronic signature phrase by visiting the Internet site of the Health Insurance Corporation server (S230).

When the electronic signature text is requested from the information collection server 6, the Health Insurance Corporation server issues the electronic signature text to the information collection server (S240).

When the electronic signature text is issued from the Health Insurance Corporation server, the information collection server 6 requests the electronic signature while transmitting the electronic signature text to the user terminal 7 (S250).

When a digital signature is requested from the information collection server 6, the certificate management unit 72 of the user terminal 7 decrypts the encrypted private key in the certificate and digitally signs it with the certificate password (S260), the information collection server ( 6) to transmit the digital signature value (S270).

When the digital signature value is transmitted from the certificate management unit 72 of the user terminal 7, the information collection server 6 transmits the digital signature value to the Health Insurance Corporation server (S280).

When the digital signature value is transmitted from the information collection server 6, the Health Insurance Corporation server responds to the information collection server 6 with the authentication result as a session key (S290).

When the authentication result is answered from the Health Insurance Corporation server, the scraping module 61 of the information collection server 6 sends a screen URL related to the health insurance payment details to the Health Insurance Corporation server to start the scraping extraction operation (S300).

Next, the Health Insurance Corporation server completes the scraping extraction operation by transmitting the health insurance payment details as the scraping extraction result together with the HTTP response of the corresponding screen to the scraping module 61 of the information collection server 6 (S310).

When the scraping extraction operation is completed, the information collection server 6 transmits the health insurance payment details as the scraping extraction result to the financial institution server (S320). As such, since the scraping extraction result is stored in the information collection server 6 instead of the user terminal 7, it is impossible to forge or falsify the scraping extraction result in the user terminal 7, so the reliability of the scraping extraction result can be increased. Then, in the information collection server 6, by adding an authentication confirmation code such as the year, month, date and time, and the authentication phrase, such as "June 1, 2021 15:35:24 sec certified" to the scraping extraction result, for example, the scraping extraction result is can be given more credibility.

When the health insurance payment details are transmitted from the information collection server 6, the financial institution server transmits the loan examination result to the user terminal 7 according to the health insurance payment details (S330).

11~1n: Organization Server 2: Internet
3, 5, 7: User terminal 4: Mediation server
6: Information collection server 31, 41, 71: Certificate
32, 42, 72: Certificate management unit 33, 43, 73: Scraping module

Claims (5)

An information collection server connected to a plurality of institutional servers through the Internet;
It includes a user terminal connected to the information collection server,
A certificate and a certificate management unit are installed in the user terminal,
A scraping module is installed in the information collection server separately from the certificate and certificate management unit of the user terminal, and the scraping module is updated by the information collection server,
The user terminal requests scraping to the information collection server and at the same time requests a phrase for an electronic signature, requests a phrase for an electronic signature by visiting an institution server by the information collection server, and sends an electronic signature from the institution server to the information collection server A signature phrase is issued, and the electronic signature is requested while transmitting the electronic signature phrase from the information collection server to the user terminal, and the certificate management unit of the user terminal decrypts the encrypted private key in the certificate to digitally sign the certificate with the password. Then, the digital signature value is transmitted to the information collection server, the digital signature value is transmitted from the information collection server to the organization server, and the authentication result is returned from the organization server to the information collection server as a session key, and the information collection A server has a hierarchical abstraction authentication function, characterized in that the server starts the scraping extraction operation for the organization server, transmits the scraping extraction result from the organization server to the information collection server, and processes the scraping extraction result in the information collection server One web scraping system. delete The step of requesting scraping from the user terminal to the information collection server and at the same time requesting a phrase for electronic signature;
requesting a phrase for digital signature by visiting an institution server by the information collection server;
issuing a text for digital signature from the institution server to the information collection server;
requesting an electronic signature while transmitting the electronic signature text from the information collection server to the user terminal;
Decrypting the encrypted private key in the certificate in the user terminal and digitally signing the certificate with the password;
transmitting a digital signature value from the user terminal to the information collection server;
transmitting the digital signature value from the information collection server to the institution server;
responding to the authentication result as a session key from the institution server to the information collection server;
starting a scraping extraction operation for the institution server in the information collection server;
transmitting the scraping extraction result from the institution server to the information collection server;
Including the step of processing the scraping extraction result in the information collection server,
The information collection server has a scraping module installed separately from the certificate and certificate management unit of the user terminal, and the scraping module is updated by the information collection server Web scraping method with a hierarchical abstraction authentication function. making a review request to a financial institution server by a user terminal;
requesting scraping from the financial institution server to the information collection server;
A step of requesting a text for digital signature from the information collection server to the institution server;
issuing a text for digital signature from the institution server to the information collection server;
requesting an electronic signature while transmitting the electronic signature text from the information collection server to the user terminal;
Decrypting the encrypted private key in the certificate with the certificate password in the user terminal to digitally sign;
transmitting a digital signature value from the user terminal to the information collection server;
transmitting the digital signature value from the information collection server to the institution server;
responding to the authentication result as a session key from the institution server to the information collection server;
starting a scraping extraction operation for the server in the information collection server;
transmitting the scraping extraction result from the institution server to the information collection server;
transmitting the scraping extraction result from the information collection server to the financial institution server;
Transmitting the examination result from the financial institution server to the user terminal according to the scraping extraction result,
The information collection server has a scraping module installed separately from the certificate and certificate management unit of the user terminal, and the scraping module is updated by the information collection server Web scraping method with a hierarchical abstraction authentication function. 5. The method of claim 3 or 4,
A web scraping method with a hierarchical abstraction authentication function, characterized in that it further comprises the step of adding an authentication confirmation code such as year, month, date and time, and an authentication phrase to the scraping extraction result in the information collection server.

Images