CN109783714A - Interface data acquisition methods and system - Google Patents
Interface data acquisition methods and system Download PDFInfo
- Publication number
- CN109783714A CN109783714A CN201910014941.3A CN201910014941A CN109783714A CN 109783714 A CN109783714 A CN 109783714A CN 201910014941 A CN201910014941 A CN 201910014941A CN 109783714 A CN109783714 A CN 109783714A
- Authority
- CN
- China
- Prior art keywords
- data
- interface
- module
- data acquisition
- interface data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a kind of interface data acquisition methods and systems, comprising: S1, the login account and modification logging for configuring interface WEB system carry out logging in for interface WEB system by the artificial mode of automatic imitation；S2, logon authentication code is identified by picture recognition technology；S3, the login account and modification logging of configuration and the logon authentication code identified are verified, when being proved to be successful, into S4；S4, it is obtained according to preset query strategy by the data of web crawlers interface differential technique mouth WEB system；S5, the interface data got is handled by correcting data error and Supplementing Data mechanism；S6, storage treated interface data.The present invention is applied in combination by manual simulation's technology, picture recognition technology, Network Data Capture technology, and allowing interface data to obtain work becomes to be simple and efficient, and saves a large amount of manpower and time.
Description
Technical field
The present invention relates to interface data acquiring technology fields, more particularly to a kind of interface data acquisition methods and system.
Background technique
Interface data transmitting between existing enterprise, majority are to provide the WEB system of all kinds of business.The work people of supplier
Member, the login system of timing/not timing, entered function, the defeated filter condition of hand carries out data query, and downloads the data to EXCEL
On the file of file or extended formatting.The file downloaded, then data preparation and conversion are carried out by artificial understanding.It is this
Mode generally requires enterprise and is equipped with fixed staff, expends a large amount of human cost and time cost.
Summary of the invention
The present invention is in view of the problems of the existing technology and insufficient, provides a kind of interface data acquisition methods and system.
The present invention is to solve above-mentioned technical problem by following technical proposals:
The present invention provides a kind of interface data acquisition methods, it is characterized in that comprising following steps:
S1, the login account and modification logging for configuring interface WEB system, carry out interface by the artificial mode of automatic imitation
WEB system logs in；
S2, logon authentication code is identified by picture recognition technology；
S3, the login account and modification logging of configuration and the logon authentication code identified are verified, verifying at
When function, S4 is entered step；
S4, it is obtained according to preset query strategy by the data of web crawlers interface differential technique mouth WEB system；
S5, the interface data got is handled by correcting data error and Supplementing Data mechanism；
S6, storage treated interface data.
Preferably, in step s 4, carrying out the files such as excel, cvs, html by web crawlers technology or content obtaining
It takes.
Preferably, in step s 6, the storage of interface data is carried out by mysql document data bank.
The present invention also provides a kind of interface data acquisition systems, it is characterized in that comprising configuration log-in module, identification mould
Block, authentication module, data acquisition module, data processing module and data memory module；
The configuration log-in module is used to configure the login account and modification logging of interface WEB system, passes through automatic imitation
Artificial mode carries out logging in for interface WEB system；
The identification module is for identifying logon authentication code by picture recognition technology；
The authentication module is used for login account and modification logging to configuration and the logon authentication code identified carries out
Verifying, data acquisition module is called when being proved to be successful；
The data acquisition module is used to pass through web crawlers interface differential technique mouth WEB system according to preset query strategy
Data are obtained；
The data processing module is used to carry out the interface data got by correcting data error and Supplementing Data mechanism
Processing；
The data memory module is for storing treated interface data.
Preferably, the data acquisition module be used to carry out by web crawlers technology the files such as excel, cvs, html or
Person's content obtains.
Preferably, the data memory module is used to carry out the storage of interface data by mysql document data bank.
On the basis of common knowledge of the art, above-mentioned each optimum condition, can any combination to get each preferable reality of the present invention
Example.
The positive effect of the present invention is that:
The present invention is applied in combination by manual simulation's technology, picture recognition technology, Network Data Capture technology, allows interface
Data acquisition work becomes to be simple and efficient, and saves a large amount of manpower and time.Moreover, the present invention passes through correcting data error and data
Completion mechanism guarantees data correctness, instead of artificial daily repetitive operation, improves efficiency 90% or more.
Detailed description of the invention
Fig. 1 is the flow chart of the interface data acquisition methods of present pre-ferred embodiments.
Fig. 2 is the structural block diagram of the interface data acquisition methods of present pre-ferred embodiments.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art
Every other embodiment obtained without making creative work, shall fall within the protection scope of the present invention.
As shown in Figure 1, the present embodiment provides a kind of interface data acquisition methods comprising following steps:
Step 101, configure interface WEB system login account and modification logging, by the artificial mode of automatic imitation into
Line interface WEB system logs in.
Login account and modification logging configuration are stepped in interface WEB system according to preset timing automatically first
Land login account and modification logging save human cost without manually inputting login account and modification logging.
Step 102 identifies logon authentication code by picture recognition technology.
Identify that logon authentication code is saved without manually inputting logon authentication code by picture recognition technology
Human cost.
Step 103 verifies the login account and modification logging of configuration and the logon authentication code identified, i.e., from
The login account and modification logging of dynamic input and the logon authentication code automatically identified and pre-stored login account, log in it is close
Code and logon authentication code carry out matching verifying one by one, show authentication success when matching one by one.
Step 104 is obtained according to preset query strategy by the data of web crawlers interface differential technique mouth WEB system
It takes, such as excel, cvs, html file or content acquisition is carried out by web crawlers technology.
Step 105 is handled the interface data got by correcting data error and Supplementing Data mechanism, to guarantee to obtain
The correctness of the interface data taken.
Step 106, the storage that treated interface data is carried out by mysql document data bank.
As shown in Fig. 2, the present embodiment also provides a kind of interface data acquisition system comprising configuration log-in module 1, identification
Module 2, authentication module 3, data acquisition module 4, data processing module 5 and data memory module 6.
The configuration log-in module 1 is used to configure the login account and modification logging of interface WEB system, passes through automatic imitation
Artificial mode carries out logging in for interface WEB system.
The identification module 2 is for identifying logon authentication code by picture recognition technology.
The authentication module 3 is used for login account and modification logging to configuration and the logon authentication code identified carries out
Data acquisition module 4 is called in verifying when being proved to be successful.
The data acquisition module 4 is used to pass through web crawlers interface differential technique mouth WEB system according to preset query strategy
Data (files such as excel, cvs, html or content) obtained.
The data processing module 5 is used to carry out the interface data got by correcting data error and Supplementing Data mechanism
Processing.
The data memory module 6 is used to carry out the storage of treated interface data by mysql document data bank.
The present invention on the internet, according to the given network address set, passes through the technical approach of similar search engine
With the query strategy preset, data acquisition is carried out to the WEB web page contents of the indirect port system of enterprise, by certain automatic point
Data are downloaded storage by analysis and filtering.
Although specific embodiments of the present invention have been described above, it will be appreciated by those of skill in the art that these
It is merely illustrative of, protection scope of the present invention is defined by the appended claims.Those skilled in the art is not carrying on the back
Under the premise of from the principle and substance of the present invention, many changes and modifications may be made, but these are changed
Protection scope of the present invention is each fallen with modification.
Claims (6)
1. a kind of interface data acquisition methods, which is characterized in that itself the following steps are included:
S1, the login account and modification logging for configuring interface WEB system, carry out interface WEB by the artificial mode of automatic imitation
System logs in；
S2, logon authentication code is identified by picture recognition technology；
S3, the login account and modification logging of configuration and the logon authentication code identified are verified, when being proved to be successful,
Enter step S4；
S4, it is obtained according to preset query strategy by the data of web crawlers interface differential technique mouth WEB system；
S5, the interface data got is handled by correcting data error and Supplementing Data mechanism；
S6, storage treated interface data.
2. interface data acquisition methods as described in claim 1, which is characterized in that in step s 4, pass through web crawlers skill
Art carries out the files such as excel, cvs, html or content obtains.
3. interface data acquisition methods as described in claim 1, which is characterized in that in step s 6, pass through mysql number of files
The storage of interface data is carried out according to library.
4. a kind of interface data acquisition system, which is characterized in that it includes configuration log-in module, identification module, authentication module, number
According to acquisition module, data processing module and data memory module；
The configuration log-in module is used to configure the login account and modification logging of interface WEB system, artificial by automatic imitation
Mode carry out logging in for interface WEB system；
The identification module is for identifying logon authentication code by picture recognition technology；
The logon authentication code that the authentication module is used for login account and modification logging to configuration and identifies is verified,
Data acquisition module is called when being proved to be successful；
The data acquisition module is used to pass through according to preset query strategy the data of web crawlers interface differential technique mouth WEB system
It is obtained；
The data processing module is used to handle the interface data got by correcting data error and Supplementing Data mechanism；
The data memory module is for storing treated interface data.
5. interface data acquisition system as claimed in claim 4, which is characterized in that the data acquisition module is for passing through net
Network crawler technology carries out the files such as excel, cvs, html or content obtains.
6. interface data acquisition system as claimed in claim 4, which is characterized in that the data memory module is for passing through
The storage of mysql document data bank progress interface data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910014941.3A CN109783714A (en) | 2019-01-08 | 2019-01-08 | Interface data acquisition methods and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910014941.3A CN109783714A (en) | 2019-01-08 | 2019-01-08 | Interface data acquisition methods and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109783714A true CN109783714A (en) | 2019-05-21 |
Family
ID=66499190
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910014941.3A Pending CN109783714A (en) | 2019-01-08 | 2019-01-08 | Interface data acquisition methods and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109783714A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110245176A (en) * | 2019-06-20 | 2019-09-17 | 中移电子商务有限公司 | A kind of data capture method, device, equipment and medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1701281A1 (en) * | 2005-03-08 | 2006-09-13 | 1&1 Internet AG | Method and system for logging into a service |
| CN103514171A (en) * | 2012-06-20 | 2014-01-15 | 同程网络科技股份有限公司 | Method for implementing self-defined crawler based on optical character recognition and vertical search |
| CN105631030A (en) * | 2015-12-30 | 2016-06-01 | 福建亿榕信息技术有限公司 | Universal web crawler login simulation method and system |
| CN106778196A (en) * | 2015-11-23 | 2017-05-31 | 北京金山安全软件有限公司 | Network station simulated login method and device and electronic equipment |
| CN107895009A (en) * | 2017-11-10 | 2018-04-10 | 北京国信宏数科技有限责任公司 | One kind is based on distributed internet data acquisition method and system |
-
2019
- 2019-01-08 CN CN201910014941.3A patent/CN109783714A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1701281A1 (en) * | 2005-03-08 | 2006-09-13 | 1&1 Internet AG | Method and system for logging into a service |
| CN103514171A (en) * | 2012-06-20 | 2014-01-15 | 同程网络科技股份有限公司 | Method for implementing self-defined crawler based on optical character recognition and vertical search |
| CN106778196A (en) * | 2015-11-23 | 2017-05-31 | 北京金山安全软件有限公司 | Network station simulated login method and device and electronic equipment |
| CN105631030A (en) * | 2015-12-30 | 2016-06-01 | 福建亿榕信息技术有限公司 | Universal web crawler login simulation method and system |
| CN107895009A (en) * | 2017-11-10 | 2018-04-10 | 北京国信宏数科技有限责任公司 | One kind is based on distributed internet data acquisition method and system |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110245176A (en) * | 2019-06-20 | 2019-09-17 | 中移电子商务有限公司 | A kind of data capture method, device, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109242326B (en) | Policy sharing system based on big data and artificial intelligence | |
| CN105095223B (en) | File classification method and server | |
| CN104915668B (en) | Text information recognition methods and device in medical image | |
| CN109214821A (en) | identity remote authentication method and terminal device | |
| CN104933138A (en) | Webpage crawler system and webpage crawling method | |
| CN106600269A (en) | Paying method and platform based on two-dimensional barcode | |
| CN105493095A (en) | Adaptive and recursive filtering for sample submission | |
| CN104753909B (en) | Method for authenticating after information updating, Apparatus and system | |
| CN109614110A (en) | A kind of method and apparatus that message-oriented middleware concentrates deployment | |
| CN106709804A (en) | Interactive wealth planning consulting robot system | |
| CN106600294A (en) | Method and system for rapidly identifying enterprise identity in field of finance | |
| CN110837998A (en) | Contract auditing method, device, equipment and medium | |
| CN106130739A (en) | Application program login process method and device | |
| CN109409093A (en) | A kind of system vulnerability scan schedule method | |
| CN109300026A (en) | Financial intelligent analysis method and its system based on automatic book keeping operation big data | |
| CN105630797A (en) | Data processing method and system | |
| CN109783714A (en) | Interface data acquisition methods and system | |
| CN107885850A (en) | A kind of localization method and device of banking class problem | |
| CN106096060A (en) | Ocean network security risk system of defense | |
| CN113283984A (en) | Personal loan information input method and device | |
| CN107277108A (en) | Message treatment method, apparatus and system at a kind of node of block chain | |
| CN111104853A (en) | Image information input method and device, electronic equipment and storage medium | |
| CN105608561A (en) | Method and apparatus for processing mail | |
| US11989199B2 (en) | Optimizing flow of data within ETL data processing pipeline | |
| CN108960950A (en) | A kind of intelligence system and method for cross-border electric business commercial affairs big data decision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190521 |
|
| RJ01 | Rejection of invention patent application after publication |