CN112148953A - Method and device for accessing difference data - Google Patents
Method and device for accessing difference data Download PDFInfo
- Publication number
- CN112148953A CN112148953A CN202011059839.4A CN202011059839A CN112148953A CN 112148953 A CN112148953 A CN 112148953A CN 202011059839 A CN202011059839 A CN 202011059839A CN 112148953 A CN112148953 A CN 112148953A
- Authority
- CN
- China
- Prior art keywords
- data
- access request
- access
- source address
- analysis result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000012545 processing Methods 0.000 claims abstract description 48
- 238000007405 data analysis Methods 0.000 claims abstract description 41
- 238000013075 data extraction Methods 0.000 claims abstract description 41
- 239000003999 initiator Substances 0.000 claims abstract description 27
- 230000004069 differentiation Effects 0.000 claims abstract description 16
- 230000004044 response Effects 0.000 claims description 20
- 238000004422 calculation algorithm Methods 0.000 claims description 9
- 230000009471 action Effects 0.000 claims description 8
- 238000004458 analytical method Methods 0.000 claims description 8
- 238000012805 post-processing Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 5
- 238000004806 packaging method and process Methods 0.000 claims description 5
- 230000008451 emotion Effects 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000000605 extraction Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 235000006545 Ziziphus mauritiana Nutrition 0.000 description 2
- 244000126002 Ziziphus vulgaris Species 0.000 description 2
- 235000008529 Ziziphus vulgaris Nutrition 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000011084 recovery Methods 0.000 description 2
- 238000012552 review Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000000246 remedial effect Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 238000013515 script Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/958—Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The disclosure relates to a method and a device for accessing difference data, an electronic device and a storage medium. Wherein, the method comprises the following steps: receiving an access request of an initiator for accessing a target webpage, and analyzing a data source address in the access request; acquiring a data extraction strategy corresponding to the data source address, and extracting corresponding characteristic data according to the data extraction strategy; carrying out structuralization processing on the characteristic data to obtain structuralized data; and acquiring the published service data, carrying out differentiation comparison on the structured data and the service data to generate a differential data analysis result, and sending the differential data analysis result to the initiator of the access request. The method and the device can respond to data access to provide timely and convenient difference data analysis and difference data analysis result feedback.
Description
Technical Field
The present disclosure relates to the field of data processing, and in particular, to a method and an apparatus for accessing difference data, an electronic device, and a computer-readable storage medium.
Background
With the rapid development of the internet, various information on the internet brings great convenience to the life of people. For example, various data query systems can provide convenient data query services for users, so that the users can know required information more conveniently. With the great increase of the data volume of the internet, the accuracy and the updating timeliness of the published service data are required to provide more accurate and timely service data and better user experience. However, at present, the update of data in service data is mostly realized by pure system programs or running scripts, and when the published service data does not conform to actual data and the service data is not updated, it is difficult for non-technical personnel who need to query difference data to distinguish and analyze a large amount of data, and further, remedial measures cannot be made in time according to the difference data, so how to respond to data access to timely and conveniently acquire the difference data is an urgent technical problem to be solved.
Accordingly, there is a need for one or more methods to address the above-mentioned problems.
It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present disclosure, and thus may include information that does not constitute prior art known to those of ordinary skill in the art.
Disclosure of Invention
An object of the present disclosure is to provide a method, an apparatus, an electronic device, and a computer-readable storage medium for accessing difference data, thereby overcoming, at least to some extent, one or more of the problems due to the limitations and disadvantages of the related art.
According to an aspect of the present disclosure, there is provided a method for accessing difference data, including:
receiving an access request of an initiator for accessing a target webpage, and analyzing a data source address in the access request;
acquiring a data extraction strategy corresponding to the data source address, and extracting corresponding characteristic data according to the data extraction strategy;
carrying out structuralization processing on the characteristic data to obtain structuralized data;
acquiring published service data, and carrying out differentiation comparison on the structured data and the service data to generate a differential data analysis result;
and sending the analysis result of the difference data to the initiator of the access request.
In an exemplary embodiment of the present disclosure, after receiving an access request from an initiator to access a target web page, and parsing a data source address corresponding to the access request, the method further includes:
adding the address identifier of the access proxy server to the header information of the access request, and sending the access request added with the address identifier to a target server corresponding to a data source address after packaging;
receiving data response information replied by the target server, and acquiring source data in the data response information;
extracting corresponding feature data according to the data extraction strategy, wherein the extracting comprises the following steps:
and extracting characteristic data corresponding to the data source address in source data according to the data extraction strategy.
In an exemplary embodiment of the present disclosure, extracting corresponding feature data according to the data extraction policy includes:
the data extraction strategy comprises dimension information corresponding to each data source address; after source data in the data response information is obtained, determining appointed dimension information corresponding to a current data source address;
extracting at least one feature data corresponding to the specified dimension information from the source data.
In an exemplary embodiment of the present disclosure, after receiving an access request from an initiator to access a target web page, and parsing a data source address corresponding to the access request, the method further includes:
setting an access information quantity identifier in the access request, packaging the access request added with the access information quantity identifier, and sending the packaged access request to a target server corresponding to a data source address;
and receiving data response information replied by the target server, wherein an access result set is included in the data response information, and the number of the access results in the access result set is the number represented by the access information number identification.
In an exemplary embodiment of the present disclosure, performing a structuring process on the feature data to obtain structured data includes:
when the feature data are numerical features, carrying out structuring processing on the numerical features according to a preset numerical feature processing rule;
when the feature data is a first text feature, matching and structuring the first text feature with a preset text feature processing rule;
when the feature data is a second text feature, extracting entity data in the second text feature; filtering the entity data according to an emotion judgment algorithm; and carrying out rule matching on the filtered entity data according to a preset expression rule, and generating structured data.
In an exemplary embodiment of the present disclosure, the differentially comparing the structured data with the service data to generate a differential data analysis result includes:
calculating the structured data according to a preset data abstract algorithm to obtain an abstract characteristic value;
traversing a pre-stored service database, calculating whether the pre-stored service database has service data matched with the abstract characteristic value, if not, determining that the structured data has differences, and generating a difference data analysis result according to the structured data.
In an exemplary embodiment of the present disclosure, the method further comprises:
responding to a post-processing instruction, and acquiring a processing action in the post-processing instruction; the processing action comprises deleting and restoring operations;
and processing the difference data in the difference data analysis result according to the processing action.
In one aspect of the present disclosure, there is provided an apparatus for accessing difference data, including:
the access request receiving module is used for receiving an access request of an initiator for accessing a target webpage and analyzing a data source address in the access request;
the characteristic data extraction module is used for acquiring a data extraction strategy corresponding to the data source address and extracting corresponding characteristic data according to the data extraction strategy;
the structured data processing module is used for carrying out structured processing on the characteristic data to obtain structured data;
the data differentiation comparison module is used for acquiring published service data, carrying out differentiation comparison on the structured data and the service data and generating a differential data analysis result;
and the analysis result sending module is used for sending the difference data analysis result to the initiator of the access request.
In one aspect of the present disclosure, there is provided an electronic device including:
a processor; and
a memory having computer readable instructions stored thereon which, when executed by the processor, implement a method according to any of the above.
In an aspect of the disclosure, a computer-readable storage medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, realizes the method according to any one of the above.
In the access method of the differential data in the exemplary embodiment of the disclosure, an access request of an initiator for accessing a target webpage is received, and a data source address in the access request is analyzed; acquiring a data extraction strategy corresponding to the data source address, and extracting corresponding characteristic data according to the data extraction strategy; carrying out structuralization processing on the characteristic data to obtain structuralized data; and acquiring the published service data, carrying out differentiation comparison on the structured data and the service data to generate a differential data analysis result, and sending the differential data analysis result to the initiator of the access request. The method and the device can respond to data access to provide timely and convenient difference data analysis and difference data analysis result feedback.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The above and other features and advantages of the present disclosure will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.
FIG. 1 illustrates an application scenario diagram of an access method of difference data according to an exemplary embodiment of the present disclosure;
FIG. 2 illustrates a flow chart of a method of accessing difference data according to an exemplary embodiment of the present disclosure;
FIG. 3 shows a schematic block diagram of an access device for difference data according to an example embodiment of the present disclosure;
FIG. 4 schematically illustrates a block diagram of an electronic device according to an exemplary embodiment of the present disclosure; and
fig. 5 schematically illustrates a schematic diagram of a computer-readable storage medium according to an exemplary embodiment of the present disclosure.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The same reference numerals denote the same or similar parts in the drawings, and thus, a repetitive description thereof will be omitted.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the embodiments of the disclosure can be practiced without one or more of the specific details, or with other methods, components, materials, devices, steps, and so forth. In other instances, well-known structures, methods, devices, implementations, materials, or operations are not shown or described in detail to avoid obscuring aspects of the disclosure.
The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in the form of software, or in one or more software-hardened modules, or in different networks and/or processor devices and/or microcontroller devices.
In the present exemplary embodiment, first, an access method of difference data is provided; referring to fig. 1, the method for accessing difference data may include the steps of:
step S110, receiving an access request of an initiator for accessing a target webpage, and analyzing a data source address in the access request;
step S120, acquiring a data extraction strategy corresponding to the data source address, and extracting corresponding characteristic data according to the data extraction strategy;
step S130, carrying out structuralization processing on the characteristic data to obtain structuralization data;
step S140, acquiring the published service data, and carrying out differentiation comparison on the structured data and the service data to generate a differential data analysis result;
step S150, the difference data analysis result is sent to the initiator of the access request.
In the access method of the differential data in the exemplary embodiment of the disclosure, an access request of an initiator for accessing a target webpage is received, and a data source address in the access request is analyzed; acquiring a data extraction strategy corresponding to the data source address, and extracting corresponding characteristic data according to the data extraction strategy; carrying out structuralization processing on the characteristic data to obtain structuralized data; and acquiring the published service data, carrying out differentiation comparison on the structured data and the service data to generate a differential data analysis result, and sending the differential data analysis result to the initiator of the access request. The method and the device can respond to data access to provide timely and convenient difference data analysis and difference data analysis result feedback.
Fig. 2 shows a network environment to which the access method of difference data in the present exemplary embodiment is applied, including an access server 210, a client 220, and a target server 230. The access method of the difference data in the present exemplary embodiment may be applied to the access server 210.
Next, a method of accessing difference data in the present exemplary embodiment will be further explained.
In step S110, an access request for the initiator to access the target web page may be received, and a data source address in the access request may be parsed.
In this exemplary embodiment, in order to facilitate query and analysis of the difference data for a specific group of people, an access agent may be installed in advance for a user to access a browser, and when the user accesses a target web page through the browser, an access request for accessing the target web page is sent to an access server. After receiving the access request, the access server parses the access request to obtain a data source address carried in the access request, where the data source address may be an access address of a target webpage, for example, when it is required to query whether lawsuit data in the published service data is different from actual data, the accessed target webpage may be a query website of the lawsuit data, such as a website of a referee document web.
In step S120, a data extraction policy corresponding to the data source address may be obtained, and corresponding feature data may be extracted according to the data extraction policy;
after a data source address carried in an access request is obtained, the address identifier of the access proxy server can be added to the header information of the access request, and the access request added with the address identifier is packaged and then sent to a target server corresponding to the data source address; and receiving the data response information replied by the target server, and acquiring the source data in the data response information.
After the data source address carried in the access request is obtained, corresponding data is obtained for a user without perception, better use experience is provided for the user, data access can be completed in an anonymous access mode, after the data source address carried in the access request is obtained by the access proxy server, the address identification (IP address of the access proxy server) of the access proxy server can be added to a sender field (such as a source IP address field) in the head information of the access request, and then the access request added with the IP address of the access proxy server is packaged and sent to a target server corresponding to the data source address. So that the target server sends the access response information to the access proxy server when replying the access response information.
Receiving an access request of an initiator for accessing a target webpage, analyzing a data source address corresponding to the access request, setting an access information quantity identifier in the access request, packaging the access request added with the access information quantity identifier, and sending the packaged access request to a target server corresponding to the data source address; and receiving data response information replied by the target server, wherein an access result set is included in the data response information, and the number of the access results in the access result set is the number represented by the access information number identification.
When a user performs a normal website access search, the target server usually returns 10 pieces of result data per page, and even if the default value set by the target server is greater than 10, the user is still required to perform a page turning operation all the time to acquire all the result data when the amount of the result data is large, so that the user is difficult to acquire all the result data through one access. In the exemplary embodiment of the disclosure, after receiving an access request from an initiator to access a target webpage and analyzing a data source address corresponding to the access request, an access information quantity identifier may be further set in a configurable field of the access request, where the access information quantity identifier is used to represent a feedback quantity of access result data, for example, the access information quantity identifier may be set to 1000, and then the access request with the access information quantity identifier 1000 added to the configurable field is encapsulated and sent to a target server corresponding to the data source address; then after the target server receives the access request, the maximum 1000 data results are recalled to generate an access result set for returning. After receiving the access response information with the access result set replied by the target server, the access proxy server may use the access result set of the access response information as the source data. Therefore, when a user searches data, all data can be acquired through one-time searching by accessing the proxy server, and all result data can be browsed on a target webpage page.
After the source data are obtained, extracting corresponding feature data according to a data extraction strategy, wherein the method comprises the following steps: and extracting characteristic data corresponding to the data source address in source data according to the data extraction strategy. Specifically, extracting feature data corresponding to the data source address from the source data according to the data extraction policy includes: the data extraction strategy comprises dimension information corresponding to each data source address; after source data in the data response information is obtained, determining the designated dimension information corresponding to the current data source address; extracting at least one feature data corresponding to the specified dimension information from the source data.
The data extraction strategy is extracted into data extraction strategies with different extraction rules which are preset according to different source addresses; for example, the published service data includes judicial data, specifically includes multiple dimensions of case information, litigation number, executed information, and the like of the judicial data, and the website accessed by acquiring the judicial data includes: https:// wenshu.court.gourt.gov.cn/, http:// bjzcfy.chinacour.gov.cn/index.shtml; then the data extraction policy includes that the extraction rule corresponding to the website is: extracting data corresponding to dimensionalities such as filing information, litigation quantity, executed information and the like. And the data corresponding to the dimensionalities such as the scheme information, the litigation quantity, the executed information and the like acquired according to the data extraction strategy is the characteristic data corresponding to the data source address in the source data.
In step S130, the feature data may be subjected to a structuring process to obtain structured data;
carrying out structuring processing on the feature data to obtain structured data, wherein the method comprises the following steps: when the feature data are numerical features, carrying out structuring processing on the numerical features according to a preset numerical feature processing rule; when the feature data is a first text feature, matching and structuring the first text feature with a preset text feature processing rule; when the feature data is a second text feature, extracting entity data in the second text feature; filtering the entity data according to an emotion judgment algorithm; and carrying out rule matching on the filtered entity data according to a preset expression rule, and generating structured data. Specifically, the method comprises the following steps:
for the time characteristics such as the set date, the opening date, the announcement time and the like, the Chinese and English extraction modes can be adopted, and for the Chinese date, xx month xx date in xx year is converted into a standard English year, month, day and time format for processing, and the Chinese date and the month xx date are unified into a 2020-1-1 form; and judging whether the symbol is a standard English symbol or not for the English date and then carrying out conversion processing. For example, for the numerical characteristics of the company, such as 2000000, 200 ten thousands of registered money, etc., the numerical characteristics can be uniformly expressed as a unit according to the preset numerical characteristic processing rule, for example, the numerical characteristics are uniformly expressed as 200 ten thousands; the quantity is unified into XX pieces for company litigation, etc. For the case number extraction of the legal document, a case number characteristic database can be established in advance according to the case number characteristics of the national regional court, and each case number establishment rule is prestored in the case number characteristic database, for example, the case number establishment rule comprises the number of characters, the meaning represented by each section of the characters and the like, and if the case number rule of the intellectual property right court in Beijing is (year) + Jing + court code + administrative/civil short for + preliminary review/review short for + running number, a character string matched with the case number establishment rule is extracted from the metadata according to the case number establishment rule of each case number, and is used as the key characteristic information, so as to accurately match the case number characteristics of the national regional court.
When the feature data is the first text feature of the phrase granularity, the phrase granularity text can be used as the structured data, or the phrase granularity text is matched into a uniform phrase according to the semantics and then is used as the structured data;
when the feature data is a second text feature, for example: the company brief introduction content is usually large text summarized according to company business information, or the entity data in the second text feature is extracted according to the discussion topic of the company, the public opinion information of the company and the like; filtering the entity data according to an emotion judgment algorithm; and carrying out rule matching on the filtered entity data according to a preset expression rule, and generating structured data.
In step S140, the published service data may be acquired, and the structured data and the service data are differentiated and compared to generate a difference data analysis result;
carrying out differentiation comparison on the structured data and the service data to generate a differentiation data analysis result, which comprises the following steps: calculating the structured data according to a preset data abstract algorithm to obtain an abstract characteristic value; traversing a pre-stored service database, calculating whether the pre-stored service database has service data matched with the abstract characteristic value, if not, determining that the structured data has differences, and generating a difference data analysis result according to the structured data.
The preset data summarization algorithm may be a hash algorithm, an MD5 value, or the like. Taking MD5 value calculation as an example, performing MD5 value calculation on the obtained structured data to obtain a corresponding digest feature value, comparing the digest feature value with digest feature values of each service data in a pre-stored service database, if the two are consistent, it indicates that the current structured data is not different from the actual data, and if the two are not consistent, it indicates that the current structured data is different from the actual data, further combining dimension information of all the structured data and the structured data to generate a difference data analysis result, and updating the service database according to the MD5 value of the current structured data.
In step S150, the difference data analysis result may be transmitted to the originator of the access request.
After the difference data analysis result is obtained, the difference data analysis result can be generated into a visual interface and sent to the access request initiating client side, so that difference data with intuitiveness and easy visibility can be provided according to the user access requirement under the condition that a user does not sense, and unnecessary troubles caused by feeding back texts without any format or style and mixed together are avoided through the structuring processing of the step S130.
When the user receives the analysis result of the difference data, the difference data can be processed to some extent according to a specific application scenario, such as deletion, recovery, and the like. After the user triggers the post-processing instruction, processing actions such as deletion, recovery and the like in the post-processing instruction can be acquired in response to the post-processing instruction; and further processing the difference data in the difference data analysis result according to the processing action, namely deleting or restoring the difference data in the difference data analysis result in the issued service data. The processing history result can be kept by the visual background service, and the user can check the data in the background at any time and restore the data in full or single mode at any time.
In the access method of the differential data in the exemplary embodiment of the disclosure, an access request of an initiator for accessing a target webpage is received, and a data source address in the access request is analyzed; acquiring a data extraction strategy corresponding to the data source address, and extracting corresponding characteristic data according to the data extraction strategy; carrying out structuralization processing on the characteristic data to obtain structuralized data; and acquiring the published service data, carrying out differentiation comparison on the structured data and the service data to generate a differential data analysis result, and sending the differential data analysis result to the initiator of the access request. The method and the device can respond to data access to provide timely and convenient difference data analysis and difference data analysis result feedback.
It should be noted that although the various steps of the methods of the present disclosure are depicted in the drawings in a particular order, this does not require or imply that these steps must be performed in this particular order, or that all of the depicted steps must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions, etc.
In addition, in the present exemplary embodiment, an apparatus for accessing difference data is also provided. Referring to fig. 3, the accessing device 300 for difference data may include: the system comprises an access request receiving module 310, a characteristic data extracting module 320, a structured data processing module 330, a data differentiation comparing module 340 and an analysis result sending module 250. Wherein:
an access request receiving module 310, configured to receive an access request for an initiator to access a target webpage, and analyze a data source address in the access request;
the feature data extraction module 320 is configured to obtain a data extraction policy corresponding to the data source address, and extract corresponding feature data according to the data extraction policy;
the structured data processing module 330 is configured to perform structured processing on the feature data to obtain structured data;
the data differentiation comparison module 340 is configured to obtain published service data, perform differentiation comparison on the structured data and the service data, and generate a differential data analysis result;
an analysis result sending module 350, configured to send the difference data analysis result to the initiator of the access request.
The specific details of the access device module for each difference data have been described in detail in the access method for the corresponding difference data, and therefore are not described herein again.
It should be noted that although in the above detailed description several modules or units of the access means 300 of difference data are mentioned, this division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.
In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.
An electronic device 400 according to such an embodiment of the invention is described below with reference to fig. 4. The electronic device 400 shown in fig. 4 is only an example and should not bring any limitation to the function and the scope of use of the embodiments of the present invention.
As shown in fig. 4, electronic device 400 is embodied in the form of a general purpose computing device. The components of electronic device 400 may include, but are not limited to: the at least one processing unit 410, the at least one memory unit 420, a bus 430 connecting different system components (including the memory unit 420 and the processing unit 410), and a display unit 440.
Wherein the storage unit stores program code that is executable by the processing unit 410 to cause the processing unit 410 to perform steps according to various exemplary embodiments of the present invention as described in the above section "exemplary methods" of the present specification. For example, the processing unit 410 may perform steps S110 to S140 as shown in fig. 1.
The storage unit 420 may include readable media in the form of volatile storage units, such as a random access memory unit (RAM)4201 and/or a cache memory unit 4202, and may further include a read only memory unit (ROM) 4204.
The storage unit 420 may also include a program/utility 4204 having a set (at least one) of program modules 4205, such program modules 4205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The electronic device 400 may also communicate with one or more external devices 470 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 400, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 400 to communicate with one or more other computing devices. Such communication may occur via input/output (I/O) interfaces 450. Also, the electronic device 400 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via the network adapter 460. As shown, the network adapter 460 communicates with the other modules of the electronic device 400 over the bus 430. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with electronic device 400, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.
In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium having stored thereon a program product capable of implementing the above-described method of the present specification. In some possible embodiments, aspects of the invention may also be implemented in the form of a program product comprising program code means for causing a terminal device to carry out the steps according to various exemplary embodiments of the invention described in the above-mentioned "exemplary methods" section of the present description, when said program product is run on the terminal device.
Referring to fig. 5, a program product 500 for implementing the above method according to an embodiment of the present invention is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited in this regard and, in the present document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
Furthermore, the above-described figures are merely schematic illustrations of processes involved in methods according to exemplary embodiments of the invention, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is to be limited only by the terms of the appended claims.
Claims (10)
1. A method for accessing difference data, which is applied to an access proxy server, and comprises the following steps:
receiving an access request of an initiator for accessing a target webpage, and analyzing a data source address in the access request;
acquiring a data extraction strategy corresponding to the data source address, and extracting corresponding characteristic data according to the data extraction strategy;
carrying out structuralization processing on the characteristic data to obtain structuralized data;
acquiring published service data, and carrying out differentiation comparison on the structured data and the service data to generate a differential data analysis result;
and sending the analysis result of the difference data to the initiator of the access request.
2. The method of claim 1, wherein after receiving an access request from an initiator to access a target webpage and parsing a data source address corresponding to the access request, the method further comprises:
adding the address identifier of the access proxy server to the header information of the access request, and sending the access request added with the address identifier to a target server corresponding to a data source address after packaging;
receiving data response information replied by the target server, and acquiring source data in the data response information;
extracting corresponding feature data according to the data extraction strategy, wherein the extracting comprises the following steps:
and extracting characteristic data corresponding to the data source address in source data according to the data extraction strategy.
3. The method of claim 2, wherein extracting corresponding feature data according to the data extraction policy comprises:
the data extraction strategy comprises dimension information corresponding to each data source address; after source data in the data response information is obtained, determining appointed dimension information corresponding to a current data source address;
extracting at least one feature data corresponding to the specified dimension information from the source data.
4. The method of claim 1, wherein after receiving an access request from an initiator to access a target webpage and parsing a data source address corresponding to the access request, the method further comprises:
setting an access information quantity identifier in the access request, packaging the access request added with the access information quantity identifier, and sending the packaged access request to a target server corresponding to a data source address;
and receiving data response information replied by the target server, wherein an access result set is included in the data response information, and the number of the access results in the access result set is the number represented by the access information number identification.
5. The method of claim 1, wherein structuring the feature data to obtain structured data comprises:
when the feature data are numerical features, carrying out structuring processing on the numerical features according to a preset numerical feature processing rule;
when the feature data is a first text feature, matching and structuring the first text feature with a preset text feature processing rule;
when the feature data is a second text feature, extracting entity data in the second text feature; filtering the entity data according to an emotion judgment algorithm; and carrying out rule matching on the filtered entity data according to a preset expression rule, and generating structured data.
6. The method of claim 1, wherein differentially comparing the structured data to the service data to generate a differential data analysis result comprises:
calculating the structured data according to a preset data abstract algorithm to obtain an abstract characteristic value;
traversing a pre-stored service database, calculating whether the pre-stored service database has service data matched with the abstract characteristic value, if not, determining that the structured data has differences, and generating a difference data analysis result according to the structured data.
7. The method of claim 1, wherein the method further comprises:
responding to a post-processing instruction, and acquiring a processing action in the post-processing instruction; the processing action comprises deleting and restoring operations;
and processing the difference data in the difference data analysis result according to the processing action.
8. An apparatus for accessing difference data, comprising:
the access request receiving module is used for receiving an access request of an initiator for accessing a target webpage and analyzing a data source address in the access request;
the characteristic data extraction module is used for acquiring a data extraction strategy corresponding to the data source address and extracting corresponding characteristic data according to the data extraction strategy;
the structured data processing module is used for carrying out structured processing on the characteristic data to obtain structured data;
the data differentiation comparison module is used for acquiring published service data, carrying out differentiation comparison on the structured data and the service data and generating a differential data analysis result;
and the analysis result sending module is used for sending the difference data analysis result to the initiator of the access request.
9. An electronic device, comprising:
a memory for storing a computer program;
a processor for executing a computer program stored in the memory, and when executed, implementing the method of any of the preceding claims 1-7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method of any one of the preceding claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011059839.4A CN112148953B (en) | 2020-09-30 | Access method and device for difference data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011059839.4A CN112148953B (en) | 2020-09-30 | Access method and device for difference data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112148953A true CN112148953A (en) | 2020-12-29 |
| CN112148953B CN112148953B (en) | 2024-06-07 |
Family
ID=
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103973682A (en) * | 2014-04-30 | 2014-08-06 | 北京奇虎科技有限公司 | Method and device for having access to webpage |
| CN104125121A (en) * | 2014-08-15 | 2014-10-29 | 携程计算机技术（上海）有限公司 | Network hijacking behavior detecting system and method |
| CN107729358A (en) * | 2017-09-01 | 2018-02-23 | 小草数语（北京）科技有限公司 | Page access method and apparatus based on browser |
| CN108446394A (en) * | 2018-03-26 | 2018-08-24 | 网易（杭州）网络有限公司 | The control methods of file difference and device |
| CN108536805A (en) * | 2018-04-04 | 2018-09-14 | 平安科技（深圳）有限公司 | web access method and device |
| CN108717435A (en) * | 2018-05-16 | 2018-10-30 | 腾讯科技（深圳）有限公司 | Webpage loading method, information processing method, computer equipment and storage medium |
| US10339646B1 (en) * | 2019-01-14 | 2019-07-02 | Sourcewater, Inc. | Image processing of aerial imagery for energy infrastructure analysis using pre-processing image selection |
| CN110502563A (en) * | 2019-08-26 | 2019-11-26 | 腾讯科技（深圳）有限公司 | A kind of processing method and processing device of the data of multi-data source, storage medium |
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103973682A (en) * | 2014-04-30 | 2014-08-06 | 北京奇虎科技有限公司 | Method and device for having access to webpage |
| CN104125121A (en) * | 2014-08-15 | 2014-10-29 | 携程计算机技术（上海）有限公司 | Network hijacking behavior detecting system and method |
| CN107729358A (en) * | 2017-09-01 | 2018-02-23 | 小草数语（北京）科技有限公司 | Page access method and apparatus based on browser |
| CN108446394A (en) * | 2018-03-26 | 2018-08-24 | 网易（杭州）网络有限公司 | The control methods of file difference and device |
| CN108536805A (en) * | 2018-04-04 | 2018-09-14 | 平安科技（深圳）有限公司 | web access method and device |
| CN108717435A (en) * | 2018-05-16 | 2018-10-30 | 腾讯科技（深圳）有限公司 | Webpage loading method, information processing method, computer equipment and storage medium |
| US10339646B1 (en) * | 2019-01-14 | 2019-07-02 | Sourcewater, Inc. | Image processing of aerial imagery for energy infrastructure analysis using pre-processing image selection |
| CN110502563A (en) * | 2019-08-26 | 2019-11-26 | 腾讯科技（深圳）有限公司 | A kind of processing method and processing device of the data of multi-data source, storage medium |
Non-Patent Citations (2)
| Title |
|---|
| SUNYOUNG CHUNG 等: "The virtualized MPTCP proxy performance in celluar network", 《2017 NINTH INTERNATIONAL CONFERENCE ON UBIQUITOUS AND FUTURE NETWORKS（ICUFN）》, pages 1 - 2 * |
| 马云龙 等: "电力调度系统广域分布式代理关键技术", 《电力系统及其自动化学报》, vol. 30, no. 3, pages 62 - 68 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108572990B (en) | Information pushing method and device | |
| EP4099170B1 (en) | Method and apparatus of auditing log, electronic device, and medium | |
| CN112162965B (en) | Log data processing method, device, computer equipment and storage medium | |
| CN112163072B (en) | Data processing method and device based on multiple data sources | |
| CN108090351B (en) | Method and apparatus for processing request message | |
| CN107203574B (en) | Aggregation of data management and data analysis | |
| CN111158795A (en) | Report generation method, device, medium and electronic equipment | |
| US10671686B2 (en) | Processing webpage data | |
| CN112084179B (en) | Data processing method, device, equipment and storage medium | |
| US11423219B2 (en) | Generation and population of new application document utilizing historical application documents | |
| WO2023092580A1 (en) | Page display method and apparatus, storage medium, and electronic device | |
| CN113934733A (en) | Problem positioning method, device, system, storage medium and electronic equipment | |
| US10346533B2 (en) | Management of content tailoring by services | |
| CN116562255B (en) | Form information generation method, form information generation device, electronic equipment and computer readable medium | |
| CN113268453A (en) | Log information compression storage method and device | |
| CN112131292A (en) | Method and device for structural processing of changed data | |
| CN113792138B (en) | Report generation method and device, electronic equipment and storage medium | |
| CN112148953B (en) | Access method and device for difference data | |
| CN112148953A (en) | Method and device for accessing difference data | |
| CN115510247A (en) | Method, device, equipment and storage medium for constructing electric carbon policy knowledge graph | |
| CN113032515A (en) | Method, system, device and storage medium for generating chart based on multiple data sources | |
| CN113760568A (en) | Data processing method and device | |
| CN113138974A (en) | Database compliance detection method and device | |
| CN114765599A (en) | Sub-domain name acquisition method and device | |
| CN112416713A (en) | Operation auditing system and method, computer readable storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |