CN110851750B - Data acquisition method and device - Google Patents
Data acquisition method and device Download PDFInfo
- Publication number
- CN110851750B CN110851750B CN201810861158.6A CN201810861158A CN110851750B CN 110851750 B CN110851750 B CN 110851750B CN 201810861158 A CN201810861158 A CN 201810861158A CN 110851750 B CN110851750 B CN 110851750B
- Authority
- CN
- China
- Prior art keywords
- basic data
- identification information
- link address
- authentication
- data acquisition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Information Transfer Between Computers (AREA)
- Storage Device Security (AREA)
Abstract
The invention discloses a data acquisition method and a data acquisition device, which are used for acquiring configuration information corresponding to a link address accessed by a user when the link address is acquired, analyzing the configuration information, acquiring identification information of each basic data in the configuration information and a basic data acquisition mode corresponding to each identification information, and acquiring the basic data with corresponding identification information by using the basic data acquisition modes corresponding to various identification information. By the method and the device, the basic data required by the page corresponding to the link address accessed by the user can be acquired, and the necessary data of each page in the Web application does not need to be acquired, so that the waste of server resources can be avoided, and the page loading time is shortened.
Description
Technical Field
The present invention relates to the field of communications, and in particular, to a data acquisition method and apparatus.
Background
When entering a certain sub-page in the Web application, all necessary data of the main page and each sub-page entering the Web application need to be acquired, where the necessary data refers to user information, such as a user name, a password, and user age.
However, if a sub-page in the Web application is entered, the necessary data required by the main page and the remaining sub-pages are not necessary, and all the necessary data are obtained at this time, which causes waste of server resources and prolongs the page loading time.
Disclosure of Invention
In view of the above, the present invention has been made to provide a data acquisition method and apparatus that overcomes or at least partially solves the above problems.
A method of data acquisition, comprising:
when a link address accessed by a user is acquired, acquiring configuration information corresponding to the link address; the configuration information comprises identification information of at least one basic data required for loading the page corresponding to the link address and a basic data acquisition mode corresponding to each identification information;
analyzing the configuration information to obtain identification information of each type of basic data in the configuration information and a basic data acquisition mode corresponding to each type of identification information;
and acquiring basic data with corresponding identification information by using various basic data acquisition modes corresponding to the identification information.
Preferably, before acquiring the link address accessed by the user, the data acquisition method further includes:
determining identification information of at least one basic data required for loading the page corresponding to the link address, and determining a basic data acquisition mode of each basic data;
and establishing corresponding relations among the identification information of each kind of basic data, the basic data acquisition mode of each kind of basic data and the link address, and generating configuration information based on at least one established corresponding relation.
Preferably, after determining the basic data obtaining mode of each basic data, the method further includes:
determining whether the page corresponding to the link address needs authentication operation or not, and if so, determining the authentication identifier corresponding to the link address as a first identifier; otherwise, determining the authentication identifier corresponding to the link address as a second identifier; the authentication identifier is used for identifying whether the page corresponding to the link address is loaded and needs authentication operation or not;
correspondingly, establishing a corresponding relationship between the identification information of each kind of basic data, the basic data acquisition mode of each kind of basic data, and the link address, and generating configuration information based on at least one of the established corresponding relationships, includes:
establishing identification information of each kind of basic data, a basic data acquisition mode of each kind of basic data, a corresponding relation between the authentication identification and the link address, and generating configuration information based on at least one established corresponding relation.
Preferably, the configuration information further includes: the authentication identifier is used for identifying whether the page corresponding to the link address needs authentication operation or not;
correspondingly, the step of analyzing the configuration information to obtain the identification information of each type of basic data in the configuration information and the basic data obtaining mode corresponding to each type of identification information includes:
analyzing the configuration information to obtain identification information of each type of basic data in the configuration information, a basic data acquisition mode corresponding to each type of identification information and the authentication identification;
after the step of analyzing the configuration information to obtain the identification information of each type of basic data in the configuration information and the basic data acquisition mode corresponding to each type of identification information, and before the step of acquiring the basic data with corresponding identification information by using the basic data acquisition mode corresponding to each type of identification information, the method further includes:
determining whether authentication is needed according to the authentication identification, if so, executing authentication processing and obtaining an authentication result; otherwise, executing the step of acquiring basic data with corresponding identification information by using the basic data acquisition mode corresponding to various identification information;
and if the authentication result is passed, then executing the basic data acquisition mode corresponding to the various identification information to acquire the basic data with the corresponding identification information.
Preferably, if the authentication result is passed, the obtaining the basic data with the corresponding identification information by using the basic data obtaining manner corresponding to the various identification information includes:
acquiring authentication information used in the authentication processing;
determining whether the authentication information is at least one of the at least one basic data;
if so, acquiring other basic data except the authentication information by using a basic data acquisition mode corresponding to the identification information of other basic data except the authentication information;
if not, acquiring basic data with corresponding identification information by using various basic data acquisition modes corresponding to the identification information.
A data acquisition apparatus comprising:
the first information acquisition module is used for acquiring configuration information corresponding to a link address accessed by a user when the link address is acquired; the configuration information comprises identification information of at least one basic data required for loading the page corresponding to the link address and a basic data acquisition mode corresponding to each identification information;
the information analysis module is used for analyzing the configuration information to obtain the identification information of each type of basic data in the configuration information and the basic data acquisition mode corresponding to each type of identification information;
and the second information acquisition module is used for acquiring basic data with corresponding identification information by using various basic data acquisition modes corresponding to the identification information.
Preferably, the method further comprises the following steps:
the information determining module is used for determining the identification information of at least one basic data required for loading the page corresponding to the link address and determining the basic data acquiring mode of each basic data before the first information acquiring module acquires the link address accessed by the user;
and the information generation module is used for establishing the corresponding relation between the identification information of each type of basic data, the basic data acquisition mode of each type of basic data and the link address, and generating configuration information based on at least one established corresponding relation.
Preferably, the method further comprises the following steps:
an operation determining module, configured to determine, by the information determining module, identification information of at least one basic data required for loading a page corresponding to the link address, and after determining a basic data obtaining manner of each basic data, determine whether an authentication operation is required for loading the page corresponding to the link address;
the first identifier determining module is used for determining that the page corresponding to the link address needs authentication operation when the operation determining module determines to load the page corresponding to the link address, and determining the authentication identifier corresponding to the link address as a first identifier; the authentication identifier is used for identifying whether the page corresponding to the link address is loaded and needs authentication operation or not;
the second identifier determining module is used for determining that the page corresponding to the link address is loaded without authentication operation by the operation determining module and determining the authentication identifier corresponding to the link address as a second identifier; the authentication identifier is used for identifying whether the page corresponding to the link address is loaded and needs authentication operation or not;
correspondingly, the information generating module is configured to establish correspondence between the identification information of each type of basic data, the basic data acquisition mode of each type of basic data, and the link address, and when generating the configuration information based on at least one of the established correspondence, specifically configured to:
establishing identification information of each basic data, a basic data acquisition mode of each basic data, a corresponding relation between the authentication identification and the link address, and generating configuration information based on at least one established corresponding relation.
A storage medium comprising a stored program, wherein the program performs the data acquisition method described above.
A processor, configured to run a program, wherein the program executes the data acquisition method.
By means of the technical scheme, the data acquisition method and the data acquisition device can acquire the basic data required by the page corresponding to the link address accessed by the user, and do not need to acquire necessary data of each page in the Web application, so that waste of server resources can be avoided, and page loading time can be shortened.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
fig. 1 shows a method flowchart of a data acquisition method according to an embodiment of the present invention;
FIG. 2 is a flow chart of a method of another data acquisition method provided by an embodiment of the invention;
FIG. 3 is a flow chart of a method of another data acquisition method according to an embodiment of the present invention;
fig. 4 shows a schematic structural diagram of a data acquisition apparatus according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
An embodiment of the present invention provides a data acquisition method, and with reference to fig. 1, the data acquisition method may include:
s11, when a link address accessed by a user is acquired, acquiring configuration information corresponding to the link address;
the configuration information comprises identification information of at least one basic data required for loading the page corresponding to the link address and a basic data acquisition mode corresponding to each identification information.
Specifically, the link address accessed by the user may be a link address input by the user, or a link address determined by the user by clicking a button of the page.
When the link address is acquired, sending a static file corresponding to the acquired link address to a background server; wherein the static file includes page rendering flow data.
Wherein, the static file may include a configuration information acquiring program, an analysis program and a data acquiring program, and the effect of steps S11-S13 can be obtained by loading the static file.
In addition, the static file may further include data calling the configuration information acquiring program, the parsing program, and the data acquiring program, and the steps S11-S13 may be further implemented by calling the configuration information acquiring program, the parsing program, and the data acquiring program.
The basic data refers to data necessary for entering a page, and the basic data may be user data, such as a user name, a password, and the like, or a page entry number, and the like. The basic data may be classified into a plurality of classes according to categories.
S12, analyzing the configuration information to obtain the identification information of each basic data in the configuration information and the basic data acquisition mode corresponding to each identification information;
specifically, the process of analyzing the configuration information is a process of acquiring the identification information and the basic data acquisition mode corresponding to each identification information from the configuration information.
And S13, acquiring basic data with corresponding identification information by using various basic data acquisition modes corresponding to the identification information.
Specifically, the number of the basic data may be one or more, without limitation. When the number of the basic data is multiple, each kind of basic data corresponds to one identification information and a basic data acquisition mode, and the identification information corresponding to each kind of basic data is different, but the basic data acquisition modes corresponding to the identification information of each kind of basic data may be the same or different. For example, each basic data may be obtained by using a different basic data obtaining method, or at least two basic data may be obtained by using the same basic data obtaining method.
In this step, for each type of basic data, the basic data acquisition mode corresponding to the basic data is used to acquire the data.
In the embodiment, the basic data required by the page corresponding to the link address accessed by the user can be acquired, and the necessary data of each page in the Web application does not need to be acquired, so that the resource waste of the server can be avoided, and the page loading time can be shortened.
Optionally, on the basis of the embodiment of the data obtaining method, referring to fig. 2, before obtaining the link address accessed by the user, the data obtaining method further includes:
s21, determining the identification information of at least one basic data required by loading the page corresponding to the link address, and determining the basic data acquisition mode of each basic data;
specifically, basic data required for loading a page corresponding to each link address is fixed, and here, identification information of the basic data needs to be obtained, for example, identification information of the basic data a is 001, and identification information of the basic data B is 002. Wherein the basic data A and the basic data B belong to different categories.
In addition, the basic data can include one or more items of content, for example, the basic data can include only one item of content of the user name, or can include two items of content of the user name and the password.
And configuring basic data acquisition modes of basic data through the routing configuration page, wherein different basic data acquisition modes can adopt different ports to acquire data, and the basic data acquisition modes can be acquired from a background server or input by a front-end user. If the data is input by a front-end user, an input interface can be rendered to enable the user to input the data.
It should be noted that the identification information of at least one basic data required for loading the page corresponding to the link address and the basic data acquisition mode of each basic data may be executed simultaneously or sequentially.
S22, establishing the corresponding relation between the identification information of each basic data, the basic data acquisition mode of each basic data and the link address;
specifically, after the identification information of the basic data and the basic data acquisition mode of the basic data are determined, the corresponding relationship among the identification information of each kind of basic data, the basic data acquisition mode of each kind of basic data, and the link address needs to be established.
A corresponding relation is formed by the identification information of basic data, the basic data acquisition mode of the basic data and the link address, and the corresponding relation is established according to the type of the basic data.
S23, generating configuration information based on the at least one established corresponding relation.
The configuration information comprises at least one identification information of each kind of basic data, and the corresponding relation between the basic data acquisition mode of each kind of basic data and the link address. And then, when the configuration information is analyzed, the identification information of each type of basic data corresponding to the link address in the configuration information and the basic data acquisition mode of each type of basic data are searched.
Optionally, on the basis of this embodiment, after the step S23, the method may further include:
determining whether the page corresponding to the link address needs authentication operation or not, and if so, determining the authentication identifier corresponding to the link address as a first identifier; otherwise, determining the authentication identifier corresponding to the link address as a second identifier;
and the authentication identifier is used for identifying whether the page corresponding to the link address is loaded and needs authentication operation.
Specifically, whether the page corresponding to the link address needs authentication operation or not is configured and loaded on the route configuration interface. The authentication operation is to determine whether the user has the right to enter the page, and partial pages do not need to authenticate the user, that is, each user can enter the page, but partial pages need the user with the right to enter the page.
If one page needs authentication operation, the authentication identifier corresponding to the link address is determined as a first identifier, and the first identifier can be 'authentication needed'.
And if one page does not need authentication operation, determining the authentication identifier corresponding to the link address as a second identifier, wherein the second identifier can be 'authentication is not needed'.
Correspondingly, establishing a corresponding relationship between the identification information of each kind of basic data, the basic data acquisition mode of each kind of basic data, and the link address, and generating configuration information based on at least one of the established corresponding relationships, includes:
establishing identification information of each basic data, a basic data acquisition mode of each basic data, a corresponding relation between the authentication identification and the link address, and generating configuration information based on at least one established corresponding relation.
Specifically, since the authentication identifier is added, the authentication identifier also needs to be associated with the link address, and thus the authentication identifier may be added to the configuration information.
In this embodiment, after the identification information of each type of basic data, the basic data acquisition mode of each type of basic data, and the authentication identifier are determined, the correspondence between the identification information of each type of basic data, the basic data acquisition mode of each type of basic data, and the link address is established, and the configuration information is generated based on at least one of the established correspondence, so that the configuration information can be generated. When a page is loaded subsequently, the configuration information corresponding to the page can be obtained, and the basic data can be obtained according to the configuration information.
It should be noted that, for the explanation of the remaining steps in fig. 2, please refer to the corresponding explanation in the above embodiment, and details are not repeated herein.
Optionally, on the basis of the embodiment corresponding to fig. 1, the configuration information further includes: the authentication identifier is used for identifying whether the page corresponding to the link address needs authentication operation or not;
for a specific explanation of the authentication identifier, please refer to the corresponding description in the above embodiment, which is not described herein again.
Further, step S12 specifically includes:
and analyzing the configuration information to obtain the identification information of each type of basic data in the configuration information, the basic data acquisition mode corresponding to each type of identification information and the authentication identification.
Further, referring to fig. 3, between the step S32 and the step S36, the method further includes:
s33, determining whether authentication is needed according to the authentication identification, if yes, executing step S34; if not, go to step S36.
S34, executing authentication processing and obtaining an authentication result;
specifically, the authentication process is to verify the identity of the user, such as verifying whether the user name and the password are correct. The authentication result is divided into a pass result and a non-pass result.
S35, judging whether the authentication result is passed; if yes, go to step S36; if not, the process is ended.
Optionally, on the basis of this embodiment, if the authentication result is that the authentication result passes, step S36 specifically includes:
1) acquiring authentication information used in the authentication processing;
specifically, if the user has logged in, the authentication information is obtained at this time, so as to perform the authentication operation. The authentication information may be data such as a user name and a password.
2) Determining whether the authentication information is at least one of the at least one basic data;
specifically, since the basic data is preset, only the judgment is needed at this time.
3) If so, acquiring other basic data except the authentication information by using a basic data acquisition mode corresponding to the identification information of other basic data except the authentication information;
4) if not, acquiring basic data with corresponding identification information by using various basic data acquisition modes corresponding to the identification information. If the authentication information is at least one of the at least one kind of basic data, taking the authentication information as one kind of basic data, and acquiring the rest of basic data by adopting a basic data acquisition mode corresponding to the identification information of the rest of basic data; and if the authentication information is not at least one of the at least one basic data, acquiring each basic data by adopting a basic data acquisition mode of acquiring each basic data.
It should be noted that, the program corresponding to the above steps may be a hook function, and the hook function may be called.
In the embodiment, the authentication operation is performed before the basic data is acquired, so that the user can log in the page only when the user is a legal user, the data leakage is avoided, and the security of the data is ensured.
Optionally, corresponding to the data acquiring method, another embodiment of the present invention provides a data acquiring apparatus, and referring to fig. 4, the data acquiring apparatus may include:
the first information acquisition module 101 is configured to acquire configuration information corresponding to a link address accessed by a user when the link address is acquired; the configuration information comprises identification information of at least one basic data required for loading the page corresponding to the link address and a basic data acquisition mode corresponding to each identification information;
the information analysis module 102 is configured to analyze the configuration information to obtain identification information of each type of basic data in the configuration information and a basic data acquisition mode corresponding to each type of identification information;
the second information obtaining module 103 is configured to obtain basic data with corresponding identification information by using various basic data obtaining manners corresponding to the identification information.
In this embodiment, the basic data required by the page corresponding to the link address accessed by the user can be acquired, and the necessary data of each page in the Web application does not need to be acquired, so that the waste of server resources can be avoided, and the page loading time can be shortened.
It should be noted that, for the working process of each module in this embodiment, please refer to the corresponding description in the above embodiments, which is not described herein again.
Optionally, on the basis of the embodiment of the data acquiring apparatus, the data acquiring apparatus further includes:
the information determining module is used for determining the identification information of at least one basic data required for loading the page corresponding to the link address and determining the basic data acquiring mode of each basic data before the first information acquiring module acquires the link address accessed by the user;
and the information generation module is used for establishing the corresponding relation among the identification information of each type of basic data, the basic data acquisition mode of each type of basic data and the link address, and generating configuration information based on at least one established corresponding relation.
Optionally, on the basis of this embodiment, the method further includes:
an operation determining module, configured to determine, by the information determining module, identification information of at least one basic data required for loading a page corresponding to the link address, and after determining a basic data obtaining manner of each basic data, determine whether an authentication operation is required for loading the page corresponding to the link address;
the first identifier determining module is used for determining that the page corresponding to the link address needs authentication operation when the operation determining module determines to load the page corresponding to the link address, and determining the authentication identifier corresponding to the link address as a first identifier; the authentication identifier is used for identifying whether the page corresponding to the link address is loaded and needs authentication operation or not;
the second identifier determining module is used for determining that the page corresponding to the link address is loaded without authentication operation by the operation determining module and determining the authentication identifier corresponding to the link address as a second identifier; the authentication identifier is used for identifying whether the page corresponding to the link address is loaded and needs authentication operation or not;
correspondingly, the information generating module is configured to establish correspondence between the identification information of each type of basic data, the basic data acquisition mode of each type of basic data, and the link address, and when generating the configuration information based on at least one of the established correspondence, specifically configured to:
establishing identification information of each basic data, a basic data acquisition mode of each basic data, a corresponding relation between the authentication identification and the link address, and generating configuration information based on at least one established corresponding relation.
In this embodiment, after the identification information of each type of basic data, the basic data acquisition manner of each type of basic data, and the authentication identification are determined, the correspondence between the identification information of each type of basic data, the basic data acquisition manner of each type of basic data, and the link address is established, and the configuration information is generated based on at least one of the established correspondence, so that the configuration information can be generated. When a page is loaded subsequently, the configuration information corresponding to the page can be obtained, and the basic data can be obtained according to the configuration information.
It should be noted that, please refer to the corresponding description in the above embodiments for the working process of each module in this embodiment, which is not described herein again.
Optionally, on the basis of any embodiment of the data obtaining apparatus, the configuration information further includes: the authentication identifier is used for identifying whether the page corresponding to the link address needs authentication operation or not;
correspondingly, the information analysis module is configured to, when analyzing the configuration information and obtaining the identification information of each type of basic data in the configuration information and the basic data obtaining manner corresponding to each type of identification information, specifically:
analyzing the configuration information to obtain identification information of each kind of basic data in the configuration information, a basic data acquisition mode corresponding to each kind of identification information and the authentication identification;
the data acquisition device further comprises;
the authentication determining module is used for determining whether authentication is needed or not according to the authentication identifier before the second information acquiring module acquires the basic data with the corresponding identification information by using the basic data acquiring mode corresponding to each identification information after the information analyzing module analyzes the configuration information and acquires the identification information of each basic data in the configuration information and the basic data acquiring mode corresponding to each identification information;
the authentication operation module is used for determining that authentication is needed by the authentication determining module according to the authentication identifier, executing authentication processing and obtaining an authentication result;
the second information obtaining module is also used for the authentication determining module to determine that the authentication is not needed or the authentication result obtained by the authentication operating module is passed according to the authentication identification, and obtaining the basic data with the corresponding identification information by using the basic data obtaining mode corresponding to various identification information.
Optionally, on the basis of this embodiment, if the authentication result is that the authentication result passes, the second information obtaining module is specifically configured to, when obtaining the basic data with the corresponding identification information by using the basic data obtaining manner corresponding to each of the identification information:
acquiring authentication information used in the authentication processing;
determining whether the authentication information is at least one of the at least one basic data;
if so, acquiring other basic data except the authentication information by using a basic data acquisition mode corresponding to the identification information of other basic data except the authentication information;
if not, acquiring basic data with corresponding identification information by using various basic data acquisition modes corresponding to the identification information.
In the embodiment, the authentication operation is performed before the basic data is acquired, so that the user can log in the page only when the user is a legal user, the data leakage is avoided, and the security of the data is ensured.
It should be noted that, please refer to the corresponding description in the above embodiments for the working process of each module in this embodiment, which is not described herein again.
Optionally, an embodiment of the present invention further provides a data acquisition apparatus, where the data acquisition apparatus includes a processor and a memory, and the first information acquisition module, the information analysis module, the second information acquisition module, and the like are all stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions.
The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can be set to be one or more than one, and the purposes of avoiding the waste of server resources and shortening the page loading time are realized by adjusting the kernel parameters.
The memory may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.
An embodiment of the present invention provides a storage medium on which a program is stored, the program implementing the data acquisition method when executed by a processor.
The embodiment of the invention provides a processor, which is used for running a program, wherein the data acquisition method is executed when the program runs.
The embodiment of the invention provides equipment, which comprises a processor, a memory and a program which is stored on the memory and can run on the processor, wherein the processor executes the program and realizes the following steps:
a method of data acquisition, comprising:
when a link address accessed by a user is acquired, acquiring configuration information corresponding to the link address; the configuration information comprises identification information of at least one basic data required for loading the page corresponding to the link address and a basic data acquisition mode corresponding to each identification information;
analyzing the configuration information to obtain identification information of each type of basic data in the configuration information and a basic data acquisition mode corresponding to each type of identification information;
and acquiring basic data with corresponding identification information by using various basic data acquisition modes corresponding to the identification information.
Further, before acquiring the link address accessed by the user, the data acquisition method further includes:
determining identification information of at least one basic data required for loading the page corresponding to the link address, and determining a basic data acquisition mode of each basic data;
and establishing corresponding relations among the identification information of each kind of basic data, the basic data acquisition mode of each kind of basic data and the link address, and generating configuration information based on at least one established corresponding relation.
Further, after determining the basic data acquisition mode of each basic data, the method further comprises the following steps:
determining whether the page corresponding to the link address needs authentication operation or not, and if so, determining the authentication identifier corresponding to the link address as a first identifier; otherwise, determining the authentication identifier corresponding to the link address as a second identifier; the authentication identifier is used for identifying whether the page corresponding to the link address is loaded and needs authentication operation or not;
correspondingly, establishing a corresponding relationship between the identification information of each kind of basic data, the basic data acquisition mode of each kind of basic data, and the link address, and generating configuration information based on at least one of the established corresponding relationships, includes:
establishing identification information of each kind of basic data, a basic data acquisition mode of each kind of basic data, a corresponding relation between the authentication identification and the link address, and generating configuration information based on at least one established corresponding relation.
Further, the configuration information further includes: the authentication identifier is used for identifying whether the page corresponding to the link address needs authentication operation or not;
correspondingly, the step of analyzing the configuration information to obtain the identification information of each type of basic data in the configuration information and the basic data obtaining mode corresponding to each type of identification information includes:
analyzing the configuration information to obtain identification information of each type of basic data in the configuration information, a basic data acquisition mode corresponding to each type of identification information and the authentication identification;
after the step of analyzing the configuration information to obtain the identification information of each type of basic data in the configuration information and the basic data acquisition mode corresponding to each type of identification information, and before the step of acquiring the basic data with corresponding identification information by using the basic data acquisition mode corresponding to each type of identification information, the method further comprises:
determining whether authentication is needed according to the authentication identification, if so, executing authentication processing and obtaining an authentication result; otherwise, executing the step of acquiring basic data with corresponding identification information by using the basic data acquisition mode corresponding to various identification information;
and if the authentication result is passed, then executing the basic data acquisition mode corresponding to the various identification information to acquire the basic data with the corresponding identification information.
Further, if the authentication result is that the authentication result passes, the obtaining of the basic data with the corresponding identification information by using the basic data obtaining manner corresponding to the various identification information includes:
acquiring authentication information used in the authentication processing;
determining whether the authentication information is at least one of the at least one basic data;
if so, acquiring other basic data except the authentication information by using a basic data acquisition mode corresponding to the identification information of other basic data except the authentication information;
if not, acquiring basic data with corresponding identification information by using various basic data acquisition modes corresponding to the identification information.
The device herein may be a server, a PC, a PAD, a mobile phone, etc.
The present application also provides a computer program product adapted to perform a program for initializing the following method steps when executed on a data processing device:
a method of data acquisition, comprising:
when a link address accessed by a user is acquired, acquiring configuration information corresponding to the link address; the configuration information comprises identification information of at least one basic data required for loading the page corresponding to the link address and a basic data acquisition mode corresponding to each identification information;
analyzing the configuration information to obtain identification information of each type of basic data in the configuration information and a basic data acquisition mode corresponding to each type of identification information;
and acquiring basic data with corresponding identification information by using various basic data acquisition modes corresponding to the identification information.
Further, before acquiring the link address accessed by the user, the data acquisition method further includes:
determining identification information of at least one basic data required for loading the page corresponding to the link address, and determining a basic data acquisition mode of each basic data;
and establishing corresponding relations among the identification information of each kind of basic data, the basic data acquisition mode of each kind of basic data and the link address, and generating configuration information based on at least one established corresponding relation.
Further, after determining the basic data acquisition mode of each basic data, the method further comprises the following steps:
determining whether the page corresponding to the link address needs authentication operation or not, and if so, determining the authentication identifier corresponding to the link address as a first identifier; otherwise, determining the authentication identifier corresponding to the link address as a second identifier; the authentication identifier is used for identifying whether the page corresponding to the link address is loaded and needs authentication operation or not;
correspondingly, establishing a corresponding relationship between the identification information of each kind of basic data, the basic data acquisition mode of each kind of basic data, and the link address, and generating configuration information based on at least one of the established corresponding relationships, includes:
establishing identification information of each basic data, a basic data acquisition mode of each basic data, a corresponding relation between the authentication identification and the link address, and generating configuration information based on at least one established corresponding relation.
Further, the configuration information further includes: the authentication identifier is used for identifying whether the page corresponding to the link address needs authentication operation or not;
correspondingly, the step of analyzing the configuration information to obtain the identification information of each type of basic data in the configuration information and the basic data obtaining mode corresponding to each type of identification information includes:
analyzing the configuration information to obtain identification information of each kind of basic data in the configuration information, a basic data acquisition mode corresponding to each kind of identification information and the authentication identification;
after the step of analyzing the configuration information to obtain the identification information of each type of basic data in the configuration information and the basic data acquisition mode corresponding to each type of identification information, and before the step of acquiring the basic data with corresponding identification information by using the basic data acquisition mode corresponding to each type of identification information, the method further includes:
determining whether authentication is needed according to the authentication identification, if so, executing authentication processing and obtaining an authentication result; otherwise, executing the step of acquiring basic data with corresponding identification information by using the basic data acquisition mode corresponding to various identification information;
and if the authentication result is passed, then executing the basic data acquisition mode corresponding to the various identification information to acquire the basic data with the corresponding identification information.
Further, if the authentication result is that the authentication result passes, the obtaining of the basic data with the corresponding identification information by using the basic data obtaining manner corresponding to the various identification information includes:
acquiring authentication information used in the authentication processing;
determining whether the authentication information is at least one of the at least one basic data;
if so, acquiring other basic data except the authentication information by using a basic data acquisition mode corresponding to the identification information of other basic data except the authentication information;
if not, acquiring basic data with corresponding identification information by using various basic data acquisition modes corresponding to the identification information.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.
The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (10)
1. A method of data acquisition, comprising:
when a link address accessed by a user is acquired, acquiring configuration information corresponding to the link address; the configuration information comprises identification information of at least one piece of basic data required by loading the page corresponding to the link address and a basic data acquisition mode corresponding to each identification information, wherein the basic data refers to user data required by entering one page;
analyzing the configuration information to obtain identification information of each type of basic data in the configuration information and a basic data acquisition mode corresponding to each type of identification information;
and acquiring basic data with corresponding identification information by using various basic data acquisition modes corresponding to the identification information.
2. The data acquisition method according to claim 1, wherein before acquiring the link address accessed by the user, the data acquisition method further comprises:
determining identification information of at least one basic data required for loading the page corresponding to the link address, and determining a basic data acquisition mode of each basic data;
and establishing corresponding relations among the identification information of each kind of basic data, the basic data acquisition mode of each kind of basic data and the link address, and generating configuration information based on at least one established corresponding relation.
3. The data acquisition method according to claim 2, further comprising, after determining the basic data acquisition mode for each basic data:
determining whether the page corresponding to the link address needs authentication operation or not, and if so, determining the authentication identifier corresponding to the link address as a first identifier; otherwise, determining the authentication identifier corresponding to the link address as a second identifier; the authentication identifier is used for identifying whether the page corresponding to the link address is loaded and needs authentication operation or not;
correspondingly, establishing a corresponding relationship between the identification information of each kind of basic data, the basic data acquisition mode of each kind of basic data, and the link address, and generating configuration information based on at least one of the established corresponding relationships, includes:
establishing identification information of each kind of basic data, a basic data acquisition mode of each kind of basic data, a corresponding relation between the authentication identification and the link address, and generating configuration information based on at least one established corresponding relation.
4. The data acquisition method of claim 1, wherein the configuration information further comprises: the authentication identifier is used for identifying whether the page corresponding to the link address needs authentication operation or not;
correspondingly, the step of analyzing the configuration information to obtain the identification information of each type of basic data in the configuration information and the basic data obtaining mode corresponding to each type of identification information includes:
analyzing the configuration information to obtain identification information of each type of basic data in the configuration information, a basic data acquisition mode corresponding to each type of identification information and the authentication identification;
after the step of analyzing the configuration information to obtain the identification information of each type of basic data in the configuration information and the basic data acquisition mode corresponding to each type of identification information, and before the step of acquiring the basic data with corresponding identification information by using the basic data acquisition mode corresponding to each type of identification information, the method further includes:
determining whether authentication is needed according to the authentication identification, if so, executing authentication processing and obtaining an authentication result; otherwise, executing the step of acquiring basic data with corresponding identification information by using the basic data acquisition mode corresponding to various identification information;
and if the authentication result is passed, then executing the basic data acquisition mode corresponding to the various identification information to acquire the basic data with the corresponding identification information.
5. The method according to claim 4, wherein if the authentication result is positive, the obtaining the basic data with the corresponding identification information using the basic data obtaining manner corresponding to each identification information comprises:
acquiring authentication information used in the authentication processing;
determining whether the authentication information is at least one of the at least one basic data;
if so, acquiring other basic data except the authentication information by using a basic data acquisition mode corresponding to the identification information of other basic data except the authentication information;
if not, acquiring basic data with corresponding identification information by using various basic data acquisition modes corresponding to the identification information.
6. A data acquisition apparatus, comprising:
the first information acquisition module is used for acquiring configuration information corresponding to a link address accessed by a user when the link address is acquired; the configuration information comprises identification information of at least one piece of basic data required by loading the page corresponding to the link address and a basic data acquisition mode corresponding to each identification information, wherein the basic data refers to user data required by entering one page;
the information analysis module is used for analyzing the configuration information to obtain identification information of each type of basic data in the configuration information and a basic data acquisition mode corresponding to each type of identification information;
and the second information acquisition module is used for acquiring basic data with corresponding identification information by using various basic data acquisition modes corresponding to the identification information.
7. The data acquisition device according to claim 6, further comprising:
the information determining module is used for determining the identification information of at least one basic data required for loading the page corresponding to the link address and determining the basic data acquiring mode of each basic data before the first information acquiring module acquires the link address accessed by the user;
and the information generation module is used for establishing the corresponding relation between the identification information of each type of basic data, the basic data acquisition mode of each type of basic data and the link address, and generating configuration information based on at least one established corresponding relation.
8. The data acquisition device according to claim 7, further comprising:
an operation determining module, configured to determine, by the information determining module, identification information of at least one type of basic data required for loading a page corresponding to the link address, and after determining a basic data obtaining manner of each type of basic data, determine whether an authentication operation is required for loading the page corresponding to the link address;
the first identifier determining module is used for determining that the page corresponding to the link address needs authentication operation when the operation determining module determines to load the page corresponding to the link address, and determining the authentication identifier corresponding to the link address as a first identifier; the authentication identifier is used for identifying whether the page corresponding to the link address is loaded and needs authentication operation or not;
the second identifier determining module is used for determining that the page corresponding to the link address is loaded without authentication operation by the operation determining module and determining the authentication identifier corresponding to the link address as a second identifier; the authentication identifier is used for identifying whether the page corresponding to the link address is loaded and needs authentication operation or not;
correspondingly, the information generating module is configured to establish a corresponding relationship between the identification information of each type of basic data, the basic data obtaining manner of each type of basic data, and the link address, and when generating the configuration information based on at least one of the established corresponding relationships, specifically configured to:
establishing identification information of each kind of basic data, a basic data acquisition mode of each kind of basic data, a corresponding relation between the authentication identification and the link address, and generating configuration information based on at least one established corresponding relation.
9. A storage medium characterized by comprising a stored program, wherein the program executes the data acquisition method of any one of claims 1 to 5.
10. A processor, characterized in that the processor is configured to run a program, wherein the program is configured to execute the data acquisition method according to any one of claims 1 to 5 when running.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810861158.6A CN110851750B (en) | 2018-08-01 | 2018-08-01 | Data acquisition method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810861158.6A CN110851750B (en) | 2018-08-01 | 2018-08-01 | Data acquisition method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110851750A CN110851750A (en) | 2020-02-28 |
| CN110851750B true CN110851750B (en) | 2022-07-22 |
Family
ID=69594356
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810861158.6A Active CN110851750B (en) | 2018-08-01 | 2018-08-01 | Data acquisition method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110851750B (en) |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070174429A1 (en) * | 2006-01-24 | 2007-07-26 | Citrix Systems, Inc. | Methods and servers for establishing a connection between a client system and a virtual machine hosting a requested computing environment |
| CN101668004B (en) * | 2008-09-04 | 2016-02-10 | 阿里巴巴集团控股有限公司 | A kind of webpage acquisition methods, Apparatus and system |
| CN105354451B (en) * | 2014-08-20 | 2020-10-16 | 腾讯科技（深圳）有限公司 | Access authentication method and system |
| US10069892B2 (en) * | 2014-09-27 | 2018-09-04 | Jianqing Wu | Versatile information management system and method thereof |
| CN107797834A (en) * | 2016-08-29 | 2018-03-13 | 阿里巴巴集团控股有限公司 | Page generation method, device and the equipment of logistical applications |
| CN107562467B (en) * | 2017-07-26 | 2020-08-25 | 阿里巴巴集团控股有限公司 | Page rendering method, device and equipment |
| CN107590281A (en) * | 2017-09-29 | 2018-01-16 | 惠州Tcl移动通信有限公司 | Control method, mobile terminal and the storage medium that a kind of webpage is switched fast |
-
2018
- 2018-08-01 CN CN201810861158.6A patent/CN110851750B/en active Active
Non-Patent Citations (1)
| Title |
|---|
| 西飞企业信息门户系统设计与实现;王珩宇;《中国优秀博硕士学位论文全文数据库(硕士) 信息科技辑》;20160315(第03期);第I138-2068页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110851750A (en) | 2020-02-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111241073B (en) | Data quality inspection method and device | |
| US20170019407A1 (en) | Method and server for providing image captcha | |
| CN109254765B (en) | Timing task management method, device, computer equipment and storage medium | |
| CN111124871A (en) | Interface test method and device | |
| CN111026080A (en) | Hardware-in-loop test method and device for controller | |
| CN106873958A (en) | The call method and device of a kind of API | |
| CN113296823A (en) | Method, device, equipment and storage medium for generating resource configuration file | |
| CN111353143A (en) | Sensitive authority detection method and device and storage medium | |
| CN110941632A (en) | Database auditing method, device and equipment | |
| CN109101819B (en) | Vulnerability detection method, terminal and storage medium | |
| Ferreira et al. | Behaviour-based Malware Detection in Mobile AndroidPlatforms Using Machine Learning Algorithms. | |
| CN111241547B (en) | Method, device and system for detecting override vulnerability | |
| CN110851750B (en) | Data acquisition method and device | |
| CN106611118B (en) | Method and device for applying login credentials | |
| CN114244826B (en) | Webpage identification information sharing method and device, storage medium and processor | |
| CN110968500A (en) | Test case execution method and device | |
| US20160241585A1 (en) | Analyze code that uses web framework using local parameter model | |
| CN109561121B (en) | Method and device for monitoring deployment | |
| CN114629807A (en) | Work order processing method, device and equipment | |
| CN113687973B (en) | Control method, equipment and medium for log dynamic output | |
| CN113672518B (en) | Test case construction method and related equipment | |
| CN112367414B (en) | External link processing method and device, storage medium and processor | |
| CN113515387B (en) | Data processing method and device and electronic device | |
| CN110955854A (en) | Thermodynamic diagram generation method and device | |
| CN113946831B (en) | Cross-platform new software based on micro-service, new system security risk analysis method, computer and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |