WO2017117912A1

WO2017117912A1 - Data acquisition method, apparatus and device, and computer storage medium

Info

Publication number: WO2017117912A1
Application number: PCT/CN2016/084343
Authority: WO
Inventors: 吴明丹; 王杨; 叶峻
Original assignee: 百度在线网络技术（北京）有限公司
Priority date: 2016-01-04
Filing date: 2016-06-01
Publication date: 2017-07-13
Also published as: CN105701175B; CN105701175A

Abstract

A data acquisition method, apparatus and device, and a computer storage medium. The method comprises: receiving an access log sent by a website, wherein the access log is generated by the website according to an operation performed by a user and executed with regard to a communication tool provided by the website (S101); obtaining candidate communication conversion data according to the access log (S102); and then acquiring real communication conversion data from the candidate communication conversion data (S103). Therefore, the method can solve the problem that an Internet operator cannot obtain real communication conversion data on a communication tool.

Description

Data acquisition method, device, device and computer storage medium

The present application claims the priority of the Chinese patent application whose application date is No.

Technical field

The present invention relates to the field of Internet application technologies, and in particular, to a data acquisition method, apparatus, device, and computer storage medium.

Background technique

At present, the Internet operator can provide a platform, which can include entries of a number of third-party websites, so that the user can directly obtain the entrance of the third-party website through the platform, and then can jump to the third-party website, or the user You can also use the communication tools provided by third-party websites on the platform to communicate online with the client services of third-party websites.

In order to track the online communication behavior of users, Internet operators need to obtain real communication conversion data of users. However, in the prior art, only the third-party website providing the communication tool can obtain the true communication conversion data, and therefore, the Internet operator cannot obtain the real communication conversion data on the communication tool.

Summary of the invention

In view of this, the embodiments of the present invention provide a data acquisition method, device, device, and computer storage medium, which are used to solve the problem that an Internet operator cannot obtain real communication conversion data on a communication tool.

An aspect of the embodiments of the present invention provides a data acquisition method, including:

Receiving an access log sent by the website, the access log being generated by the website according to an operation performed by a communication tool provided by the user for the website;

Obtaining candidate communication conversion data according to the access log;

Obtain real communication conversion data from the candidate communication conversion data.

The aspect as described above and any possible implementation manner further provide an implementation manner, the operations performed by the user for the communication tool provided by the website include: the communication tool provided by the user in the browser for the website Click operation

The access log includes a uniform resource locator URL that the user accesses the website and a page element that the user clicks.

The aspect as described above and any possible implementation manner further provide an implementation manner of obtaining real communication conversion data from the candidate communication conversion data, including:

Obtaining, according to the access log, an operation performed by a user on a communication tool provided by the website, and obtaining a hypertext transfer protocol HTTP request returned by the website after the simulation is completed;

Determining, according to the HTTP request, whether the candidate communication conversion data obtained according to the access log is real communication conversion data.

The aspect as described above, and any possible implementation manner, further provide an implementation, according to the HTTP request, determining whether the candidate communication conversion data obtained according to the access log is real communication conversion data, including:

Matching the HTTP request with a preset related request of the communication tool to obtain a matching result;

And determining, according to the matching result, whether the candidate communication conversion data obtained according to the access log is real communication conversion data.

The aspect as described above, and any possible implementation manner, further providing an implementation manner, determining, according to the matching result, candidate communication conversion data obtained according to the access log is Whether to convert data for real communication, including:

If the HTTP request does not match the related request of the communication tool, determining that the candidate communication conversion data obtained by the access log is not real communication conversion data; or

If the HTTP request matches the related request of the communication tool, determining that the candidate communication conversion data obtained by the access log is real communication conversion data.

An aspect of an embodiment of the present invention provides a data acquiring apparatus, including:

a receiving module, configured to receive an access log sent by a website, where the access log is generated by the website according to an operation performed by a communication tool provided by the user for the website;

a processing module, configured to obtain candidate communication conversion data according to the access log;

An obtaining module, configured to obtain real communication conversion data from the candidate communication conversion data.

The above-mentioned aspect and any possible implementation manner further provide an implementation manner, where the acquiring module is specifically configured to:

An aspect of the above, and any possible implementation, further providing an implementation manner, The obtaining module is configured to determine, according to the HTTP request, whether the candidate communication conversion data obtained according to the access log is real communication conversion data, specifically used for:

The foregoing aspect and any possible implementation manner further provide an implementation manner, where the obtaining module is configured to determine, according to the matching result, whether the candidate communication conversion data obtained according to the access log is real communication conversion data. When specifically used to:

It can be seen from the above technical solutions that the embodiments of the present invention have the following beneficial effects:

According to the technical solution provided by the embodiment of the present invention, the real communication conversion data on the communication tool can be obtained according to the access log sent by the website providing the communication tool, thereby solving the problem that the Internet operator cannot obtain the real communication conversion data on the communication tool. Efficiently and simply realizes the acquisition of real communication conversion data, and then can make resource delivery decisions based on real communication conversion data. In addition, when obtaining real communication conversion data, the candidate real communication conversion data can be filtered, the real communication conversion data is determined, and the accuracy of the real communication conversion data is improved.

DRAWINGS

1 is a schematic flowchart of a data acquisition method according to an embodiment of the present invention;

FIG. 2 is a functional block diagram of a data acquiring apparatus according to an embodiment of the present invention.

detailed description

The present invention will be described in detail below with reference to the drawings and specific embodiments.

It should be understood that the described embodiments are only a part of the embodiments of the invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

The terms used in the embodiments of the present invention are for the purpose of describing particular embodiments only and are not intended to limit the invention. The singular forms "a", "the" and "the"

It should be understood that the term "and/or" as used herein is merely an association describing the associated object, indicating that there may be three relationships, for example, A and/or B, which may indicate that A exists separately, while A and B, there are three cases of B alone. In addition, the character "/" in this article generally indicates that the contextual object is an "or" relationship.

Depending on the context, the word "if" as used herein may be interpreted as "when" or "when" or "in response to determining" or "in response to detecting." Similarly, depending on the context, the phrase "if determined" or "if detected (conditions or events stated)" may be interpreted as "when determined" or "in response to determination" or "when detected (stated condition or event) "Time" or "in response to a test (condition or event stated)".

An embodiment of the present invention provides a data acquisition method. Referring to FIG. 1 , it is a schematic flowchart of a data acquisition method according to an embodiment of the present invention. As shown in the figure, the method includes the following steps:

S101. Receive an access log sent by a website, where the access log is generated by the website according to an operation performed by a communication tool provided by the user for the website.

Specifically, the technical solution of the embodiment of the present invention can be applied to a system including a website, a site, and a server where the website is located. The website belongs to a third-party website for the site and the server, and the communication tool provided by the website has a corresponding entrance on the site. The user of the website can perform operations on the communication tool on the site, and the user can implement the problem consultation by performing the operation on the communication tool on the site to obtain the required information.

For example, the communication tool may include a communication tool in the form of a page on a browser such as Business Communication, 53 Customer Service, 51talk, QQ, or Music.

In a specific implementation process, a monitoring module may be pre-installed in the website, and the monitoring module may automatically monitor the user of the website to perform operations on the communication tool on the website.

In the embodiment of the present invention, the monitoring module can be implemented by using a script (Java Script, JS) code. For example, the JS code can be code for counting website data.

In a specific implementation process, if the user performs an operation on the communication tool on the site, the monitoring module may generate an access log according to the operation performed by the communication tool provided by the user for the website. For example, the user clicks the button of the open communication tool provided by the website in the browser, so that the monitoring module can monitor the user to perform a click operation on the communication tool provided by the website in the browser, and then locate the unified resource according to the user visiting the website. The Uniform Resource Locator (URL) and the page elements clicked by the user generate an access log.

Further, after generating an access log, the monitoring module may send the access log to the server where the site is located, so that the server can receive the access date sent by the website. Zhi. Alternatively, the monitoring module may also store the generated access log locally, and then send all access logs to the server for a period of time at intervals. Each access log may include a URL of the user visiting the website and a page element clicked by the user.

For example, the page element that the user clicks can be a button for opening a communication tool provided by the website that the user clicks in the browser. Correspondingly, the operation performed by the user for the communication tool provided by the website may be: a click operation of the communication tool provided by the user in the browser for the website.

S102. Obtain candidate communication conversion data according to the access log.

Specifically, after receiving the access log sent by the website, the server stores the access log, and uses the access log as a candidate communication conversion data. In this way, the server can obtain an access log from the monitoring module of each third-party website, thereby obtaining a large number of access logs.

S103. Obtain real communication conversion data from the candidate communication conversion data.

Specifically, in the embodiment of the present invention, the monitoring module can monitor and capture the URL of the user visiting the website and the page element clicked by the user, thereby generating candidate communication conversion data, but these are based on the captured information. The generated candidate communication conversion data is not the official data provided by the website where the communication tool is located, and since there is a button corresponding to a certain URL, the button is used to open the communication tool after being clicked, but due to the setting of the URL and the button It is controlled by a third-party website. The URL and button may become other pages instead of the page that provides the communication tool entry. The button may also become a click and open other tools, so the candidate communication conversion data may not necessarily be It is a real communication conversion data. Only when the page corresponding to the URL of the user visiting the website is a page providing a communication tool entry, and the page element clicked by the user is a button for opening the communication tool after being clicked, the generated candidate communication is generated. Conversion data is the actual communication conversion data. Therefore, in the embodiment of the present invention, After obtaining the candidate communication conversion data, it is required to further obtain real communication conversion data from the candidate communication conversion data.

In a specific implementation process, the server may periodically obtain real communication conversion data from the candidate communication conversion data according to a preset period.

For example, in the embodiment of the present invention, the method for the server to obtain the real communication conversion data from the candidate communication conversion data may include:

First, the server simulates an operation performed by a user on a communication tool provided by the website according to the access log, and obtains a Hyper Text Transfer Protocol (HTTP) returned by the website after the simulation is completed. request. Then, the server determines, according to the HTTP request, whether the candidate communication conversion data obtained according to the access log is real communication conversion data.

In a specific implementation process, since the access log includes the URL of the user visiting the website and the page element clicked by the user, the server may use the crawler tool to open the URL of the user visiting the website, and then use the simulation tool to correspond to the opened URL. The page simulates user behavior and clicks on the page elements contained in the access log to simulate what the user is doing with the communication tools provided by the site.

It can be understood that after simulating the operation performed by the user for the communication tool provided by the website, the HTTP request returned by the website can be monitored, so that the HTTP request returned by the website can be obtained.

For example, the method for the server to determine, according to the HTTP request, whether the candidate communication conversion data obtained according to the access log is real communication conversion data may include, but is not limited to:

First, the server associates the HTTP request with a preset communication tool Request a match to get a match. Then, the server determines, according to the matching result, whether the candidate communication conversion data obtained according to the access log is real communication conversion data.

Further, if the matching result is that the HTTP request does not match the related request of the communication tool, the server determines that the candidate communication conversion data obtained by the access log is not real communication conversion data. Alternatively, if the matching result is that the HTTP request matches the related request of the communication tool, the server determines that the candidate communication conversion data obtained by the access log is real communication conversion data.

In a specific implementation process, the server may pre-set the relevant request of the communication tool, and then, after obtaining the HTTP request returned by the website providing the communication tool, perform the HTTP request and the related request of the preset communication tool. Matching, such as performing format matching of the HTTP request. If the format of the HTTP request matches the format of the related request of the preset communication tool, it is considered that the URL opened before the HTTP request is obtained and the page element of the click is the user's real access. Providing a page of communication tools and opening a communication tool, thereby determining that the candidate communication conversion data on which the simulation operation is based is real communication conversion data, thereby realizing real communication conversion data from the candidate communication conversion data.

On the other hand, if the format of the HTTP request does not match the format of the related request of the preset communication tool, it is considered that the URL opened before the HTTP request is obtained and the page element of the click is not the user accessing the page providing the communication tool, and is opened. The communication tool can determine that the candidate communication conversion data on which the simulation operation is based is not the real communication conversion data, thereby realizing the acquisition of the non-real communication conversion data from the candidate communication conversion data, and matching operations of each candidate communication conversion data. Real communication conversion data and non-real communication conversion data can be obtained from the candidate communication conversion data.

Embodiments of the present invention further provide an apparatus embodiment for implementing the steps and methods in the foregoing method embodiments.

Please refer to FIG. 2 , which is a functional block diagram of a data acquiring apparatus according to an embodiment of the present invention. As shown, the device includes:

The receiving module 21 is configured to receive an access log sent by the website, where the access log is generated by the website according to an operation performed by a communication tool provided by the user for the website;

The processing module 22 is configured to obtain candidate communication conversion data according to the access log.

The obtaining module 23 is configured to obtain real communication conversion data from the candidate communication conversion data.

In a specific implementation process, the operations performed by the user for the communication tool provided by the website include: a click operation of the communication tool provided by the user in the browser for the website.

In a specific implementation process, the access log includes: a uniform resource locator URL that the user accesses the website and a page element that the user clicks.

In a specific implementation process, the obtaining module 23 is specifically configured to:

In a specific implementation process, the obtaining module 23 is configured to determine, according to the HTTP request, whether the candidate communication conversion data obtained according to the access log is real communication conversion data, specifically used for:

In a specific implementation process, the obtaining module 23 is configured to determine, according to the matching result, whether the candidate communication conversion data obtained according to the access log is real communication conversion data, specifically used for:

Since the units in this embodiment can perform the method shown in FIG. 1, and the parts not described in detail in this embodiment, reference may be made to the related description of FIG.

The technical solution of the embodiment of the invention has the following beneficial effects:

In the embodiment of the present invention, an access log sent by a website is received by a server, where the access log is generated by the website according to an operation performed by a communication tool provided by the user for the website; thus, the server according to the access log Obtaining candidate communication conversion data; and further, the server obtains real communication conversion data from the candidate communication conversion data.

According to the technical solution provided by the embodiment of the present invention, the real communication conversion data on the communication tool can be obtained according to the access log sent by the website providing the communication tool, thereby solving the problem that the Internet operator cannot obtain the real communication conversion data on the communication tool. Efficiently and simply realizes the acquisition of real communication conversion data, and then can make resource delivery decisions based on real communication conversion data.

In addition, when obtaining real communication conversion data, the candidate real communication conversion data can be filtered, the real communication conversion data is determined, and the accuracy of the real communication conversion data is improved.

A person skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.

In the several embodiments provided by the present invention, it should be understood that the disclosed system, apparatus, and method may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner. For example, multiple units or components may be combined. Or it can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

The above integrated unit implemented in the form of a software functional unit can be stored in one meter The computer can be read in the storage medium. The above software functional unit is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to perform the methods of the various embodiments of the present invention. Part of the steps. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program codes. .

The above are only the preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalents, improvements, etc., which are made within the spirit and principles of the present invention, should be included in the present invention. Within the scope of protection.

Claims

A data acquisition method, the method comprising:

Receiving an access log sent by the website, the access log being generated by the website according to an operation performed by a communication tool provided by the user for the website;

Obtaining candidate communication conversion data according to the access log;

Obtain real communication conversion data from the candidate communication conversion data.
The method of claim 1 wherein

The operations performed by the user for the communication tool provided by the website include: a click operation of the communication tool provided by the user in the browser for the website;

The access log includes a uniform resource locator URL that the user accesses the website and a page element that the user clicks.
The method according to claim 1, wherein the real communication conversion data is obtained from the candidate communication conversion data, including:

Obtaining, according to the access log, an operation performed by a user on a communication tool provided by the website, and obtaining a hypertext transfer protocol HTTP request returned by the website after the simulation is completed;

Determining, according to the HTTP request, whether the candidate communication conversion data obtained according to the access log is real communication conversion data.
The method according to claim 3, wherein determining whether the candidate communication conversion data obtained according to the access log is real communication conversion data according to the HTTP request comprises:

Matching the HTTP request with a preset related request of the communication tool to obtain a matching result;

And determining, according to the matching result, whether the candidate communication conversion data obtained according to the access log is real communication conversion data.
The method according to claim 4, wherein determining whether the candidate communication conversion data obtained according to the access log is real communication conversion data according to the matching result comprises:

If the HTTP request does not match the related request of the communication tool, determining that the candidate communication conversion data obtained by the access log is not real communication conversion data; or

If the HTTP request matches the related request of the communication tool, determining that the candidate communication conversion data obtained by the access log is real communication conversion data.
A data acquisition device, characterized in that the device comprises:

a receiving module, configured to receive an access log sent by a website, where the access log is generated by the website according to an operation performed by a communication tool provided by the user for the website;

a processing module, configured to obtain candidate communication conversion data according to the access log;

An obtaining module, configured to obtain real communication conversion data from the candidate communication conversion data.
The device of claim 6 wherein:

The operations performed by the user for the communication tool provided by the website include: a click operation of the communication tool provided by the user in the browser for the website;

The access log includes a uniform resource locator URL that the user accesses the website and a page element that the user clicks.
The device according to claim 6, wherein the obtaining module is specifically configured to:

Obtaining, according to the access log, an operation performed by a user on a communication tool provided by the website, and obtaining a hypertext transfer protocol HTTP returned by the website after the simulation is completed. request;

Determining, according to the HTTP request, whether the candidate communication conversion data obtained according to the access log is real communication conversion data.
The device according to claim 8, wherein the obtaining module is configured to determine, according to the HTTP request, whether the candidate communication conversion data obtained according to the access log is real communication conversion data, specifically used for:

Matching the HTTP request with a preset related request of the communication tool to obtain a matching result;

And determining, according to the matching result, whether the candidate communication conversion data obtained according to the access log is real communication conversion data.
The device according to claim 9, wherein the obtaining module is configured to determine, according to the matching result, whether the candidate communication conversion data obtained according to the access log is real communication conversion data, specifically used for:

If the HTTP request does not match the related request of the communication tool, determining that the candidate communication conversion data obtained by the access log is not real communication conversion data; or

If the HTTP request matches the related request of the communication tool, determining that the candidate communication conversion data obtained by the access log is real communication conversion data.
a device, including

One or more processors;

Memory

One or more programs, the one or more programs being stored in the memory, when executed by the one or more processors:

Receiving an access log sent by the website, the access log is for the website according to the user Generated by the operations performed by the communication tools provided by the website;

Obtaining candidate communication conversion data according to the access log;

Obtain real communication conversion data from the candidate communication conversion data.
A computer storage medium encoded with a computer program, when executed by one or more computers, causes the one or more computers to perform the following operations:

Receiving an access log sent by the website, the access log being generated by the website according to an operation performed by a communication tool provided by the user for the website;

Obtaining candidate communication conversion data according to the access log;

Obtain real communication conversion data from the candidate communication conversion data.