CN113689232A - Method and device for detecting crowd recall service and electronic equipment - Google Patents

Method and device for detecting crowd recall service and electronic equipment Download PDF

Info

Publication number
CN113689232A
CN113689232A CN202110851532.6A CN202110851532A CN113689232A CN 113689232 A CN113689232 A CN 113689232A CN 202110851532 A CN202110851532 A CN 202110851532A CN 113689232 A CN113689232 A CN 113689232A
Authority
CN
China
Prior art keywords
crowd
service
recall
type
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110851532.6A
Other languages
Chinese (zh)
Other versions
CN113689232B (en
Inventor
郝佳
侯广宇
王正
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Huabei Technology Co ltd
Original Assignee
Alibaba China Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba China Co Ltd filed Critical Alibaba China Co Ltd
Priority to CN202110851532.6A priority Critical patent/CN113689232B/en
Publication of CN113689232A publication Critical patent/CN113689232A/en
Application granted granted Critical
Publication of CN113689232B publication Critical patent/CN113689232B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • G06Q30/0251—Targeted advertisements
    • G06Q30/0255—Targeted advertisements based on user history
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/953—Querying, e.g. by the use of web search engines
    • G06F16/9535—Search customisation based on user profiles and personalisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • G06Q30/0251—Targeted advertisements
    • G06Q30/0253—During e-commerce, i.e. online transactions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • G06Q30/0251—Targeted advertisements
    • G06Q30/0255—Targeted advertisements based on user history
    • G06Q30/0256—User search
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02W—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO WASTEWATER TREATMENT OR WASTE MANAGEMENT
    • Y02W90/00—Enabling technologies or technologies with a potential or indirect contribution to greenhouse gas [GHG] emissions mitigation

Abstract

The embodiment of the application discloses a method and a device for detecting people group recall service and electronic equipment, wherein the method comprises the following steps: acquiring a first mapping relation, and determining at least one recallable crowd type according to the first mapping relation; filtering the at least one recallable crowd type from a full set of crowd types and determining at least one target crowd type to be detected; acquiring a second mapping relation, and determining a plurality of target users for the target crowd type according to the second mapping relation; and simulating and constructing a service calling request by utilizing the identifications of the target users, calling the service provided by the crowd recall service system, and detecting whether the crowd recall service system can recall the target crowd type. Through the embodiment of the application, the influence of the real-time verification process on the actual online flow can be reduced while the real-time verification is effectively carried out on the crowd recall service system.

Description

Method and device for detecting crowd recall service and electronic equipment
Technical Field
The present application relates to the field of people recall service technologies, and in particular, to a method and an apparatus for detecting people recall service, and an electronic device.
Background
The crowd recall service system is a system for providing crowd recall service, a requester can initiate a crowd recall request to the crowd recall service system by taking a user ID and the like in an application system as parameters, and correspondingly, the crowd recall service system can return crowd information to which the user belongs, such as whether the user belongs to a crowd with high purchasing power, whether the user belongs to a crowd who visits a certain shop in the last three days, and the like. The requesting party can use the crowd information corresponding to the specific user to perform advertisement putting and the like.
In the process of providing the crowd recall service, the quality assurance problem is also important, and the correctness of the external service provided by the crowd recall service system needs to be sensed in real time. Here, the correctness means that it is necessary to ensure that all people can be correctly recalled, that is, for any user ID, information on the people to which the user belongs can be returned. Thereby improving the trust of the advertisement owner and the like to the group recall service system.
In order to achieve the purpose of sensing the correctness of the service provided by the crowd recall service system externally, a detection system for providing real-time detection service for the crowd recall service system can be provided. Since it is necessary to verify whether all people can be recalled correctly in the process of providing services to the outside in real time, one way may be: the method comprises the steps of obtaining IDs of all users, constructing a request by utilizing the user IDs, simulating an input engine to send the request to a crowd recall service system, and if the crowd recall service system can return information of the crowd to which the user ID belongs aiming at each user ID, proving that the correctness of the crowd recall service system passes the verification.
The above method can theoretically verify the correctness of the crowd recall service system, but in practical application, because the magnitude of the total number of users is usually very large, for example, in a certain commodity object information system, the total number of users may be in the magnitude of billions, if request simulation is respectively performed on all the user IDs and the user IDs are respectively sent to the crowd recall service system for processing, the processing task of the crowd recall service system is very heavy; moreover, the detection process is performed simultaneously in the process of normally providing the service online by the crowd recall service system, and the simulation request of billions of orders of magnitude can bring fluctuation to the online flow, so that the correctness of the crowd recall service system is difficult to be checked in real time by using the scheme in practical application.
Therefore, how to effectively perform real-time verification on the crowd recall service system and reduce the influence of the real-time verification process on the actual online traffic becomes a technical problem to be solved by the technical personnel in the field.
Disclosure of Invention
The application provides a method and a device for detecting a people group recall service and electronic equipment, which can effectively check a people group recall service system in real time and reduce the influence of a real-time checking process on the actual on-line flow.
The application provides the following scheme:
a method of detecting a crowd recall service, comprising:
acquiring a first mapping relation between a plurality of first user identifications and a first crowd type according to real-time log record information generated by a crowd recall service system in the service providing process, and determining at least one recallable crowd type according to the first mapping relation;
filtering the at least one recallable crowd type from a full-volume crowd type set, and determining at least one target crowd type to be detected according to a filtered result;
acquiring a second mapping relation between a plurality of second user identifications and a second crowd type according to historical log record information generated by the crowd recall service system in the service providing process, and determining a plurality of target users for the target crowd type according to the second mapping relation;
and simulating and constructing a service calling request by utilizing the identifications of the target users, calling the service provided by the crowd recall service system, and detecting whether the crowd recall service system can recall the target crowd type.
An apparatus that detects a crowd recall service, comprising:
the recalling crowd type determining unit is used for acquiring a first mapping relation between a plurality of first user identifications and a first crowd type according to real-time log record information generated by the crowd recall service system in the service providing process, and determining at least one recalling crowd type according to the first mapping relation;
the filtering unit is used for filtering the at least one recallable crowd type from a full crowd type set and determining at least one target crowd type to be detected according to a filtered result;
the target user determining unit is used for acquiring a second mapping relation between a plurality of second user identifications and a second crowd type according to historical log record information generated by the crowd recall service system in the service providing process, and determining a plurality of target users for the target crowd type according to the second mapping relation;
and the service calling unit is used for simulating and constructing a service calling request by utilizing the identifiers of the target users, calling the service provided by the crowd recall service system and detecting whether the crowd recall service system can recall the target crowd type.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of any of the preceding claims.
An electronic device, comprising:
one or more processors; and
a memory associated with the one or more processors for storing program instructions that, when read and executed by the one or more processors, perform the steps of the method of any of the preceding claims.
According to the specific embodiments provided herein, the present application discloses the following technical effects:
through the embodiment of the application, at least one recallable crowd type can be determined according to the real-time log recording information generated by the crowd recall service system in the service providing process, namely, the part of crowd types can be verified through the real-time log recording information, so that the part of crowd types can be filtered from the full crowd type set, and the recallable crowd types are required to be further detected only if the recallable crowd types cannot be determined from the current real-time log recording information. Thus, the number of target population types that need to be detected is reduced. In addition, since specific target users corresponding to the target crowd types can be obtained from the historical log record information, only the ID of the target users is needed to be used for simulating and constructing service requests and initiating requests to the crowd recall service, and then whether the target crowd types can be recalled or not can be determined according to response information returned by the crowd recall service. By the method, only part of target crowd types need to be detected by simulating and constructing the service call request, and target users having historical mapping relation with the target crowd types can be selected in a directional mode to construct the service call request, so that the number of the service call requests required to be constructed and sent can be greatly reduced, and the influence of a specific real-time detection process on the flow on the crowd recall service actual line is reduced.
Of course, it is not necessary for any product to achieve all of the above-described advantages at the same time for the practice of the present application.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a schematic diagram of a system architecture provided by an embodiment of the present application;
FIG. 2 is a flow chart of a method provided by an embodiment of the present application;
FIG. 3 is a schematic process flow diagram provided by an embodiment of the present application;
FIG. 4 is a schematic view of an apparatus provided by an embodiment of the present application;
fig. 5 is a schematic diagram of an electronic device provided in an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments that can be derived from the embodiments given herein by a person of ordinary skill in the art are intended to be within the scope of the present disclosure.
First, in order to facilitate understanding of the technical solutions provided in the embodiments of the present application, first, an application scenario in which information such as "advertisement" is placed in commodity object information is taken as an example, and a crowd recall service system and a related crowd recall flow are briefly introduced below.
Referring to fig. 1, the commodity object information system 11 mainly includes a seller user (or a merchant user, etc.) and a buyer user, the seller user can open an online virtual store through the commodity object information system to issue specific commodity object information, and the buyer user can browse the specific commodity object information and perform various operations such as collecting, entering a "shopping cart" and placing an order for purchase.
In practical applications, the commodity object information system may also have an associated information delivery system 12 for delivering information such as "advertisement" to users in the commodity object information system 11. That is, the information specifically presented to the buyer user may include some "advertising" content in addition to the information about the merchandise objects published by the specific seller user. In order to achieve the desired delivery effect, the targeted delivery of the divided groups can be performed. For example, an "ad" may need to be delivered to a population with a higher purchasing power, to achieve a higher browse-purchase conversion, and so on.
In order to achieve the goal of delivering the crowd oriented information, the specific crowd recall service system 13 may be used to provide the service. Specifically, the crowd recall service system 13 may include a crowd construction system 131, and the crowd construction system 14 may add crowd labels to specific users through crowd construction, policy analysis, and the like. For example, according to the data of user's data, browsing, collecting, buying, ordering, etc. in the commodity object information system, specific crowd labels are added to the user, including high purchasing power, treasure mother, etc., for example.
Information delivery system 12 may provide a delivery task configuration system 121 for a delivery task publisher, such as an "advertiser," who may create a specific delivery task through the delivery task configuration system 121. In creating the delivery task, a variety of selectable crowd labels may be provided according to the crowd construction situation of the crowd-specific building system 131. The advertiser can select the needed crowd labels according to actual requirements. Thus, a specific delivery task may include specific content to be delivered and crowd label information, that is, the corresponding content to be delivered needs to be delivered to the user having the corresponding crowd label. If there are multiple crowd labels selected by the "advertiser," a logical relationship between the crowd labels may be set, for example, the crowd labels may be delivered to users associated with multiple crowd labels simultaneously, or users having any one of the crowd labels may be delivered. In addition, an "advertiser" may also select a particular delivery channel (also referred to as an "ad spot") when submitting a delivery task. For example, a particular "ad slot" may include a live channel in merchandise object information system 11, a particular search keyword, and so on. Thus, for the former, when a user accesses a live channel, whether to put an advertisement to the live channel can be judged according to the population to which the specific user belongs; for the latter, when a user initiates a search with a certain search keyword, whether to put an advertisement to the user may be determined according to the population to which the specific user belongs, and so on.
Specifically, when "advertisement" is delivered, it is determined whether or not "advertisement" is delivered to the user and what kind of advertisement is delivered, based on the fact that the user actually accesses the product object information system 11. To this end, the information delivery system 12 may further provide a delivery engine 122, wherein the delivery engine 122 may be provided in a plurality, for example, in a merchandise object information system, different "ad spots" may correspond to different product lines (e.g., including the aforementioned live broadcasts, searches, etc.), and at this time, the corresponding delivery engine 122 may be provided for the different product lines respectively. Such a placement engine 122 may interface with a particular merchandise object information system 11. When a user accesses a product line in the commodity object information system, the commodity object information system may provide information such as the ID of the current visitor user to the corresponding delivery engine 122. The crowd recall service system 13 may also provide a crowd recall service engine 132. After receiving the user ID provided by the commodity object information system, the delivery engine 122 may issue a request to the crowd recall service engine 132 using information such as the user ID and a corresponding channel identifier as parameters. The crowd recall service engine 132 may determine and return crowd type information to which the user belongs. The placement engine 122 can then determine whether or not to place an "advertisement" to a particular user based on the demographic type information to which the user belongs, and if so, what "advertisement" to place.
For example, when a user searches for a keyword in a commodity object information system, the commodity object information system may provide information such as the user's ID to the delivery engine 122; the delivery engine 122 initiates a call request to the crowd recall service engine 132 by using the information of the user's ID, channel, etc. as parameters; after the crowd recall service engine 132 returns the crowd type information corresponding to the user, the delivery engine 122 performs "advertisement" delivery for the user according to the crowd type information to which the user specifically belongs. For example, in the process of presenting search results through multiple resource slots in a returned search results page, the "advertisement" content may be presented at a certain resource slot, and so on.
It can be seen that, in a specific crowd recall flow, firstly, a user accesses the commodity object information system 11, in this process, the commodity object information system 11 provides the user ID information to the delivery engine 122, the delivery engine 122 initiates a request to the crowd recall service engine 132 according to the user ID, the crowd recall service engine 132 can determine the crowd type information to which the user belongs and return the crowd type information to the delivery engine 122, and the delivery engine 122 determines the "advertisement" delivery policy for the user according to the crowd type information to which the user belongs.
When providing the crowd recall service, the crowd recall service system 13 may include information obtained in the process of constructing historical crowds and information constructed in real time, and when specifically implementing, may combine the two pieces of information and provide the crowd information to which the user belongs. For example, a user belongs to a high purchasing power crowd, and the crowd label can be determined according to information such as historical purchasing records of the user; also, the user has visited a store on the last three days, so the tag can be analyzed in real time after receiving a specific crowd recall service request, and so on.
In summary, the crowd recall service system 13 may be constructed with a large number of types of crowd, and whether all of the crowd can be recalled correctly in the process of providing external services is an important standard for checking the service quality of the crowd recall service system 13. That is, it is not known in advance when a user accesses the commodity object information system, and therefore, it is required to ensure that when a crowd recall request for any user is received at any time, the user can accurately return to the crowd to which the user belongs, and reliable service can be provided for the advertiser in deed, and trust of the advertiser and the like on the crowd recall service system 13 is obtained. Because the visitor user in the specific merchandise object information system has the characteristic of randomness, the detection system 14 needs to be provided, and the detection system 14 can provide real-time online detection service for the crowd recall service system 13 to determine the accuracy and reliability of the external service provided by the crowd recall service system 13.
However, in the prior art, since the specific detection system 14 does not know the mapping relationship between the specific user ID and the group of people to which the specific detection system 14 belongs, in order to check whether the crowd recall service system 13 can correctly recall all the people, as described in the background section, only the IDs of all the users can be acquired, and then a request is constructed by using the user IDs, and the delivery engine 122 is simulated to send the request to the crowd recall service engine 132. However, since the above detection process needs to be performed during the normal on-line out-of-service process of the crowd recall service engine 132, if a large number of simulation requests are requested, the normal flow of the crowd recall service system 13 is severely affected.
In view of the above situation, in the embodiment of the present application, the real-time log record information generated by the crowd recall service system 13 in the process of providing the service to the outside may be obtained, and the request parameters associated with the multiple service invocation requests received by the crowd recall service system 13 in the target detection period (for example, in the last N minutes, and the like) and the response parameters associated with the corresponding response messages may be determined according to the real-time log record information. The request parameters include user IDs such as user IDs, and the corresponding response parameters include crowd type information matched for the user IDs by the crowd recall service system 13. That is, from such real-time log record information, the mapping relationship between the user ID and the crowd type can be acquired. Of course, since not all users will access the application systems such as the commodity object information system 11 in the last N minutes, the mapping relationship determined in the real-time log record information only includes the user IDs of some users and some types of people. On the other hand, since the real-time log record information in the last N minutes can substantially reflect the real-time service condition of the crowd recall service system 13, the crowd type in the mapping relationship can be determined as the first crowd type (which may be one or more, and usually is a plurality of) that can be recalled. That is, since the mapping relationship between the part of the user IDs and the crowd types can be obtained from the real-time log of the last N minutes, it can be proved that the part of the crowd types can be recalled normally at least in the last N minutes, and therefore, in the embodiment of the present application, the part of the crowd types can be determined as recallable crowd types. In addition, since the map relationship is determined from the recently acquired real-time log record information, the type of the crowd to be recalled can be considered to be correct as long as the format and the like are in expectation.
Since the at least one recallable crowd type is verified as recallable and correct, the recallable crowd types can be filtered from the full set of crowd types (the information can be obtained in advance), and the at least one target crowd type to be detected is determined according to the filtered result. That is, only the types of people who are not recalled in the real-time log record information may need to be further checked whether the people recall service system can recall the target types of people by constructing a service call request. In specific implementation, invalid, expired and target list hit crowd types can be filtered from the full crowd type set, so that the number of crowd types needing to be detected is further reduced.
For the target crowd type to be detected, the identities of the target users having a mapping relationship with the target crowd type may be obtained according to historical log record information (for example, the past 48 hours, and the like) generated by the crowd recall service system 13 in the process of providing the service to the outside. That is, a user may not access a commodity object information system 11 within the past N minutes, but may access the commodity object information system 11 within the past 48 hours at a certain time, and the relevant delivery engine 122 calls a crowd recall service with the ID of the user as a parameter, at this time, corresponding request and response records also exist in the history log record information, and a mapping relationship between the user ID and the crowd tag can be determined from the history log record information.
That is to say, for part of target crowd types that need to be detected, it can be determined from the historical log records which specific target users have a mapping relationship with the target crowd types, so that a service invocation request can be constructed by using the identifiers of the target users and sent to the crowd recall service system 13, and according to the response result of the crowd recall service system 13, it can be detected whether the crowd recall service system 13 can recall the target crowd types.
In this way, since at least one recallable crowd type can be determined according to the real-time log record information and filtered from the full crowd type set, only at least one target crowd type that can not be recalled can be further detected if the current real-time log record information is not determined, and therefore, the crowd types that need to be detected are reduced, and the magnitude of service requests that need to be constructed is reduced. In addition, since specific target users corresponding to the target crowd types can be obtained from the historical log record information, only the ID of the target users is needed to be used for simulating and constructing service requests and initiating requests to the crowd recall service, and then whether the target crowd types can be recalled or not can be determined according to response information returned by the crowd recall service.
The following describes in detail the implementation provided by the embodiments of the present application.
Example one
First, this embodiment provides a method for verifying a people recall service from the perspective of the detection system 14 described in fig. 1, and with reference to fig. 2, the method may include:
s201: according to real-time log record information generated in the process of providing services by the crowd recall service system, a first mapping relation between a plurality of first user identifications and a first crowd type is obtained, and at least one recallable crowd type is determined according to the first mapping relation.
In the crowd recall service system, a log center system is usually associated, the log center can be a log management system integrating log collection, transmission and storage, and log data can be conveniently stored in real time through the log center. In the embodiment of the application, a part of recallable crowd types can be determined by recording information in real time. Specifically, a relatively short detection period may be set, for example, if the detection period may be one minute, the specific real-time log may specifically refer to log record information acquired in the last minute. The logged information collected during the last minute may then be used to determine a portion of the retrievable crowd type. And when the next detection period comes, continuously taking the log record information collected in the last minute as the real-time log record information, and the like.
The specific real-time log recording information may include the following contents: request parameters associated with a plurality of service call requests received in a current detection period, and response parameters associated with corresponding response messages. The specific request parameter may include an identifier of the first user, and the response parameter includes information of the first crowd type, so that the first mapping relationship between the plurality of first user identifiers and the first crowd type may be obtained according to the identifier of the first user included in the specific request parameter and the first crowd type information included in the corresponding response parameter. The first mapping relationship belongs to a real-time mapping relationship. For this portion of the first population type, this portion of the population type may prove recallable as being recalled in real time. In addition, as long as the format and the like are in accordance with expectations, the correctness of the recall result can be proved, and no additional detection is needed. Therefore, the mode of constructing the recall request in a simulated mode is not needed any more, and whether the part of the crowd types can be recalled or not is detected.
S202: and filtering the at least one recallable crowd type from the full-volume crowd type set, and determining at least one target crowd type to be detected according to the filtered result.
After the recallable crowd type is determined according to the real-time log record information, since the recallable crowd type can be recalled when the part of the crowd type is verified, whether the recallable crowd type can be recalled or not does not need to be checked in addition by sending a calling request to the crowd recall service. Thus, this portion of the population type may be filtered out of the full set of population types. The remaining crowd types are not verified in the current real-time log record information and therefore can be verified by sending a call request to the crowd recall service.
The total crowd type can be obtained by querying the crowd construction system and the like, and in the specific implementation, the crowd construction system can push real-time total crowd type information to the detection system, or the detection system can pull the information from the crowd construction system, and the like.
In practical applications, the total number of types of people is usually large, for example, in a commodity object information system, the total number of types of people may include millions of types of people. In the embodiment of the application, by filtering the crowd types which are already verified in the real-time log record information, the number of the remaining target crowd types which need to be detected is greatly reduced.
In addition, in practical applications, the total population type set may include some population types that are out of date, invalid, and/or hit some target lists (e.g., black list, switched traffic gray list, on-list, etc.), and the population types may also be filtered out, thereby further reducing the number of target population types that need to be detected.
That is, as shown in fig. 3, in a specific implementation, a funnel-type crowd type filtering scheme may be further provided, and by filtering the whole amount of crowd types layer by layer, the number of target crowd types that are finally left to be detected will be greatly reduced.
S203: and acquiring a second mapping relation between a plurality of second user identifications and a second crowd type according to historical log record information generated by the crowd recall service system in the service providing process, and determining a plurality of target users for the target crowd type according to the second mapping relation.
After the target crowd type needing to be detected is determined, a second mapping relation between a plurality of second user identifications and a second crowd type can be obtained from historical log record information. This second mapping belongs to a historical mapping. Wherein the historical logging information may be logging information over a longer period of time, such as the last 48 hours of logging information, and so forth. The method comprises a plurality of request parameters related to historical service invoking requests and response parameters related to corresponding historical response messages, and further can determine the mapping relation between the second user identification and the second crowd type according to the second user identification included in the request parameters and the second crowd type information included in the corresponding response parameters. Since such historical logging information may relate to a longer time span, which may include more user identifications, and more types of people, which may include target people types to be detected, it may be known which users have a mapping relationship with these target people types. These users may then be identified as target users for the simulation of the construction of the invocation service request.
That is, since the mapping relationship between the user and the crowd type is known, when some crowd types need to be detected, the simulation of invoking the service request can be performed by directly using the user identifier having the mapping relationship with the crowd type. In specific implementation, all users in the history log record information having a mapping relationship with the target crowd type may be determined as target users, or a part of users may be selected from multiple users corresponding to the same target crowd type as target users, so as to further reduce the number of call requests to be sent. For example, actual tests show that the number of target people types to be detected can be reduced to one thousandth of the number of the total people types or even lower after the filtering is performed in the various manners. That is, the number of millions of people to be detected can be reduced to thousands of orders.
S204: and simulating and constructing a service calling request by utilizing the identifications of the target users, calling the service provided by the crowd recall service system, and detecting whether the crowd recall service system can recall the target crowd type.
After the specific target user is determined, a service calling request can be constructed in a simulated mode by using user identifications such as an ID (identity) of the target user, and then the simulated service calling request can be sent to a people group recall service system for calling the people group recall service. And detecting whether the target crowd type can be recalled by the crowd recall service system or not through response information returned by the crowd recall service. For example, if a target crowd type is included in the response message returned by the crowd recall service, it may be proved that the crowd recall service can recall the target crowd type.
In the specific implementation, a request associated with a certain user identifier may occur, and the crowd recall service cannot recall the corresponding crowd type, for example, a certain target crowd type is that "a certain shop has been visited in the last three days", and it is determined that a target user corresponding to the target crowd type has a user A, B, C through historical log record information, and the like; after the crowd recall service call request is respectively constructed by using the information such as the ID of each user and is sent to the crowd recall service system, the crowd recall service system may not include the target crowd type for the crowd type that the user a may return. However, since there is also a user B, C or the like for checking the target crowd type, it can be determined that the target crowd type can be recalled normally as long as the returned crowd type includes the target crowd type for the request constructed by another user such as the user B, C.
In addition, in practical applications, since the total number of people types may be relatively large, for example, in the millions as described in the foregoing examples, a specific people recall service system may divide a specific people recall service into a plurality of sub-services, and deploy the sub-services in a plurality of different rooms, where each sub-service may provide a recall service for a part of people types. For example, assuming a total of 100 ten thousand crowd types, 10 sub-services may be divided, deployed in 10 different rooms, each providing a recall service for 10 ten thousand crowd types, and so on. Each sub-service corresponds to a specific crowd type, and may be configured by a crowd recall service system.
In this case, specifically, when the service provided by the crowd recall service system is called, since the target crowd type corresponding to the specific target user is known, the service configuration information of the crowd recall service system may also be acquired, and the correspondence between the sub-service and the crowd type may be acquired according to the service configuration information. Therefore, the sub-service corresponding to the target crowd type can be called according to the corresponding relation. For example, for a service invocation request constructed according to an ID of a certain target user, when the request is sent, a sub-service corresponding to the target crowd type may be determined according to the target crowd type corresponding to the target user, and then the sub-service may be invoked by using the request. By the method, active detection of the individual rooms for the crowd recall service can be realized.
In particular, in implementation, the service configuration information of the specific crowd recall service system may be dynamically changed, that is, the corresponding relationship between each sub-service and the specific crowd type may be changed in real time. Therefore, the detection system can also acquire the service configuration information of the crowd recall service system in real time so as to acquire the latest corresponding relation between the sub-service and the crowd type.
Through the method, as long as the response returned by the specific crowd recall service comprises the specific target crowd type, the target crowd type can be determined to be recalled normally. In addition, during specific implementation, the correctness of the specific recalled crowd type can be checked. Specifically, in the embodiment of the present application, since the mapping relationship between the plurality of second user identifiers and the second crowd type can be determined in advance through the historical log record information, the mapping relationship can be used as an expected result. And after the constructed service calling request is used for calling the service provided by the crowd recall service system, determining a mapping relation between the target user and the third crowd type according to the response result as an actual result. In this way, the correctness of the recall result can be verified by judging whether the actual result is consistent with the expected result, that is, whether the third crowd type corresponding to the same user identifier is consistent with the second crowd type.
For example, the target user determined for a certain target crowd type 1 according to the history log record information includes a user a, that is, a mapping relationship exists between the user a and the target crowd type 1. After the service call request is constructed by using the user a identifier and sent to the crowd recall service system, if the corresponding crowd type is also the crowd type 1 in the returned response parameters, the fact that the crowd type recalled by the specific crowd recall service system is correct can be proved.
Of course, in practical applications, there may be a case where the third crowd type corresponding to a certain user identifier is inconsistent with the second crowd type. This may be due to errors in the crowd recall result from the crowd recall service system, or due to changes in the crowd construction result in the crowd construction system, etc. Therefore, when the inconsistency occurs, the fourth crowd type corresponding to the user identifier can be determined by querying the crowd construction system. A correctness check of the recall result may also be determined if the third population type is consistent with the fourth population type.
After the detection of the target crowd types is completed, the crowd types which cannot be recalled can be counted and summarized, and then attribution analysis can be carried out on specific crowd types which cannot be recalled, and the specific reason why the crowd types cannot be recalled is analyzed. After the attribution analysis is completed, the specific problem can be automatically repaired according to the specific analysis result, so that the self-healing is realized; and for the problem which cannot be automatically repaired, early warning information can be provided, and the like.
In a word, at least one recallable crowd type can be determined according to the real-time log record information generated by the crowd recall service system in the service providing process, namely, the part of crowd types can be verified through the real-time log record information, so that the part of crowd types can be filtered from the full crowd type set, and the recallable crowd types only need to be further detected if the recallable crowd types cannot be determined from the current real-time log record information. Thus, the number of target population types that need to be detected is reduced. In addition, since specific target users corresponding to the target crowd types can be obtained from the historical log record information, only the ID of the target users is needed to be used for simulating and constructing service requests and initiating requests to the crowd recall service, and then whether the target crowd types can be recalled or not can be determined according to response information returned by the crowd recall service. By the method, only part of target crowd types need to be detected by simulating and constructing the service call request, and target users having historical mapping relation with the target crowd types can be selected in a directional mode to construct the service call request, so that the number of the service call requests required to be constructed and sent can be greatly reduced, and the influence of a specific real-time detection process on the flow on the crowd recall service actual line is reduced.
It should be noted that, in the embodiments of the present application, the user data may be used, and in practical applications, the user-specific personal data may be used in the scheme described herein within the scope permitted by the applicable law, under the condition of meeting the requirements of the applicable law and regulations in the country (for example, the user explicitly agrees, the user is informed, etc.).
Corresponding to the foregoing method embodiment, an embodiment of the present application further provides an apparatus for detecting a people recall service, and referring to fig. 4, the apparatus may include:
a recalling crowd type determining unit 401, configured to obtain a first mapping relationship between a plurality of first user identifiers and a first crowd type according to real-time log record information generated by a crowd recall service system in a service providing process, and determine at least one recalling crowd type according to the first mapping relationship;
a filtering unit 402, configured to filter the at least one recallable crowd type from the full crowd type set, and determine at least one target crowd type to be detected according to a filtered result;
a target user determining unit 403, configured to obtain a second mapping relationship between a plurality of second user identifiers and a second crowd type according to historical log record information generated by the crowd recall service system in the process of providing services, and determine a plurality of target users for the target crowd type according to the second mapping relationship;
a service invoking unit 404, configured to simulate and construct a service invoking request by using the identifiers of the multiple target users, and detect whether the crowd recall service system can recall the target crowd type by invoking a service provided by the crowd recall service system.
Specifically, the recallable people type determining unit may be specifically configured to:
determining request parameters associated with a plurality of service call requests received in the current detection period and response parameters associated with corresponding response messages according to the real-time log recording information;
and acquiring a first mapping relation between the first user identifications and the first crowd type according to the identification of the first user included in the request parameter and the information of the first crowd type included in the corresponding response parameter.
Additionally, the filtration unit may be further operable to:
filtering out the expired, failed and/or target list people types from the full population type set so as to determine the target people types needing to be detected.
The target user determination unit may specifically be configured to:
acquiring request parameters associated with a plurality of historical service calling requests and response parameters associated with corresponding historical response messages according to the historical log record information;
and determining a mapping relation between the identifier of the second user and the second crowd type according to the identifier of the second user included in the request parameter and the information of the second crowd type included in the corresponding response parameter.
The crowd recall service system comprises a plurality of sub-services, wherein the sub-services are respectively deployed in a plurality of different servers, and each sub-service is used for providing recall service of part of crowd types;
at this time, the service invoking unit may specifically be configured to:
acquiring service configuration information of the crowd recall service system, and acquiring a corresponding relation between the sub-service and a crowd type according to the service configuration information;
and calling the sub-service corresponding to the target crowd type according to the corresponding relation.
Wherein the service configuration information may be dynamically changed;
at this time, the service invoking unit may specifically be configured to:
and acquiring the service configuration information of the crowd recall service system in real time.
In addition, the apparatus may further include:
a response result determining unit, configured to determine, after the constructed service invocation request is used to invoke the service provided by the crowd recall service system, a third crowd type recalled for the target user in a returned response result;
and the correctness checking unit is used for checking the correctness of the recall result by judging whether the third crowd type corresponding to the same user identifier is consistent with the second crowd type.
In addition, the apparatus may further include:
the query unit is used for determining a fourth crowd type corresponding to a certain user identifier by querying a crowd construction system if the third crowd type corresponding to the certain user identifier is inconsistent with the second crowd type; and if the third crowd type is consistent with the fourth crowd type, determining that the correctness check of the recall result is passed.
Furthermore, the apparatus may further include:
and the summarizing result providing unit is used for determining the nonrecoverable crowd type after detecting whether the target crowd type can be recalled or not by the crowd recall service system and providing the summarizing result of the nonrecoverable crowd type.
In addition, the apparatus may further include:
an attribution analysis unit for performing attribution analysis for the non-recallable crowd type;
and the repair early warning unit is used for repairing the problems existing in the crowd recall service system according to the attribution analysis result or providing early warning information.
In addition, the present application also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps of the method described in any of the preceding method embodiments.
And an electronic device comprising:
one or more processors; and
a memory associated with the one or more processors for storing program instructions that, when read and executed by the one or more processors, perform the steps of the method of any of the preceding method embodiments.
Fig. 5 exemplarily shows an architecture of an electronic device, and may specifically include a processor 510, a video display adapter 511, a disk drive 512, an input/output interface 513, a network interface 514, and a memory 520. The processor 510, the video display adapter 511, the disk drive 512, the input/output interface 513, the network interface 514, and the memory 520 may be communicatively connected by a communication bus 530.
The processor 510 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits, and is configured to execute related programs to implement the technical solution provided in the present Application.
The Memory 520 may be implemented in the form of a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, or the like. The memory 520 may store an operating system 521 for controlling the operation of the electronic device 500, a Basic Input Output System (BIOS) for controlling low-level operations of the electronic device 500. In addition, a web browser 523, a data storage management system 524, and a service detection processing system 525, among others, may also be stored. The service detection processing system 525 may be an application program that implements the operations of the foregoing steps in this embodiment of the application. In summary, when the technical solution provided in the present application is implemented by software or firmware, the relevant program codes are stored in the memory 520 and called to be executed by the processor 510.
The input/output interface 513 is used for connecting an input/output module to realize information input and output. The i/o module may be configured as a component in a device (not shown) or may be external to the device to provide a corresponding function. The input devices may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output devices may include a display, a speaker, a vibrator, an indicator light, etc.
The network interface 514 is used for connecting a communication module (not shown in the figure) to realize communication interaction between the device and other devices. The communication module can realize communication in a wired mode (such as USB, network cable and the like) and also can realize communication in a wireless mode (such as mobile network, WIFI, Bluetooth and the like).
Bus 530 includes a path that transfers information between the various components of the device, such as processor 510, video display adapter 511, disk drive 512, input/output interface 513, network interface 514, and memory 520.
It should be noted that although the above-mentioned devices only show the processor 510, the video display adapter 511, the disk drive 512, the input/output interface 513, the network interface 514, the memory 520, the bus 530, etc., in a specific implementation, the device may also include other components necessary for normal operation. Furthermore, it will be understood by those skilled in the art that the apparatus described above may also include only the components necessary to implement the solution of the present application, and not necessarily all of the components shown in the figures.
From the above description of the embodiments, it is clear to those skilled in the art that the present application can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the present application may be essentially or partially implemented in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments of the present application.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the system or system embodiments are substantially similar to the method embodiments and therefore are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for related points. The above-described system and system embodiments are only illustrative, wherein the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
The method, the device and the electronic device for detecting the people recall service provided by the application are introduced in detail, a specific example is applied in the description to explain the principle and the implementation mode of the application, and the description of the embodiment is only used for helping to understand the method and the core idea of the application; meanwhile, for a person skilled in the art, according to the idea of the present application, the specific embodiments and the application range may be changed. In view of the above, the description should not be taken as limiting the application.

Claims (13)

1. A method of detecting a crowd recall service, comprising:
acquiring a first mapping relation between a plurality of first user identifications and a first crowd type according to real-time log record information generated by a crowd recall service system in the service providing process, and determining at least one recallable crowd type according to the first mapping relation;
filtering the at least one recallable crowd type from a full-volume crowd type set, and determining at least one target crowd type to be detected according to a filtered result;
acquiring a second mapping relation between a plurality of second user identifications and a second crowd type according to historical log record information generated by the crowd recall service system in the service providing process, and determining a plurality of target users for the target crowd type according to the second mapping relation;
and simulating and constructing a service calling request by utilizing the identifications of the target users, calling the service provided by the crowd recall service system, and detecting whether the crowd recall service system can recall the target crowd type.
2. The method of claim 1,
the obtaining a first mapping relationship between a plurality of first user identifiers and a first type of a crowd includes:
determining request parameters associated with a plurality of service call requests received in the current detection period and response parameters associated with corresponding response messages according to the real-time log recording information;
and acquiring a first mapping relation between the first user identifications and the first crowd type according to the identification of the first user included in the request parameter and the information of the first crowd type included in the corresponding response parameter.
3. The method of claim 1, further comprising:
filtering out the expired, failed and/or target list people types from the full population type set so as to determine the target people types needing to be detected.
4. The method of claim 1,
the obtaining a second mapping relationship between a plurality of second user identifiers and a second crowd type includes:
acquiring request parameters associated with a plurality of historical service calling requests and response parameters associated with corresponding historical response messages according to the historical log record information;
and determining a mapping relation between the identifier of the second user and the second crowd type according to the identifier of the second user included in the request parameter and the information of the second crowd type included in the corresponding response parameter.
5. The method of claim 1,
the service provided by the crowd recall service system comprises a plurality of sub-services which are respectively deployed in a plurality of different servers, and each sub-service is used for providing recall service of part of crowd types;
the calling the service provided by the people group recall service system comprises the following steps:
acquiring service configuration information of the crowd recall service system, and acquiring a corresponding relation between the sub-service and a crowd type according to the service configuration information;
and calling the sub-service corresponding to the target crowd type according to the corresponding relation.
6. The method of claim 5,
the service configuration information is dynamically changing;
the acquiring of the service configuration information of the crowd recall service system includes:
and acquiring the service configuration information of the crowd recall service system in real time.
7. The method of claim 1, further comprising:
after the constructed service calling request is used for calling the service provided by the crowd recall service system, determining a third crowd type recalled for the target user in a returned response result;
and checking the correctness of the recall result by judging whether the third crowd type corresponding to the same user identifier is consistent with the second crowd type.
8. The method of claim 7, further comprising:
if the third crowd type corresponding to a certain user identification is inconsistent with the second crowd type, determining a fourth crowd type corresponding to the user identification by inquiring the crowd construction system;
and if the third crowd type is consistent with the fourth crowd type, determining that the correctness check of the recall result is passed.
9. The method of claim 1, further comprising:
and after detecting whether the target crowd type can be recalled or not by the crowd recall service system, determining the non-recallable crowd type and providing a summarizing result of the non-recallable crowd type.
10. The method of claim 9, further comprising:
performing attribution analysis for the non-recallable population type;
and repairing the problems existing in the crowd recall service system according to the attribution analysis result, or providing early warning information.
11. An apparatus for detecting a crowd recall service, comprising:
the recalling crowd type determining unit is used for acquiring a first mapping relation between a plurality of first user identifications and a first crowd type according to real-time log record information generated by the crowd recall service system in the service providing process, and determining at least one recalling crowd type according to the first mapping relation;
the filtering unit is used for filtering the at least one recallable crowd type from a full crowd type set and determining at least one target crowd type to be detected according to a filtered result;
the target user determining unit is used for acquiring a second mapping relation between a plurality of second user identifications and a second crowd type according to historical log record information generated by the crowd recall service system in the service providing process, and determining a plurality of target users for the target crowd type according to the second mapping relation;
and the service calling unit is used for simulating and constructing a service calling request by utilizing the identifiers of the target users, calling the service provided by the crowd recall service system and detecting whether the crowd recall service system can recall the target crowd type.
12. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 10.
13. An electronic device, comprising:
one or more processors; and
a memory associated with the one or more processors for storing program instructions that, when read and executed by the one or more processors, perform the steps of the method of any of claims 1 to 10.
CN202110851532.6A 2021-07-27 2021-07-27 Method and device for detecting crowd recall service and electronic equipment Active CN113689232B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110851532.6A CN113689232B (en) 2021-07-27 2021-07-27 Method and device for detecting crowd recall service and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110851532.6A CN113689232B (en) 2021-07-27 2021-07-27 Method and device for detecting crowd recall service and electronic equipment

Publications (2)

Publication Number Publication Date
CN113689232A true CN113689232A (en) 2021-11-23
CN113689232B CN113689232B (en) 2023-07-28

Family

ID=78578005

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110851532.6A Active CN113689232B (en) 2021-07-27 2021-07-27 Method and device for detecting crowd recall service and electronic equipment

Country Status (1)

Country Link
CN (1) CN113689232B (en)

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110119142A1 (en) * 2009-09-11 2011-05-19 Nintendo Of America Inc. System and/or method for handling recalled product purchases and/or return/warranty requests
US20110258065A1 (en) * 2009-04-30 2011-10-20 Fordyce Iii Edward W Product Recall Platform Apparatuses, Methods And Systems
US20150032639A1 (en) * 2013-07-29 2015-01-29 Verizon Patent And Licensing Inc. System and method for providing notifications on product recalls
CN106227884A (en) * 2016-08-08 2016-12-14 深圳市未来媒体技术研究院 A kind of recommendation method of calling a taxi online based on collaborative filtering
CN108510309A (en) * 2018-02-27 2018-09-07 阿里巴巴集团控股有限公司 The method and device that advertisement is recalled
CN108897640A (en) * 2018-06-11 2018-11-27 上海交通大学 The detection system and method for erroneous position data in intelligent perception
US20190163809A1 (en) * 2017-11-30 2019-05-30 Bby Solutions, Inc. Streaming events analysis for search recall improvements
CN111008675A (en) * 2019-12-26 2020-04-14 口碑(上海)信息技术有限公司 Method and device for sampling and processing recall area
CN111444438A (en) * 2020-03-24 2020-07-24 北京百度网讯科技有限公司 Method, device, equipment and storage medium for determining recall permission rate of recall strategy
US20200242622A1 (en) * 2014-06-01 2020-07-30 Hadafsoft, LLC System and Method for Unified Product Recalls Analytics & Notification Platform
US20200402118A1 (en) * 2019-06-20 2020-12-24 Shopify Inc. Systems and methods for recommending merchant discussion groups based on merchant categories
CN112131456A (en) * 2019-06-24 2020-12-25 腾讯科技（北京）有限公司 Information pushing method, device, equipment and storage medium
CN113010790A (en) * 2021-03-30 2021-06-22 广州酷狗计算机科技有限公司 Content recommendation method, device, server and storage medium
CN113111216A (en) * 2020-01-13 2021-07-13 百度在线网络技术（北京）有限公司 Advertisement recommendation method, device, equipment and storage medium

Patent Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110258065A1 (en) * 2009-04-30 2011-10-20 Fordyce Iii Edward W Product Recall Platform Apparatuses, Methods And Systems
US20110119142A1 (en) * 2009-09-11 2011-05-19 Nintendo Of America Inc. System and/or method for handling recalled product purchases and/or return/warranty requests
US20150032639A1 (en) * 2013-07-29 2015-01-29 Verizon Patent And Licensing Inc. System and method for providing notifications on product recalls
US20200242622A1 (en) * 2014-06-01 2020-07-30 Hadafsoft, LLC System and Method for Unified Product Recalls Analytics & Notification Platform
CN106227884A (en) * 2016-08-08 2016-12-14 深圳市未来媒体技术研究院 A kind of recommendation method of calling a taxi online based on collaborative filtering
US20190163809A1 (en) * 2017-11-30 2019-05-30 Bby Solutions, Inc. Streaming events analysis for search recall improvements
CN108510309A (en) * 2018-02-27 2018-09-07 阿里巴巴集团控股有限公司 The method and device that advertisement is recalled
CN108897640A (en) * 2018-06-11 2018-11-27 上海交通大学 The detection system and method for erroneous position data in intelligent perception
US20200402118A1 (en) * 2019-06-20 2020-12-24 Shopify Inc. Systems and methods for recommending merchant discussion groups based on merchant categories
CN112131456A (en) * 2019-06-24 2020-12-25 腾讯科技（北京）有限公司 Information pushing method, device, equipment and storage medium
CN111008675A (en) * 2019-12-26 2020-04-14 口碑(上海)信息技术有限公司 Method and device for sampling and processing recall area
CN113111216A (en) * 2020-01-13 2021-07-13 百度在线网络技术（北京）有限公司 Advertisement recommendation method, device, equipment and storage medium
CN111444438A (en) * 2020-03-24 2020-07-24 北京百度网讯科技有限公司 Method, device, equipment and storage medium for determining recall permission rate of recall strategy
CN113010790A (en) * 2021-03-30 2021-06-22 广州酷狗计算机科技有限公司 Content recommendation method, device, server and storage medium

Also Published As

Publication number Publication date
CN113689232B (en) 2023-07-28

Similar Documents

Publication Publication Date Title
CN112559361A (en) Flow playback method, device, equipment and computer readable medium
WO2020248508A1 (en) Method, device and apparatus for evaluating advertising channel, and storage medium
EP2748781B1 (en) Multi-factor identity fingerprinting with user behavior
US20100281008A1 (en) Universal Tracking Agent System and Method
US20110125593A1 (en) Online marketing payment monitoring method and system
CN111680068B (en) Verification method, device, equipment and storage medium
US20130004933A1 (en) Increasing confidence in responses to electronic surveys
CN106911801B (en) method for associating user information and information pushing system
WO2010037003A1 (en) Evaluating online marketing efficiency
CN105894313A (en) Methods And Apparatus To Associate Transactions With Media Impressions
US20170053307A1 (en) Techniques for detecting and verifying fraudulent impressions
US20160004974A1 (en) Detecting unexpected behavior
US9306958B2 (en) Methods, systems and media for detecting non-intended traffic using co-visitation information
CN110309373B (en) Information processing method and device
WO2011094809A1 (en) A method for notifying a sales person of a sales prospect
CN108650123B (en) Fault information recording method, device, equipment and storage medium
CN113378152A (en) Operation and maintenance auditing method and device, storage medium and electronic equipment
CN111767481B (en) Access processing method, device, equipment and storage medium
CN109003117A (en) Generation method, device and the computer readable storage medium of questionnaire
CN111045935B (en) Automatic version auditing method, device, equipment and storage medium
CN110825609B (en) Service testing method, device and system
CN113689232B (en) Method and device for detecting crowd recall service and electronic equipment
CN111563765A (en) Cheating user screening method, device and equipment and readable storage medium
CN111144987A (en) Abnormal shopping behavior limiting method, limiting assembly and shopping system
CN108229127B (en) System and method for generating authentication data in advance to distinguish clients

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right

Effective date of registration: 20230628

Address after: Room 101, Warehouse 1, No. 88, Jingbin Avenue, Jingbin Industrial Park, Wuqing District, Tianjin 301700

Applicant after: Alibaba Huabei Technology Co.,Ltd.

Address before: 310000 room 508, 5th floor, building 4, No.699 Wangshang Road, Changhe street, Binjiang District, Hangzhou City, Zhejiang Province

Applicant before: Alibaba (China) Co.,Ltd.

TA01 Transfer of patent application right
GR01 Patent grant
GR01 Patent grant