WO2016192488A1 - 灰度发布的评估方法、装置和系统 - Google Patents

灰度发布的评估方法、装置和系统 Download PDF

Info

Publication number
WO2016192488A1
WO2016192488A1 PCT/CN2016/080252 CN2016080252W WO2016192488A1 WO 2016192488 A1 WO2016192488 A1 WO 2016192488A1 CN 2016080252 W CN2016080252 W CN 2016080252W WO 2016192488 A1 WO2016192488 A1 WO 2016192488A1
Authority
WO
WIPO (PCT)
Prior art keywords
version
value
test parameter
upgraded
evaluation
Prior art date
Application number
PCT/CN2016/080252
Other languages
English (en)
French (fr)
Inventor
牟磊
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2016192488A1 publication Critical patent/WO2016192488A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/50—Testing arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08—Configuration management of networks or network elements
    • H04L41/0803—Configuration setting
    • H04L41/0813—Configuration setting characterised by the conditions triggering a change of settings
    • H04L41/082—Configuration setting characterised by the conditions triggering a change of settings the condition being updates or upgrades of network functionality
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08—Configuration management of networks or network elements
    • H04L41/0803—Configuration setting
    • H04L41/0823—Configuration setting characterised by the purposes of a change of settings, e.g. optimising configuration for enhancing reliability

Definitions

  • the present invention relates to the field of communications and, more particularly, to an evaluation method, apparatus and system for grayscale distribution.
  • Upgrading Internet software can include fixing vulnerabilities or bugs in the original version, optimizing the functionality of the original version, or releasing new features, and so on.
  • the evaluation method of grayscale release is generally adopted when upgrading the Internet software.
  • Grayscale publishing refers to a publishing method that can smoothly transition between black and white, in which some users continue to use the old version of the Internet software, while another part of the user uses the new version of the Internet software, if the other Some users have no objection to the new version, then gradually expand the scope of use of the new version, and finally migrate all users to the new version.
  • Grayscale publishing can instantly discover and adjust problems in the new version to ensure the overall stability of the system.
  • the user feedback and feedback can be collected through the feedback page or the forum, and the administrator can count the feedback of the user and make a version decision according to the statistical result. For example, to determine whether to release the new version on a large scale or to make improvements to the new version.
  • this whole process relies on user behavior and user initiative, lacking big data analysis for the entire version of the entire user base, thus affecting the quality and user satisfaction of the new version.
  • Embodiments of the present invention provide a method, an apparatus, and a system for evaluating grayscale distribution, which can be independent of Evaluation of grayscale release for user feedback.
  • the first aspect provides a method for evaluating a grayscale release, including: acquiring session information of multiple first sessions corresponding to an upgraded version of the service; determining a version of the upgraded version according to session information of the multiple first sessions The value of the test parameter, wherein the version test parameter includes at least one of the following parameters: a user behavior parameter and a version quality parameter; and determining an evaluation result of the upgrade version according to a value of the version test parameter of the upgrade version.
  • the evaluating method before the determining the evaluation result of the upgrade version according to the value of the version test parameter of the upgrade version, the evaluating method further includes: acquiring the original version corresponding to the service The session information of the second session is determined, and the value of the version test parameter of the original version is determined according to the session information of the plurality of second sessions; and the evaluation result of the upgrade version is determined according to the value of the version test parameter of the upgrade version, Including: determining the evaluation result of the upgrade version according to the value of the version test parameter of the original version and the value of the version test parameter of the upgrade version.
  • the evaluating method further includes: generating an upgrade version evaluation log, where the upgrade version evaluation log includes a version test of the original version and the upgrade version The comparison of the values of the parameters.
  • the determining method further includes: determining whether a value of the version test parameter of the upgraded version meets an intervention event triggering condition; if the version of the upgraded version is tested The value of the parameter satisfies the trigger condition of the intervention event, triggering the intervention event.
  • the intervention event includes at least one of: sending an alert message; changing a routing configuration of the at least one user from accessing the upgraded version to accessing The original version.
  • the intervention event triggering condition includes: comparing a value of a version test parameter of the upgraded version with a value of a version test parameter of the original version At or below a preset threshold; or the intervention event triggering condition includes: the value of the version test parameter of the upgraded version is higher or lower than a threshold, wherein the threshold is tested according to the version of the original version The value of the parameter is determined.
  • the obtaining The session information of the plurality of first sessions corresponding to the upgraded version of the service includes: receiving, by the server, session information of the plurality of first sessions corresponding to the upgraded version, where the server is configured to provide an upgraded version of the service.
  • the second aspect provides an apparatus for evaluating a grayscale distribution, comprising: an obtaining unit, configured to acquire session information of a plurality of first sessions corresponding to an upgraded version of the service; and a determining unit, configured to acquire the The session information of the first session determines the value of the version test parameter of the upgraded version, wherein the version test parameter includes at least one of the following parameters: a user behavior parameter and a version quality parameter, and according to the version of the upgrade version Test the value of the parameter to determine the evaluation result of the upgraded version.
  • the acquiring unit is further configured to: before the determining unit determines the evaluation result of the upgraded version according to the value of the version test parameter of the upgraded version acquired by the obtaining unit, The session information of the plurality of second sessions corresponding to the original version of the service; the determining unit is further configured to determine, according to the session information of the plurality of second sessions acquired by the acquiring unit, a value of the version test parameter of the original version; The unit is configured to determine the evaluation result of the upgrade version according to the value of the version test parameter of the original version and the value of the version test parameter of the upgrade version.
  • the evaluating apparatus further includes: a log generating unit, configured to generate an upgrade version evaluation log, where the upgrade version evaluation log includes the determining unit determining The result of the comparison of the original version with the value of the version test parameter of the upgraded version.
  • the determining unit is further configured to determine whether a value of the version test parameter of the upgraded version meets an intervention event triggering condition; the evaluating apparatus further includes: triggering And a unit, configured to trigger an intervention event if the determining unit determines that the value of the version test parameter of the upgraded version meets the intervention event trigger condition.
  • the intervention event includes at least one of: sending an alert message; changing a routing configuration of the at least one user by accessing the upgraded version to accessing The original version.
  • the intervention event triggering condition includes: a value of a version test parameter of the upgraded version and a version test of the original version
  • the comparison result of the value of the parameter is higher or lower than the preset threshold; or the trigger condition of the intervention event includes: the value of the version test parameter of the upgraded version is higher or lower than a threshold, wherein the threshold is Determined based on the value of the original version of the test parameter.
  • the acquiring unit is configured to receive, by using a server, session information of multiple first sessions corresponding to the upgrade version, where the server uses Provide an upgraded version of the service.
  • an evaluation system for grayscale distribution comprising: a first server, at least one second server, and at least one third server, wherein the at least one second server provides an upgraded version of the service, the at least one The third server provides an original version of the service; the at least one second server is configured to send, to the first server, session information of the plurality of first sessions corresponding to the upgraded version of the service; the at least one third server is configured to The first server sends the session information of the plurality of second sessions corresponding to the original version of the service; the first server is configured to determine, according to the session information of the plurality of first sessions, a value of the version test parameter of the upgraded version, And determining, according to the session information of the plurality of second sessions, a value of the version test parameter of the original version, where the version test parameter includes at least one of the following parameters: a user behavior parameter and a version quality parameter; the first server Also used to test the value of the parameter according to the version of the original version and the version test parameter of the upgraded
  • the method, the device and the system for evaluating the grayscale distribution obtain the session information of the plurality of first sessions corresponding to the upgraded version of the service according to the session information of the plurality of first sessions.
  • Determine the value of the version test parameter of the upgraded version and determine the evaluation result of the upgraded version of the service according to the value of the version test parameter of the upgraded version, and can perform big data on the upgraded version of the grayscale release without relying on user feedback. Analyze to get a more accurate assessment.
  • FIG. 1 is a schematic structural diagram of an evaluation system for grayscale distribution applied in an embodiment of the present invention.
  • FIG. 2 is a schematic diagram of an operational example of an evaluation system for grayscale distribution to which the embodiment of the present invention is applied.
  • FIG. 3 is a schematic diagram of a session in an embodiment of the present invention.
  • FIG. 4 is a schematic flowchart of a method for evaluating grayscale distribution according to an embodiment of the present invention.
  • FIG. 5 is a schematic block diagram of an apparatus for evaluating grayscale distribution according to an embodiment of the present invention.
  • FIG. 6 is a schematic block diagram of another grayscale publishing evaluation apparatus according to an embodiment of the present invention.
  • FIG. 1 is a schematic structural diagram of an evaluation system 100 for grayscale distribution according to an embodiment of the present invention.
  • the grayscale-issued evaluation system 100 includes a first server 102, which may be specifically a server for performing operations and maintenance, the first server 102 being configured with a version evaluation system for determining The value of the version test parameter of the upgraded version of the grayscale published service (ie the new version) and the value of the version test parameter of the original version of the service (ie the old version), and the test parameter and the original according to the version of the upgraded version
  • the value of the version test parameter of the version determines the evaluation result of the upgraded version of the service of the grayscale release, wherein the evaluation result of the upgraded version may include: whether the upgraded version has a vulnerability or a bug, and the upgraded version of the user Whether the experience meets expectations, and so on.
  • the version test parameter may include a user behavior parameter for reflecting user behavior and/or a version quality parameter for indicating a version quality, but the embodiment of the present invention is not limited thereto
  • the service may be specifically a software or an application, and the service may include an Internet service or a telecommunication service, but the embodiment of the present invention is not limited thereto.
  • the original version of the service ie the old version
  • the upgraded version of the service is the original An improved version of the original version, such as fixing bugs or bugs in the original version, optimizing the functionality of the original version, having new features not available in the original version, and so on.
  • an upgraded version of the service is published in grayscale, only a subset of users are allowed to use the upgraded version, while other users still use the original version of the service.
  • the grayscale published evaluation system 100 further includes one or more second servers 104 and one or more third servers 106, wherein the second server 104 is configured with an upgraded version of the service, and Providing an upgraded version of the service to the user upon receiving the service usage request of the user for the service; the third server 106 is configured with the original version of the same service, and is configured to receive the service usage request of the service for the service , to provide users with the original version of the service.
  • the second server 104 is configured with an upgraded version of the Internet system and/or the application service system
  • the third server 106 is configured with an original version of the Internet system and/or the application service system, but the embodiment of the present invention is not limited thereto.
  • FIG. 1 exemplarily shows only one second server 104 and one third server 106, but the embodiment of the present invention is not limited thereto.
  • the second server 104 is further configured to acquire session information of multiple first sessions corresponding to the upgraded version, and send session information of the multiple first sessions to the first server 102, and correspondingly, the first The server 102 may determine the value of the version test parameter of the upgraded version according to the session information of the plurality of first sessions.
  • the service is embodied as a website, and a session may correspond to a user's access to the website.
  • the session information may include user information, page access information, and service content.
  • the session information may include one or more of the following information: user information, total number of visited pages, time of stay (or duration of the session, indicating the total duration of the session), page time (representation) The length of time that the visitor stays on each page viewed, the number of service schedules, the service scheduling success rate, and the service call delay, specifically, user information (such as user accounts, etc.) and page access information (for example, total number of visited pages, The website stay time and the page time, etc.) may be acquired by the Internet system configured in the second server 104 and sent to the first server 102, and the service content invoking information (eg, the number of service schedules, the service schedule success rate, and Service call delay, etc.) may be obtained and directed by the application service system configured in the second server 104
  • the first server 102 transmits, but the embodiment of the present invention is not limited thereto.
  • the Internet system and the application service system are configured in the second server 104.
  • the Internet system configured in the second server 102 can acquire and The server 102 reports the user information and the page related information of the current access, and the Internet system can invoke the application service system configured in the second server 104, and the application service system acquires and reports the service to the first server 102.
  • the call information of the content for example, the number of calls to query the goods service, whether the call is successful, and so on.
  • the first server 102 may perform statistics according to the information of the multiple accesses reported by the Internet system and the application server, and obtain the evaluation result of the upgraded version, but the embodiment of the present invention is not limited thereto. In this way, the administrator can evaluate the evaluation results obtained by the system according to the version, determine whether the evaluation result of the upgraded version meets the expectations and make a decision.
  • Figure 3 shows an example of a visitor's session (or access) to a website.
  • the visitor requests the second server 104 to open the home page of the website, and accordingly, the second server 104 can create a session based on the request.
  • the visitor browsed the other two pages of the site (ie, the second page and the third page).
  • the second server 104 can acquire session information of the current session of the visitor. Taking the session information including the webpage dwell time and the website dwell time as an example, the second server 104 can determine, according to the following content recorded in the user access log, that the time at which the visitor sends a request to open the website home page to the server is 10 o'clock:
  • the third server 106 is further configured to acquire session information of multiple second sessions corresponding to the original version of the service, and send session information of the multiple second sessions to the first server 102
  • the Internet system configured in the third server 106 can acquire and send information such as the page stay time, the website stay time, and the like of the second session to the first server 102
  • the application service system configured in the third server 106 can obtain and
  • the first server 102 transmits information such as a service call success rate, a service call delay, and the like corresponding to the plurality of second sessions.
  • the first server 102 may determine the value of the version test parameter of the original version of the service according to the session information of the multiple second sessions, but the embodiment of the present invention is not limited thereto.
  • the second server 104 and the third server 106 respectively collect the user behavior data of each session in the upgraded version and the original version, so that the first server 102 can obtain a large amount of user behavior data of the original version and the upgraded version. This makes it possible to make a more accurate assessment of the upgraded version without relying on user feedback.
  • FIG. 2 and FIG. 3 are intended to help those skilled in the art to better understand the embodiments of the present invention and not to limit the scope of the embodiments of the present invention.
  • a person skilled in the art will be able to make various modifications or changes in the embodiments according to the examples of FIG. 2 and FIG. 3, and such modifications or variations are also within the scope of the embodiments of the present invention.
  • the grayscale publishing evaluation system 100 may further include a grayscale publishing manager, configured to determine a first type of user and a second type of user, wherein the first type of user accesses an upgraded version of the service (ie, a new version) The second type of user accesses the original version of the service (ie, the old version), and issues the first type of user and The routing configuration of the second type of user.
  • the first server 102 may also trigger the grayscale release manager to change the routing configuration of the first type user or the second type of user according to the value of the version test parameter of the upgraded version.
  • the first server 102 sends the indication information to the grayscale publishing manager after determining that the number of accesses of the upgraded version is reduced to less than half of the original version, and the grayscale publishing manager changes the first type of users according to the indication information.
  • the routing configuration of some or all of the users is such that the part or all of the users are changed from accessing the upgraded version to accessing the original version, but the embodiment of the present invention is not limited thereto.
  • the evaluation system of the grayscale distribution acquires and transmits the session information of the plurality of first sessions corresponding to the upgraded version of the service to the first server by the at least one second server, and the at least one third server acquires the And sending, by the first server, session information of the plurality of first sessions corresponding to the original version of the service, the first server determining, according to the session information of the multiple first sessions, a value of the version test parameter of the upgraded version, according to the multiple The session information of the second session, determining the value of the version test parameter of the original version, determining the value of the version test parameter of the upgrade version according to the session information of the plurality of first sessions, and comparing the original version and the upgrade version
  • the version of the test parameter value, the upgrade version of the service is evaluated, and the big version of the upgraded version of the grayscale release can be analyzed without relying on user feedback, thereby obtaining a more accurate evaluation result.
  • FIG. 4 shows an evaluation method 200 for grayscale distribution provided by an embodiment of the present invention.
  • the evaluation method may be performed by the evaluation device of the grayscale distribution, wherein the evaluation device may be specifically the first server 102 in the above embodiment or the first server 102, and the embodiment of the present invention is not limited thereto.
  • the session information of the plurality of first sessions may include one or more of the following information: user information, total number of visited pages, website time (or session duration, indicating the total duration of the session), and page time (indicating that the visitor is present)
  • user information total number of visited pages
  • website time or session duration, indicating the total duration of the session
  • page time indicating that the visitor is present
  • the duration of the stay on each page of the browsing, the number of service schedules, the service scheduling success rate, and the service call delay but the embodiment of the present invention is not limited thereto.
  • the evaluation device may obtain session information of the plurality of first sessions by using a plurality of manners.
  • the evaluation device may acquire session information of the plurality of first sessions from at least one server for providing the upgraded version, but Embodiments of the invention are not limited thereto.
  • S220 Determine, according to the session information of the multiple first sessions, a version test parameter of the upgraded version. Value.
  • the evaluation device may obtain session information of each of the plurality of first sessions, and determine, according to session information of each of the plurality of first sessions, version test parameters of the upgraded version. value. For example, the evaluation device may perform statistical processing on the session information of each of the plurality of first sessions to obtain a value of the version test parameter of the upgraded version. For example, the evaluation apparatus may determine the amount of browsing users and/or the number of loyal visitors of the upgraded version according to the page staying time of each of the plurality of first sessions, but the embodiment of the present invention is not limited thereto.
  • the version test parameter of the upgraded version may include at least one of the following parameters: a user behavior parameter and a version quality parameter.
  • the user behavior parameter is used to reflect the user's behavior for the upgraded version.
  • the user behavior parameter may include at least one of the following parameters: website rate (Take Rates or Conversions Rates), returning person rate ( Repeat Visitor Share), Heavy User Share, Committed Visitor Share, Committed Visitor Index, Committed Visitor Volume, Visitor Participation Index Visitor Engagement Index, Scanning Visitor Share, Scanning Visitor Index, and Scanning Visitor Volume, where the website conversion rate is equal to the total number of visits for the corresponding action.
  • the proportion of the action may be specifically a login, registration, subscription, download, purchase, etc., and may represent a visitor-to-user conversion ratio, which is used to measure the degree of attraction of the website content to the visitor and the promotion effect of the website.
  • the return visitor ratio is equal to the number of returning visitors in independence
  • the proportion of the number of inquirers used to measure the attraction of the content of the website to the visitor and the usefulness of the website; the ratio of active visitors is equal to the proportion of visitors who have visited the preset value in the total number of visits.
  • the proportion of visitors who are highly interested in the content of the website; the percentage of loyal visitors is equal to the proportion of the number of users whose access time exceeds the preset value in the total number of users; the loyal visitor index is equal to the stay time exceeding the preset value.
  • the number of visited pages divided by the number of visitors whose access duration exceeds the preset value is used to indicate the average number of visited pages per long-time visitor; the number of loyal visitors equals the number of visited pages whose stay duration exceeds the preset value on the total access page
  • the proportion of the number; the visitor participation index is equal to the total number of visits divided by the number of independent visitors, used to represent the average session of each visitor; the browse user ratio is equal to the stay The number of visitors whose time is shorter than 1 minute divided by the total number of visits; the browse user index is equal to the proportion of the number of visits with a stay time shorter than 1 minute in the number of visitors whose stay time is shorter than 1 minute; the number of browse users is equal to The ratio of the number of browsing pages with a dwell
  • This version quality parameter can be used to reflect whether the version has a vulnerability or bug.
  • the version quality parameter may include at least one of the following parameters: a service call success rate and a service call average delay, or other information, and the embodiment of the present invention is not limited thereto.
  • the evaluation result of the upgraded version may include whether the upgraded version has a vulnerability or a bug, whether the user experience of the upgraded version meets expectations, and the like.
  • the evaluation apparatus may determine, according to the value of the version quality parameter of the upgraded version, whether the upgraded version has a vulnerability or a bug, or the evaluation apparatus may determine the upgraded version of the user according to the value of the upgraded version of the user behavior parameter. Whether the experience is in line with expectations, but embodiments of the present invention are not limited thereto.
  • the evaluation method of the grayscale distribution determines the version test of the upgraded version according to the session information of the plurality of first sessions by acquiring the session information of the plurality of first sessions corresponding to the upgraded version of the service.
  • the value of the parameter, and based on the value of the version test parameter of the upgraded version determines the evaluation result of the upgraded version of the service, and can perform big data analysis on the upgraded version of the grayscale release without relying on user feedback, thereby obtaining more accurate Evaluation results.
  • the grayscale publishing evaluation method 200 further includes:
  • S230 determining, according to the value of the version test parameter of the upgraded version, the evaluation result of the upgraded version, including:
  • the evaluation result of the upgrade version is determined according to the value of the version test parameter of the original version and the value of the version test parameter of the upgrade version.
  • the specific information included in the session information of the multiple second sessions may be related to the multiple first
  • the version test parameter of the upgrade version may include at least one of the following parameters: a user behavior parameter and a version quality parameter, and the user behavior parameter and the version quality parameter of the original version are
  • the specific parameters included may correspond to the specific parameter types included in the user behavior parameter and the version quality parameter of the upgraded version, and are not described herein for brevity.
  • the evaluation device may acquire session information of each second session of the plurality of second sessions, and determine a version test of an original version of the service according to session information of each second session of the plurality of second sessions The value of the parameter.
  • the evaluating device may obtain session information of the plurality of second sessions from at least one server for providing the original version, but the embodiment of the present invention is not limited thereto.
  • the evaluation device may determine the upgrade version by comparing the value of the original version and the corresponding version test parameter of the upgrade version.
  • the result of the evaluation for example, the user version of the upgrade version relative to the original version.
  • the evaluation device may determine whether the user experience of the upgraded version meets expectations by comparing the values of the original user behavior and the corresponding user behavior parameter of the upgraded version, but the embodiment of the present invention is not limited thereto.
  • the grayscale publishing evaluation method 200 further includes:
  • An upgrade version evaluation log is generated, wherein the upgrade version evaluation log includes a comparison result of the original version and the value of the version test parameter of the upgrade version.
  • the administrator can determine the advantages and disadvantages of the upgraded version compared with the original version according to the comparison result between the upgraded version recorded in the upgraded version evaluation log and the corresponding version of the original version, so that the upgraded version can be used.
  • the upgrade version evaluation log may be specifically a table, for example, a comparison result of page stay times corresponding to various pages in the original version and the upgrade version is shown in Table 1.
  • the administrator can determine according to the table 1 that the upgraded version of the home page and payment page design needs further improvement, but this The embodiment of the invention is not limited thereto.
  • the upgrade version evaluation log may also include a comparison result of the number of service calls of the various service contents included in the original version and the upgrade version, so that the administrator can compare according to the comparison shown in Table 2. As a result, it is determined whether the quality of the upgraded version is as expected, but the embodiment of the present invention is not limited thereto.
  • the grayscale publishing evaluation method 200 further includes:
  • an intervention event is triggered.
  • the intervention event is an active intervention event, wherein the active intervention event may include one or more
  • the event may be triggered by different trigger conditions, but the embodiment of the present invention is not limited thereto.
  • the intervention event trigger condition may be preset.
  • the intervention event trigger condition includes: comparing the value of the version test parameter of the upgrade version with the value of the version test parameter of the original version is higher or lower than the pre-pre- Set the threshold.
  • the value of the at least one parameter included in the version test parameter of the upgrade version and the value of the at least one corresponding parameter included in the version test parameter of the original version may satisfy a preset condition, for example, a proportion of returning visitors of the upgraded version.
  • the interference event may be triggered if the difference or ratio of the ratio of the returnees of the original version is lower than a preset threshold, but the embodiment of the present invention is not limited thereto.
  • the intervention event triggering condition may include: the value of the version test parameter of the upgraded version is higher or lower than a threshold value, wherein the threshold value is determined according to a value of a version test parameter of the original version.
  • the evaluating apparatus may determine the threshold value according to the value of the at least one parameter included in the version test parameter of the original version, and determine whether the value of the at least one corresponding parameter included in the version test parameter of the upgraded version exceeds the gate. Limit. For example, the evaluation device may determine the threshold value as 120% of the active visitor proportion of the original version, and determine whether the active visitor ratio of the upgraded version is below the threshold value, and the active visitor in the upgraded version When the ratio is lower than the threshold, an intervention event is triggered, but the embodiment of the invention is not limited thereto.
  • the intervention event comprises at least one of the following:
  • the evaluation device may trigger the sending of the alert message when it is determined that the service health is degraded or fails, for example, if the upgrade service payment service call success rate is reduced by more than half compared with the original version, the evaluation device may Trigger the sending of an alert message.
  • the evaluation device may send an indication message to the mail server for instructing the mail server to send an alert email to the preset email address, where the preset email address may be an email address preset by the administrator; or, the evaluation The device may send an indication message to the short message gateway for instructing the short message gateway to send an alarm message to the preset number, where the preset number may be a preset number set by the administrator. In this way The sending of the triggered alarm message can notify the administrator of the decrease in the health of the service or the occurrence of the fault, thereby facilitating the administrator to immediately repair or adjust the upgraded version.
  • the evaluating apparatus may also trigger a change of the routing configuration when the number of accessing the landing page of the upgraded version is reduced to less than half of the original version, that is, the routing configuration of the at least one user currently allowed to access the upgraded version is accessed by The upgrade version was changed to access the original version.
  • the evaluation device may send a routing configuration change indication to the grayscale publishing manager to instruct the grayscale publishing manager to change the routing configuration of the at least one user, but the embodiment of the present invention is not limited thereto. In this way, by triggering a change in the routing configuration, it is possible to reduce the risk of user loss due to poor user experience due to the grayscale release of the upgraded version.
  • the evaluation method of the grayscale distribution determines the version test of the upgraded version according to the session information of the plurality of first sessions by acquiring the session information of the plurality of first sessions corresponding to the upgraded version of the service.
  • the value of the parameter, and based on the value of the version test parameter of the upgraded version determines the evaluation result of the upgraded version of the service, and can perform big data analysis on the upgraded version of the grayscale release without relying on user feedback, thereby obtaining more accurate Evaluation results.
  • determining the evaluation result of the upgrade version can provide a more visual quantification for the administrator.
  • the administrator can immediately repair or adjust the upgraded version, and reduce the risk of user loss due to poor user experience of the grayscale release of the upgraded version. .
  • FIG. 5 shows an evaluation device 300 for grayscale distribution according to an embodiment of the present invention.
  • the evaluation device 300 includes:
  • the obtaining unit 310 is configured to obtain session information of multiple first sessions corresponding to the upgraded version of the service
  • the determining unit 320 is configured to determine, according to the session information of the plurality of first sessions acquired by the acquiring unit 310, a value of a version test parameter of the upgraded version, where the version test parameter includes at least one of the following parameters: a user
  • the behavior parameter and the version quality parameter and the evaluation result of the upgrade version is determined according to the value of the version test parameter of the upgrade version.
  • the obtaining unit 310 is further configured to: after the determining unit 320 determines the evaluation result of the upgraded version according to the value of the version test parameter of the upgraded version acquired by the obtaining unit 310, obtain an original version corresponding to the service. Session information of multiple second sessions;
  • the determining unit 320 is further configured to determine a value of the version test parameter of the original version according to the session information of the plurality of second sessions acquired by the acquiring unit 310.
  • the determining unit 320 is configured to determine an evaluation result of the upgraded version according to the value of the version test parameter of the original version and the value of the version test parameter of the upgraded version.
  • the evaluation device 300 further includes:
  • a log generation unit configured to generate an upgrade version evaluation log, where the upgrade version evaluation log includes a comparison result between the original version determined by the determining unit 320 and a value of the version test parameter of the upgrade version.
  • the determining unit 320 is further configured to determine whether a value of the version test parameter of the upgraded version meets an intervention event triggering condition
  • the evaluation device 300 further includes:
  • the triggering unit is configured to trigger an intervention event if the determining unit 320 determines that the value of the version test parameter of the upgraded version meets the intervention event triggering condition.
  • the intervention event includes at least one of: sending an alert message; changing the routing configuration of the at least one user from accessing the upgraded version to accessing the original version.
  • the intervention event triggering condition includes: comparing the value of the version test parameter of the upgraded version with the value of the version test parameter of the original version is higher or lower than a preset threshold; or
  • the intervention event triggering condition includes: the value of the version test parameter of the upgraded version is higher or lower than a threshold value, wherein the threshold value is determined according to the value of the version test parameter of the original version.
  • the obtaining unit 310 is specifically configured to receive session information of multiple first sessions corresponding to the upgrade version sent by the server, where the server is configured to provide an upgraded version of the service.
  • the evaluation apparatus 300 for grayscale distribution according to an embodiment of the present invention may correspond to the first server 102 in the evaluation system 100 of the grayscale distribution according to the embodiment of the present invention, and the above and other operations of the respective modules in the evaluation apparatus 300
  • the functions and/or functions are respectively implemented in order to implement the corresponding processes of the respective methods in FIG. 4, and are not described herein for brevity.
  • the evaluation apparatus of the grayscale distribution determines the version test of the upgraded version according to the session information of the plurality of first sessions by acquiring the session information of the plurality of first sessions corresponding to the upgraded version of the service.
  • the value of the parameter, and based on the value of the version test parameter of the upgraded version determines the evaluation result of the upgraded version of the service, and can perform big data analysis on the upgraded version of the grayscale release without relying on user feedback, thereby obtaining more accurate Evaluation results.
  • determining the evaluation result of the upgrade version can provide a more visual quantification for the administrator.
  • the administrator can immediately repair or adjust the upgraded version, and reduce the risk of user loss due to poor user experience of the grayscale release of the upgraded version. .
  • FIG. 6 shows an evaluation device 400 for grayscale distribution according to an embodiment of the present invention.
  • the evaluation device 400 includes a processor 410 for:
  • a value of a version test parameter of the upgraded version where the version test parameter includes at least one of the following parameters: a user behavior parameter and a version quality parameter;
  • the evaluation result of the upgrade version is determined according to the value of the version test parameter of the upgraded version.
  • the processor 410 is further configured to:
  • the processor 410 is specifically configured to determine an evaluation result of the upgraded version according to the value of the version test parameter of the original version and the value of the version test parameter of the upgraded version.
  • the processor 410 is further configured to generate an upgrade version evaluation log, where the upgrade version evaluation log includes a comparison result of the value of the original version and the version test parameter of the upgrade version.
  • processor 410 is further configured to:
  • an intervention event is triggered.
  • the intervention event includes at least one of: sending an alert message; changing the routing configuration of the at least one user from accessing the upgraded version to accessing the original version.
  • the intervention event triggering condition includes: comparing the value of the version test parameter of the upgraded version with the value of the version test parameter of the original version is higher or lower than a preset threshold; or
  • the intervention event triggering condition includes: the value of the version test parameter of the upgraded version is higher or lower than a threshold value, wherein the threshold value is determined according to the value of the version test parameter of the original version.
  • the processor 410 is specifically configured to receive session information of multiple first sessions corresponding to the upgrade version sent by the server, where the server is configured to provide an upgraded version of the service.
  • the evaluation apparatus 400 for grayscale distribution according to an embodiment of the present invention may correspond to the first server 102 in the evaluation system 100 of the grayscale distribution according to the embodiment of the present invention, and the above and other operations of the respective modules in the evaluation apparatus 400
  • the functions and/or functions are respectively implemented in order to implement the corresponding processes of the respective methods in FIG. 4, and are not described herein for brevity.
  • the processor 410 may be a central processing unit (“CPU"), and the processor 410 may also be other general-purpose processors, digital signal processors (DSPs). , application specific integrated circuit (ASIC), off-the-shelf programmable gate array (FPGA) or Other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and the like.
  • the general purpose processor may be a microprocessor or the processor or any conventional processor or the like.
  • the evaluation device 400 can also include a memory, which can include read only memory and random access memory, and provides instructions and data to the processor 410.
  • a portion of the memory may also include a non-volatile random access memory.
  • the memory can also store information of the device type.
  • each step of the foregoing method may be completed by an integrated logic circuit of hardware in the processor 410 or an instruction in a form of software.
  • the steps of the method disclosed in the embodiments of the present invention may be directly implemented as a hardware processor, or may be performed by a combination of hardware and software modules in the processor.
  • the software module can be located in a conventional storage medium such as random access memory, flash memory, read only memory, programmable read only memory or electrically erasable programmable memory, registers, and the like.
  • the storage medium is located in the memory, and the processor 410 reads the information in the memory and completes the steps of the above method in combination with the hardware thereof. To avoid repetition, it will not be described in detail here.
  • the evaluation apparatus of the grayscale distribution determines the version test of the upgraded version according to the session information of the plurality of first sessions by acquiring the session information of the plurality of first sessions corresponding to the upgraded version of the service.
  • the value of the parameter, and based on the value of the version test parameter of the upgraded version determines the evaluation result of the upgraded version of the service, and can perform big data analysis on the upgraded version of the grayscale release without relying on user feedback, thereby obtaining more accurate Evaluation results.
  • determining the evaluation result of the upgrade version can provide a more visual quantification for the administrator.
  • the administrator can immediately repair or adjust the upgraded version, and reduce the risk of user loss due to poor user experience of the grayscale release of the upgraded version. .
  • An embodiment of the present invention further provides an evaluation system for grayscale distribution, comprising: a first server, at least one second server, and at least one third server, wherein the at least one second server provides an upgraded version of the service, the at least a third server provides an original version of the service; the at least one second server is configured to send, to the first server, session information of a plurality of first sessions corresponding to an upgraded version of the service; the at least one third server is configured to: Send the original with the service to the first server The session information of the plurality of second sessions corresponding to the version; the first server is configured to determine, according to the session information of the plurality of first sessions, a value of a version test parameter of the upgraded version, and according to the session of the multiple second sessions Information determining a value of the version test parameter of the original version, wherein the version test parameter includes at least one of the following parameters: a user behavior parameter and a version quality parameter; the first server is further configured to test according to the version of the original version The value of the parameter and the value of the
  • association relationship describing the associated object indicates that there may be three relationships.
  • a and/or B may indicate that A exists separately, and A and B exist simultaneously, and B cases exist alone.
  • the character / in this paper generally indicates that the contextual object is an OR relationship.
  • the disclosed systems, devices, and methods may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, or an electrical, mechanical or other form of connection.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the embodiments of the present invention.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium.
  • the instructions include a plurality of instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the evaluation method for grayscale distribution in accordance with various embodiments of the present invention.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program codes. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Transfer Between Computers (AREA)
  • Debugging And Monitoring (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

本发明公开了一种灰度发布的评估方法、装置和系统。该评估方法包括：获取服务的升级版本对应的多个第一会话的会话信息；根据该多个第一会话的会话信息，确定该升级版本的版本测试参数的值，其中，该版本测试参数包括下列参数中的至少一种：用户行为参数和版本质量参数；根据该升级版本的版本测试参数的值，确定该升级版本的评估结果。本发明公开的灰度发布的评估方法、装置和系统，能够不依赖于用户反馈，对灰度发布的升级版本进行大数据分析，从而获得更为准确的评估结果。

Description

灰度发布的评估方法、装置和系统
本申请要求于2015年6月1日提交中国专利局、申请号为201510293522.X，发明名称为“灰度发布的评估方法、装置和系统”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及通信领域，并且更具体地，涉及灰度发布的评估方法、装置和系统。
背景技术
互联网软件的升级可以包括修复原始版本中的漏洞或错误、对原始版本的功能进行优化或发布新功能，等等。为了降低在互联网软件的升级过程中出现的用户流失、系统崩溃等风险，一般在升级互联网软件时采用灰度发布的评估方法。灰度发布是指在黑与白之间能够平滑过渡的一种发布方式，在该方式中，一部分用户继续使用互联网软件的旧版本，而另一部分用户使用该互联网软件的新版本，如果该另一部分用户对新版本没有反对意见，那么逐步扩大新版本的使用范围，最后将所有用户都迁移到新版本。
灰度发布可以即时发现并调整新版本中存在的问题，保证系统的整体稳定性。现有技术中，在允许一部分用户体验新版本之后，可以通过意见反馈页面或论坛收集用户对新版本的体验和反馈意见，管理员可以统计用户的反馈意见，并根据统计结果做出版本决策，例如确定将该新版本进行大规模发布或确定对该新版本进行改进后发布。然而，这整个过程依赖于用户行为以及用户主动性，缺乏针对新版本的整个用户群的大数据分析，从而影响新版本的质量和用户满意度。
发明内容
本发明实施例提供了一种灰度发布的评估方法、装置和系统，能够不依赖 于用户反馈进行灰度发布的评估。
第一方面，提供了一种灰度发布的评估方法，包括：获取服务的升级版本对应的多个第一会话的会话信息；根据该多个第一会话的会话信息，确定该升级版本的版本测试参数的值，其中，该版本测试参数包括下列参数中的至少一种：用户行为参数和版本质量参数；根据该升级版本的版本测试参数的值，确定该升级版本的评估结果。
在第一方面的第一可能的实现方式中，在该根据该升级版本的版本测试参数的值，确定该升级版本的评估结果之前，该评估方法还包括：获取该服务的原始版本对应的多个第二会话的会话信息；根据该多个第二会话的会话信息，确定该原始版本的版本测试参数的值；该根据该升级版本的版本测试参数的值，确定该升级版本的评估结果，包括：根据该原始版本的版本测试参数的值和该升级版本的版本测试参数的值，确定该升级版本的评估结果。
结合上述可能的实现方式，在第一方面的第二可能的实现方式中，该评估方法还包括：生成升级版本评估日志，其中，该升级版本评估日志包括该原始版本与该升级版本的版本测试参数的值的对比结果。
结合上述可能的实现方式，在第一方面的第三可能的实现方式中，该评估方法还包括：确定该升级版本的版本测试参数的值是否满足干预事件触发条件；若该升级版本的版本测试参数的值满足该干预事件触发条件，触发干预事件。
结合上述可能的实现方式，在第一方面的第四可能的实现方式中，该干预事件包括下列中的至少一项：发送警报消息；将至少一个用户的路由配置由访问该升级版本更改为访问该原始版本。
结合上述可能的实现方式，在第一方面的第五可能的实现方式中，该干预事件触发条件包括：该升级版本的版本测试参数的值与该原始版本的版本测试参数的值的比较结果高于或低于预设门限值；或者该干预事件触发条件包括：该升级版本的版本测试参数的值高于或低于门限值，其中，该门限值是根据该原始版本的版本测试参数的值确定的。
结合上述可能的实现方式，在第一方面的第六可能的实现方式中，该获取 服务的升级版本对应的多个第一会话的会话信息，包括：接收服务器发送的与该升级版本对应的多个第一会话的会话信息，其中，该服务器用于提供该服务的升级版本。
第二方面，提供了一种灰度发布的评估装置，包括：获取单元，用于获取服务的升级版本对应的多个第一会话的会话信息；确定单元，用于根据该获取单元获取的该多个第一会话的会话信息，确定该升级版本的版本测试参数的值，其中，该版本测试参数包括下列参数中的至少一种：用户行为参数和版本质量参数，并且根据该升级版本的版本测试参数的值，确定该升级版本的评估结果。
在第二方面的第一可能的实现方式中，该获取单元还用于在该确定单元根据该获取单元获取的该升级版本的版本测试参数的值，确定该升级版本的评估结果之前，获取该服务的原始版本对应的多个第二会话的会话信息；该确定单元还用于根据该获取单元获取的该多个第二会话的会话信息，确定该原始版本的版本测试参数的值；该确定单元用于根据该原始版本的版本测试参数的值和该升级版本的版本测试参数的值，确定该升级版本的评估结果。
结合上述可能的实现方式，在第二方面的第二可能的实现方式中，该评估装置还包括：日志生成单元，用于生成升级版本评估日志，其中，该升级版本评估日志包括该确定单元确定的该原始版本与该升级版本的版本测试参数的值的对比结果。
结合上述可能的实现方式，在第二方面的第三可能的实现方式中，该确定单元还用于确定该升级版本的版本测试参数的值是否满足干预事件触发条件；该评估装置还包括：触发单元，用于若该确定单元确定该升级版本的版本测试参数的值满足该干预事件触发条件，触发干预事件。
结合上述可能的实现方式，在第二方面的第四可能的实现方式中，该干预事件包括下列中的至少一项：发送警报消息；将至少一个用户的路由配置由访问该升级版本更改为访问该原始版本。
结合上述可能的实现方式，在第二方面的第五可能的实现方式中，该干预事件触发条件包括：该升级版本的版本测试参数的值与该原始版本的版本测试 参数的值的比较结果高于或低于预设门限值；或者该干预事件触发条件包括：该升级版本的版本测试参数的值高于或低于门限值，其中，该门限值是根据该原始版本的版本测试参数的值确定的。
结合上述可能的实现方式，在第二方面的第六可能的实现方式中，该获取单元具体用于接收服务器发送的与该升级版本对应的多个第一会话的会话信息，其中，该服务器用于提供该服务的升级版本。
第三方面，提供了一种灰度发布的评估系统，包括：第一服务器、至少一个第二服务器和至少一个第三服务器，其中，该至少一个第二服务器提供服务的升级版本，该至少一个第三服务器提供该服务的原始版本；该至少一个第二服务器用于向该第一服务器发送与该服务的升级版本对应的多个第一会话的会话信息；该至少一个第三服务器用于向该第一服务器发送与该服务的原始版本对应的多个第二会话的会话信息；该第一服务器用于根据该多个第一会话的会话信息，确定该升级版本的版本测试参数的值，并且根据该多个第二会话的会话信息，确定该原始版本的版本测试参数的值，其中，该版本测试参数包括下列参数中的至少一种：用户行为参数和版本质量参数；该第一服务器还用于根据该原始版本的版本测试参数的值和该升级版本的版本测试参数的值，确定该升级版本的评估结果。
基于上述技术方案，根据本发明实施例的灰度发布的评估方法、装置和系统，通过获取服务的升级版本对应的多个第一会话的会话信息，根据该多个第一会话的会话信息，确定该升级版本的版本测试参数的值，并根据该升级版本的版本测试参数的值，确定该服务的升级版本的评估结果，能够不依赖于用户反馈，对灰度发布的升级版本进行大数据分析，从而获得更为准确的评估结果。
附图说明
为了更清楚地说明本发明实施例的技术方案，下面将对本发明实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面所描述的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造 性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本发明实施例应用的灰度发布的评估系统的架构示意图。
图2是本发明实施例应用的灰度发布的评估系统的运行示例的示意图。
图3是本发明实施例的一次会话的示意图。
图4是本发明实施例提供的灰度发布的评估方法的示意性流程图。
图5是本发明实施例提供的灰度发布的评估装置的示意性框图。
图6是本发明实施例提供的另一灰度发布的评估装置的示意性框图。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明的一部分实施例，而不是全部实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都应属于本发明保护的范围。
图1是本发明实施例应用的灰度发布的评估系统100的架构示意图。该灰度发布的评估系统100包括：第一服务器102，该第一服务器102可以具体为用于进行运营和维护的服务器，该第一服务器102配置有版本评估系统，该版本评估系统用于确定灰度发布的服务的升级版本(即新版本)的版本测试参数的值以及该服务的原始版本(即老版本)的版本测试参数的值，并且根据该升级版本的版本测试参数的和该原始版本的版本测试参数的值，确定该灰度发布的服务的升级版本的评估结果，其中，该升级版本的评估结果可以包括：该升级版本是否存在漏洞或故障(bug)，该升级版本的用户体验是否符合期望，等等。可选地，该版本测试参数可以包括用于反映用户行为的用户行为参数和/或用于表示版本质量的版本质量参数，但本发明实施例不限于此。
应理解，在本发明实施例中，服务可以具体为软件或应用程序，该服务可以包括互联网服务或电信服务，但本发明实施例不限于此。服务的原始版本(即老版本)可以是当前大规模发布的版本，在进行该服务的升级版本(即新版本)的灰度发布之前，所有用户均被允许使用该原始版本。服务的升级版本是该原 始版本的改进版本，例如修复了原始版本中的漏洞或错误、对原始版本的功能进行了优化、具有该原始版本所不具有的新功能，等等。在灰度发布该服务的升级版本时，只有一部分用户被允许使用该升级版本，而其他用户仍使用该服务的原始版本。
如图1所示，该灰度发布的评估系统100还包括一个或多个第二服务器104以及一个或多个第三服务器106，其中，该第二服务器104配置有服务的升级版本，并且用于在接收到用户针对该服务的服务使用请求时，为用户提供服务的升级版本；该第三服务器106配置有同一服务的原始版本，并且用于在接收到用户针对该服务的服务使用请求时，为用户提供该服务的原始版本。例如，该第二服务器104配置有互联网系统和/或应用服务系统的升级版本，而该第三服务器106配置有互联网系统和/或应用服务系统的原始版本，但本发明实施例不限于此。应理解，图1仅示例性地示出了一个第二服务器104和一个第三服务器106，但本发明实施例不限于此。
此外，该第二服务器104还用于获取与该升级版本对应的多个第一会话的会话信息，并且向该第一服务器102发送该多个第一会话的会话信息，相应地，该第一服务器102可以根据该多个第一会话的会话信息，确定该升级版本的版本测试参数的值。在本发明实施例中，以该服务具体体现为网站为例，一个会话可以对应于用户对网站的一次访问，此时，可选地，会话信息可以包括用户信息、页面访问信息和服务内容调用信息中的至少一种，例如，会话信息可以包括下列信息中的一种或多种：用户信息、访问页面总数、网站停留时间(或会话时长，表示会话的总时长)、页面停留时间(表示访客在浏览的每个页面上停留的时长)、服务调度次数、服务调度成功率和服务调用时延，具体地，用户信息(例如用户账号等等)和页面访问信息(例如，访问页面总数、网站停留时间和页面停留时间，等等)可以由该第二服务器104中配置的互联网系统获取并向该第一服务器102发送，而服务内容调用信息(例如，服务调度次数、服务调度成功率和服务调用时延，等等)可以由该第二服务器104中配置的应用服务系统获取并向该第一服务器102发送，但本发明实施例不限于此。
如图2所示，以第二服务器104中配置有互联网系统和应用服务系统为例，当用户访问该升级版本的页面时，该第二服务器102中配置的互联网系统可以获取并向该第一服务器102上报本次访问的用户信息和页面相关信息，并且该互联网系统可以调用该第二服务器104中配置的应用服务系统，该应用服务系统获取并向该第一服务器102上报本次访问的服务内容的调用信息，例如，查询商品服务的调用次数，调用是否成功，等等。此时，该第一服务器102可以根据该互联网系统和该应用服务器上报的多次访问的信息进行统计，获得该升级版本的评估结果，但本发明实施例不限于此。这样，管理员可以根据该版本评估系统获得的评估结果，确定该升级版本的评估结果是否符合预期并由此进行决策。
图3示出了访客对网站的某一次会话(或访问)的示例。首先，该访客向该第二服务器104请求打开该网站的首页，相应地，该第二服务器104可以根据该请求创建会话。随后，在离开该网站之前，该访客又浏览了该网站的另外两个页面(即第二个页面和第三个页面)。此时，该第二服务器104可以获取该访客的本次会话的会话信息。以该会话信息包括网页停留时间和网站停留时间为例，该第二服务器104可以根据用户访问日志中记录的如下内容，确定访客向服务器发出打开网站首页的请求的时刻为10点整：
111.111.111.111--[08/Oct/2009:10:00:00-0400]
"GET/index.html HTTP/1.1 200 10801
"http://www.google.com/search？q＝avinash+kaushik&ie＝utf-8&oe＝utf-8&aq＝t&rls＝org.mozilla:en-US:official&client＝firefox-a"
"Mozilla/5.0(Windows；U；Windows NT 5.2；en-US；rv:1.8.1.7)Gecko/20070914Firefox/2.0.0.7
此外，该第二服务器104还可以从用户访问日志中获取该访客发出的请求从该首页进入第二个页面的请求的时刻，例如，该访客在10:01请求打开第二个页面，则该第二服务器104可以根据该访客请求打开首页的时刻以及该访客请求从首页进入第二个页面的时刻，确定该访客在首页停留的时长，即TP(首页) ＝10:01-10:00＝1min。类似地，该第二服务器104可以从该用户访问日志获取该访客请求从第二个页面进入第三个页面的时刻为10:04，并由此确定该访客在第二个页面停留的时长TP(第二个页面)＝3min。该访客在浏览第三个页面之后离开了该网站，此时，该第二服务器104可以从用户访问日志获取该访客离开该第三个页面的时刻，或者通过页面中添加的额外代码，获取该访客离开该第三个页面的时刻，并由此确定该访客在第三个页面停留的时长TP(第三个页面)＝1min。随后，该第二服务器104可以确定本次会话的网站停留时间TS＝TP(首页)+TP(第二个页面)+TP(第三个页面)＝5min。
类似地，该第三服务器106还用于获取与该服务的原始版本对应的多个第二会话的会话信息，并向该第一服务器102发送该多个第二会话的会话信息，具体地，该第三服务器106中配置的互联网系统可以获取并向该第一服务器102发送多个第二会话的页面停留时间、网站停留时间等信息，该第三服务器106中配置的应用服务系统可以获取并向该第一服务器102发送多个第二会话对应的服务调用成功率、服务调用时延等信息。此时，该第一服务器102可以根据该多个第二会话的会话信息，确定该服务的原始版本的版本测试参数的值，但本发明实施例不限于此。
这样，通过该第二服务器104和第三服务器106分别主动搜集该升级版本和原始版本中各个会话的用户行为数据，可以使得该第一服务器102获得该原始版本和升级版本的大量用户行为数据，从而能够在不依赖于用户反馈的情况下对该升级版本做出较为准确的评估。
应注意，图2和图3的例子是为了帮助本领域技术人员更好地理解本发明实施例，而非要限制本发明实施例的范围。本领域技术人员根据所给出的图2和图3的例子，显然可以进行各种等价的修改或变化，这样的修改或变化也落入本发明实施例的范围内。
可选地，该灰度发布的评估系统100还可以包括灰度发布管理器，用于确定第一类用户和第二类用户，其中，第一类用户访问服务的升级版本(即新版本)，第二类用户访问服务的原始版本(即老版本)，以及下发该第一类用户和 该第二类用户的路由配置。此外，可选地，该第一服务器102还可以根据该升级版本的版本测试参数的值，触发该灰度发布管理器更改第一类用户或第二类用户的路由配置。例如，该第一服务器102在确定升级版本的访问数降低至原始版本的一半以下，则向该灰度发布管理器发送指示信息，该灰度发布管理器根据该指示信息更改第一类用户中的部分或全部用户的路由配置，以使得该部分或全部用户由访问升级版本更改为访问原始版本，但本发明实施例不限于此。
因此，根据本发明实施例的灰度发布的评估系统，通过至少一个第二服务器获取并向第一服务器发送服务的升级版本对应的多个第一会话的会话信息，至少一个第三服务器获取并向第一服务器发送服务的原始版本对应的多个第一会话的会话信息，该第一服务器根据该多个第一会话的会话信息，确定该升级版本的版本测试参数的值，根据该多个第二会话的会话信息，确定该原始版本的版本测试参数的值，根据该多个第一会话的会话信息，确定该升级版本的版本测试参数的值，并通过比较该原始版本和该升级版本的版本测试参数的值，对该服务的升级版本进行评估，能够不依赖于用户反馈，对灰度发布的升级版本进行大数据分析，从而获得更为准确的评估结果。
图4示出了本发明实施例提供的灰度发布的评估方法200。该评估方法可以由灰度发布的评估装置执行，其中，该评估装置可以具体为上述实施例中的第一服务器102或者设置于第一服务器102，本发明实施例不限于此。
S210，获取服务的升级版本对应的多个第一会话的会话信息。
该多个第一会话的会话信息可以包括下列信息中的一种或多种：用户信息、访问页面总数、网站停留时间(或会话时长，表示会话的总时长)、页面停留时间(表示访客在浏览的每个页面上停留的时长)、服务调度次数、服务调度成功率和服务调用时延，但本发明实施例不限于此。
该评估装置可以通过多种方式获取该多个第一会话的会话信息，可选地，该评估装置可以从用于提供该升级版本的至少一个服务器获取该多个第一会话的会话信息，但本发明实施例不限于此。
S220，根据该多个第一会话的会话信息，确定该升级版本的版本测试参数 的值。
该评估装置可以获取该多个第一会话中的每个第一会话的会话信息，并根据该多个第一会话中的每个第一会话的会话信息，确定该升级版本的版本测试参数的值。例如，该评估装置可以对该多个第一会话中的每个第一会话的会话信息进行统计处理，以获得该升级版本的版本测试参数的值。例如，该评估装置可以根据多个第一会话中的每个第一会话的页面停留时间，确定该升级版本的浏览用户量和/或忠实访问者量，但本发明实施例不限于此。
可选地，该升级版本的版本测试参数可以包括下列参数中的至少一种：用户行为参数和版本质量参数。该用户行为参数用于反映用户对于该升级版本的用于行为，可选地，该用户行为参数可以包括下列参数中的至少一种：网站转换率(Take Rates或Conversions Rates)、回访者比率(Repeat Visitor Share)、积极访问者比例(Heavy User Share)、忠实访问者比率(Committed Visitor Share)、忠实访问者指数(Committed Visitor Index)、忠实访问者量(Committed Visitor Volume)、访问者参与指数(Visitor Engagement Index)、浏览用户比率(Scanning Visitor Share)、浏览用户指数(Scanning Visitor Index)和浏览用户量(Scanning Visitor Volume)，其中，网站转换率等于进行了相应动作的访问量在总访问量中所占的比例，其中，该相应动作可以具体为登录、注册、订阅、下载、购买等行为，可以表示访客到用户的转换比率，用于衡量网站内容对访问者的吸引程度以及网站的宣传效果；回访者比率等于回访者数量在独立访问者数量中所占的比例，用于衡量网站内容对访问者的吸引程度和网站的实用性；积极访问者比率等于访问超过预设值的访问者在总访问数中所占的比例，用于衡量对网站的内容具有高度兴趣的访问者比例；忠实访问者比率等于访问时长超过预设值的用户数在总用户数中所占的比例；忠实访问者指数等于停留时间超过预设值的访问页数除以访问时长超过预设值的访问者数，用于表示每个长时间访问者的平均访问页数；忠实访问者量等于停留时长超过预设值的访问页数在总访问页数中所占的比例；访问者参与指数等于总访问数除以独立访问者数，用于表示每个访问者的平均会话；浏览用户比率等于停留 时间短于1分钟的访问者数除以总访问数；浏览用户指数等于停留时间短于1分钟的访问页面数在停留时间短于1分钟的访问者数中所占的比例；浏览用户量等于停留时间短于1分钟的浏览页数在总浏览页数中所占的比例，用于衡量在一分钟内完成的访问页面数的比率。可选地，该用户行为参数还可以包括其它参数，本发明实施例不限于此。
该版本质量参数可以用于反映版本是否存在漏洞或bug。可选地，该版本质量参数可以包括下列参数中的至少一种：服务调用成功率和服务调用平均时延，或者包括其它信息，本发明实施例不限于此。
S230，根据该升级版本的版本测试参数的值，确定该升级版本的评估结果。
该升级版本的评估结果可以包括该升级版本是否存在漏洞或bug，该升级版本的用户体验是否符合期望，等等。具体地，该评估装置可以根据该升级版本的版本质量参数的值，确定该升级版本是否存在漏洞或bug，或者该评估装置可以根据该升级版本的用户行为参数的值，确定该升级版本的用户体验是否符合预期，但本发明实施例不限于此。
因此，根据本发明实施例的灰度发布的评估方法，通过获取服务的升级版本对应的多个第一会话的会话信息，根据该多个第一会话的会话信息，确定该升级版本的版本测试参数的值，并根据该升级版本的版本测试参数的值，确定该服务的升级版本的评估结果，能够不依赖于用户反馈，对灰度发布的升级版本进行大数据分析，从而获得更为准确的评估结果。
作为另一个可选实施例，在S230之前，该灰度发布的评估方法200还包括：
获取该服务的原始版本对应的多个第二会话的会话信息；
根据该多个第二会话的会话信息，确定该原始版本的版本测试参数的值；
相应地，S230，根据该升级版本的版本测试参数的值，确定该升级版本的评估结果，包括：
根据该原始版本的版本测试参数的值和该升级版本的版本测试参数的值，确定该升级版本的评估结果。
可选地，该多个第二会话的会话信息中包括的具体信息可以与该多个第一 会话的会话信息中包括的具体信息相对应，该升级版本的版本测试参数可以包括下列参数中的至少一种：用户行为参数和版本质量参数，并且该原始版本的用户行为参数和版本质量参数中包括的具体参数可以分别与该升级版本的用户行为参数和版本质量参数中包括的具体参数类型相对应，为了简洁，这里不再赘述。
该评估装置可以获取该多个第二会话中的每个第二会话的会话信息，并根据该多个第二会话中的每个第二会话的会话信息，确定该服务的原始版本的版本测试参数的值。可选地，该评估装置可以从用于提供该原始版本的至少一个服务器获取该多个第二会话的会话信息，但本发明实施例不限于此。
在确定该原始版本的版本测试参数的值并且确定该升级版本的版本测试参数的值以后，该评估装置可以通过比较该原始版本和该升级版本的对应的版本测试参数的值，确定该升级版本的评估结果，例如，该升级版本相对于原始版本的用户活跃度。例如，该评估装置可以通过比较该原始版本和该升级版本的对应的用户行为参数的值，确定该升级版本的用户体验是否符合预期，但本发明实施例不限于此。
作为另一个可选实施例，该灰度发布的评估方法200还包括：
生成升级版本评估日志，其中，该升级版本评估日志包括该原始版本与该升级版本的版本测试参数的值的对比结果。
管理员可以根据该升级版本评估日志中记录的该升级版本与该原始版本的对应版本测试参数的值的对比结果，确定该升级版本与原始版本相比的优势和不足，从而可以对该升级版本进行改进或者确定该升级版本的灰度配置信息，以扩大该升级版本的发布范围，其中，该灰度配置信息包括允许使用该升级版本的用户的信息，例如，用户的用户账号、地理位置、个人属性，等等，本发明实施例不限于此。
可选地，该升级版本评估日志可以具体为表格，例如表1中示出了原始版本和升级版本中的各种页面对应的页面停留时间的对比结果。这样，管理员可以根据该表1确定该升级版本的首页和付款页面的设计需要进一步改进，但本 发明实施例不限于此。
表1
Figure PCTCN2016080252-appb-000001
再例如，如表2所示，该升级版本评估日志还可以包括原始版本与升级版本中包括的各种服务内容的服务调用次数的对比结果，这样，管理员可以根据表2中示出的对比结果，确定升级版本的质量是否符合预期，但本发明实施例不限于此。
表2
Figure PCTCN2016080252-appb-000002
作为另一个可选实施例，该灰度发布的评估方法200还包括：
确定该升级版本的版本测试参数的值是否满足干预事件触发条件；
若该升级版本的版本测试参数的值满足该干预事件触发条件，触发干预事件。
该干预事件属于主动干预事件，其中，该主动干预事件可以包括一个或多 个事件，该多个事件可以分别由不同的触发条件触发，但本发明实施例不限于此。可选地，该干预事件触发条件可以预先设置，例如，该干预事件触发条件包括：该升级版本的版本测试参数的值与该原始版本的版本测试参数的值的比较结果高于或低于预设门限值。
此时，该升级版本的版本测试参数中包括的至少一个参数的值与该原始版本的版本测试参数中包括的至少一个对应参数的值可以满足预设条件，例如，该升级版本的回访者比例与该原始版本的回访者比例的差或比值低于预设阈值，则可以触发干预事件，但本发明实施例不限于此。
或者，该干预事件触发条件可以包括：该升级版本的版本测试参数的值高于或低于门限值，其中，该门限值是根据该原始版本的版本测试参数的值确定的。
此时，该评估装置可以根据该原始版本的版本测试参数中包括的至少一个参数的值，确定门限值，并确定该升级版本的版本测试参数中包括的至少一个对应参数的值是否超过门限值。例如，该评估装置可以将门限值确定为该原始版本的积极访问者比例的120％，并且确定该升级版本的积极访问者比例是否低于该门限值，并且在该升级版本的积极访问者比例低于该门限值时，触发干预事件，但本发明实施例不限于此。
可选地，该干预事件包括下列中的至少一项：
发送警报消息；
将至少一个用户的路由配置由访问该升级版本更改为访问该原始版本。
具体地，该评估装置可以在确定服务健康度下降或出现故障时，触发警报消息的发送，例如，如果该升级版本的付款服务调用成功率与原始版本相比降低一半以上，则该评估装置可以触发警报消息的发送。具体地，该评估装置可以向邮件服务器发送用于指示该邮件服务器向预设邮箱地址发送警报邮件的指示消息，其中，该预设邮箱地址可以为管理员预先设置的邮箱地址；或者，该评估装置可以向短信网关发送用于指示该短信网关向预设号码发送警报短信的指示消息，其中，该预设号码可以为管理员设置的预先设置的号码。这样，通 过触发警报消息的发送，可以通知管理员服务健康度的下降或故障的出现，从而有利于管理员即时对该升级版本进行修复或调整。
可选地，该评估装置还可以在确定升级版本的登陆页面访问次数降低至该原始版本的一半以下时，触发路由配置的更改，即将当前允许访问该升级版本的至少一个用户的路由配置由访问该升级版本更改为访问该原始版本。具体地，该评估装置可以向灰度发布管理器发送路由配置更改指示，以指示该灰度发布管理器更改至少一个用户的路由配置，但本发明实施例不限于此。这样，通过触发路由配置的更改，能够降低由于升级版本的灰度发布的用户体验较差，而造成用户的流失的风险。
因此，根据本发明实施例的灰度发布的评估方法，通过获取服务的升级版本对应的多个第一会话的会话信息，根据该多个第一会话的会话信息，确定该升级版本的版本测试参数的值，并根据该升级版本的版本测试参数的值，确定该服务的升级版本的评估结果，能够不依赖于用户反馈，对灰度发布的升级版本进行大数据分析，从而获得更为准确的评估结果。
此外，通过确定该服务的原始版本的版本测试参数的值，并且通过比较该升级版本与原始版本的对应版本测试参数的值，确定该升级版本的评估结果，能够为管理员提供更为形象量化的评估结果，通过设置干预事件触发条件并且触发干预事件，有利于管理员即时对该升级版本进行修复或调整，降低由于升级版本的灰度发布的用户体验较差，而造成用户的流失的风险。
应理解，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本发明实施例的实施过程构成任何限定。
上文中结合图4，详细描述了根据本发明实施例的灰度发布的评估方法，下面将结合图5和图6，描述根据本发明实施例的灰度发布的评估装置。
图5示出了本发明实施例提供的灰度发布的评估装置300。该评估装置300包括：
获取单元310，用于获取服务的升级版本对应的多个第一会话的会话信息；
确定单元320，用于根据该获取单元310获取的该多个第一会话的会话信息，确定该升级版本的版本测试参数的值，其中，该版本测试参数包括下列参数中的至少一种：用户行为参数和版本质量参数，并且根据该升级版本的版本测试参数的值，确定该升级版本的评估结果。
可选地，该获取单元310还用于在该确定单元320根据该获取单元310获取的该升级版本的版本测试参数的值，确定该升级版本的评估结果之前，获取该服务的原始版本对应的多个第二会话的会话信息；
该确定单元320还用于根据该获取单元310获取的该多个第二会话的会话信息，确定该原始版本的版本测试参数的值。
此时，该确定单元320用于根据该原始版本的版本测试参数的值和该升级版本的版本测试参数的值，确定该升级版本的评估结果。
作为另一个可选实施例，该评估装置300还包括：
日志生成单元，用于生成升级版本评估日志，其中，该升级版本评估日志包括该确定单元320确定的该原始版本与该升级版本的版本测试参数的值的对比结果。
作为另一个可选实施例，该确定单元320还用于确定该升级版本的版本测试参数的值是否满足干预事件触发条件；
相应地，该评估装置300还包括：
触发单元，用于若该确定单元320确定该升级版本的版本测试参数的值满足该干预事件触发条件，触发干预事件。
作为一个可选实施例，该干预事件包括下列中的至少一项：发送警报消息；将至少一个用户的路由配置由访问该升级版本更改为访问该原始版本。
作为另一个可选实施例，该干预事件触发条件包括：该升级版本的版本测试参数的值与该原始版本的版本测试参数的值的比较结果高于或低于预设门限值；或者
该干预事件触发条件包括：该升级版本的版本测试参数的值高于或低于门限值，其中，该门限值是根据该原始版本的版本测试参数的值确定的。
作为另一个可选实施例，该获取单元310具体用于接收服务器发送的与该升级版本对应的多个第一会话的会话信息，其中，该服务器用于提供该服务的升级版本。
根据本发明实施例的灰度发布的评估装置300可以对应于根据本发明实施例的灰度发布的评估系统100中的第一服务器102，并且该评估装置300中的各个模块的上述和其它操作和/或功能分别为了实现图4中的各个方法的相应流程，为了简洁，在此不再赘述。
因此，根据本发明实施例的灰度发布的评估装置，通过获取服务的升级版本对应的多个第一会话的会话信息，根据该多个第一会话的会话信息，确定该升级版本的版本测试参数的值，并根据该升级版本的版本测试参数的值，确定该服务的升级版本的评估结果，能够不依赖于用户反馈，对灰度发布的升级版本进行大数据分析，从而获得更为准确的评估结果。
此外，通过确定该服务的原始版本的版本测试参数的值，并且通过比较该升级版本与原始版本的对应版本测试参数的值，确定该升级版本的评估结果，能够为管理员提供更为形象量化的评估结果，通过设置干预事件触发条件并且触发干预事件，有利于管理员即时对该升级版本进行修复或调整，降低由于升级版本的灰度发布的用户体验较差，而造成用户的流失的风险。
图6示出了本发明实施例提供的灰度发布的评估装置400。该评估装置400包括：处理器410，用于：
获取服务的升级版本对应的多个第一会话的会话信息；
根据该多个第一会话的会话信息，确定该升级版本的版本测试参数的值，其中，该版本测试参数包括下列参数中的至少一种：用户行为参数和版本质量参数；
根据该升级版本的版本测试参数的值，确定该升级版本的评估结果。
可选地，该处理器410还用于：
在根据该升级版本的版本测试参数的值，确定该升级版本的评估结果之前，获取该服务的原始版本对应的多个第二会话的会话信息；
根据该多个第二会话的会话信息，确定该原始版本的版本测试参数的值。
此时，该处理器410具体用于根据该原始版本的版本测试参数的值和该升级版本的版本测试参数的值，确定该升级版本的评估结果。
作为另一个可选实施例，该处理器410还用于生成升级版本评估日志，其中，该升级版本评估日志包括该原始版本与该升级版本的版本测试参数的值的对比结果。
作为另一个可选实施例，该处理器410还用于：
确定该升级版本的版本测试参数的值是否满足干预事件触发条件；
若该升级版本的版本测试参数的值满足该干预事件触发条件，触发干预事件。
作为一个可选实施例，该干预事件包括下列中的至少一项：发送警报消息；将至少一个用户的路由配置由访问该升级版本更改为访问该原始版本。
作为另一个可选实施例，该干预事件触发条件包括：该升级版本的版本测试参数的值与该原始版本的版本测试参数的值的比较结果高于或低于预设门限值；或者
该干预事件触发条件包括：该升级版本的版本测试参数的值高于或低于门限值，其中，该门限值是根据该原始版本的版本测试参数的值确定的。
作为另一个可选实施例，该处理器410具体用于接收服务器发送的与该升级版本对应的多个第一会话的会话信息，其中，该服务器用于提供该服务的升级版本。
根据本发明实施例的灰度发布的评估装置400可以对应于根据本发明实施例的灰度发布的评估系统100中的第一服务器102，并且该评估装置400中的各个模块的上述和其它操作和/或功能分别为了实现图4中的各个方法的相应流程，为了简洁，在此不再赘述。
应理解，在本发明实施例中，该处理器410可以是中央处理单元(Central Processing Unit，简称为“CPU”)，该处理器410还可以是其他通用处理器、数字信号处理器(DSP)、专用集成电路(ASIC)、现成可编程门阵列(FPGA)或 者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
可选地，该评估装置400还可以包括存储器，该存储器可以包括只读存储器和随机存取存储器，并向处理器410提供指令和数据。存储器的一部分还可以包括非易失性随机存取存储器。例如，存储器还可以存储设备类型的信息。
在实现过程中，上述方法的各步骤可以通过处理器410中的硬件的集成逻辑电路或者软件形式的指令完成。结合本发明实施例所公开的方法的步骤可以直接体现为硬件处理器执行完成，或者用处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器，处理器410读取存储器中的信息，结合其硬件完成上述方法的步骤。为避免重复，这里不再详细描述。
因此，根据本发明实施例的灰度发布的评估装置，通过获取服务的升级版本对应的多个第一会话的会话信息，根据该多个第一会话的会话信息，确定该升级版本的版本测试参数的值，并根据该升级版本的版本测试参数的值，确定该服务的升级版本的评估结果，能够不依赖于用户反馈，对灰度发布的升级版本进行大数据分析，从而获得更为准确的评估结果。
此外，通过确定该服务的原始版本的版本测试参数的值，并且通过比较该升级版本与原始版本的对应版本测试参数的值，确定该升级版本的评估结果，能够为管理员提供更为形象量化的评估结果，通过设置干预事件触发条件并且触发干预事件，有利于管理员即时对该升级版本进行修复或调整，降低由于升级版本的灰度发布的用户体验较差，而造成用户的流失的风险。
本发明实施例还提供了一种灰度发布的评估系统，包括：第一服务器、至少一个第二服务器和至少一个第三服务器，其中，该至少一个第二服务器提供服务的升级版本，该至少一个第三服务器提供该服务的原始版本；该至少一个第二服务器用于向该第一服务器发送与该服务的升级版本对应的多个第一会话的会话信息；该至少一个第三服务器用于向该第一服务器发送与该服务的原始 版本对应的多个第二会话的会话信息；该第一服务器用于根据该多个第一会话的会话信息，确定该升级版本的版本测试参数的值，并且根据该多个第二会话的会话信息，确定该原始版本的版本测试参数的值，其中，该版本测试参数包括下列参数中的至少一种：用户行为参数和版本质量参数；该第一服务器还用于根据该原始版本的版本测试参数的值和该升级版本的版本测试参数的值，确定该升级版本的评估结果。
应理解，在本发明实施例中，术语和/或仅仅是一种描述关联对象的关联关系，表示可以存在三种关系。例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符/，一般表示前后关联对象是一种或的关系。
本领域普通技术人员可以意识到，结合本文中所公开的实施例中描述的各方法步骤和单元，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各实施例的步骤及组成。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。本领域普通技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另外，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口、装置或单元的间接耦合或通信连接，也可以是电的，机械的或其它的形式连接。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本发明实施例方案的目的。
另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以是两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分，或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述灰度发布的评估方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到各种等效的修改或替换，这些修改或替换都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应以权利要求的保护范围为准。

Claims (15)

  1. 一种灰度发布的评估方法，其特征在于，包括：
    获取服务的升级版本对应的多个第一会话的会话信息；
    根据所述多个第一会话的会话信息，确定所述升级版本的版本测试参数的值，其中，所述版本测试参数包括下列参数中的至少一种：用户行为参数和版本质量参数；
    根据所述升级版本的版本测试参数的值，确定所述升级版本的评估结果。
  2. 根据权利要求1所述的评估方法，其特征在于，在所述根据所述升级版本的版本测试参数的值，确定所述升级版本的评估结果之前，所述评估方法还包括：
    获取所述服务的原始版本对应的多个第二会话的会话信息；
    根据所述多个第二会话的会话信息，确定所述原始版本的版本测试参数的值；
    所述根据所述升级版本的版本测试参数的值，确定所述升级版本的评估结果，包括：
    根据所述原始版本的版本测试参数的值和所述升级版本的版本测试参数的值，确定所述升级版本的评估结果。
  3. 根据权利要求2所述的评估方法，其特征在于，所述评估方法还包括：
    生成升级版本评估日志，其中，所述升级版本评估日志包括所述原始版本与所述升级版本的版本测试参数的值的对比结果。
  4. 根据权利要求2或3所述的评估方法，其特征在于，所述评估方法还包括：
    确定所述升级版本的版本测试参数的值是否满足干预事件触发条件；
    若所述升级版本的版本测试参数的值满足所述干预事件触发条件，触发干预事件。
  5. 根据权利要求4所述灰度发布的评估方法，其特征在于，所述干预事件 包括下列中的至少一项：
    发送警报消息；
    将至少一个用户的路由配置由访问所述升级版本更改为访问所述原始版本。
  6. 根据权利要求4或5所述的评估方法，其特征在于，所述干预事件触发条件包括：所述升级版本的版本测试参数的值与所述原始版本的版本测试参数的值的比较结果高于或低于预设门限值；或者
    所述干预事件触发条件包括：所述升级版本的版本测试参数的值高于或低于门限值，其中，所述门限值是根据所述原始版本的版本测试参数的值确定的。
  7. 根据权利要求1至6中任一项所述的评估方法，其特征在于，所述获取服务的升级版本对应的多个第一会话的会话信息，包括：
    接收服务器发送的与所述升级版本对应的多个第一会话的会话信息，其中，所述服务器用于提供所述服务的升级版本。
  8. 一种灰度发布的评估装置，其特征在于，包括：
    获取单元，用于获取服务的升级版本对应的多个第一会话的会话信息；
    确定单元，用于根据所述获取单元获取的所述多个第一会话的会话信息，确定所述升级版本的版本测试参数的值，其中，所述版本测试参数包括下列参数中的至少一种：用户行为参数和版本质量参数，并且根据所述升级版本的版本测试参数的值，确定所述升级版本的评估结果。
  9. 根据权利要求8所述的评估装置，其特征在于，所述获取单元还用于在所述确定单元根据所述获取单元获取的所述升级版本的版本测试参数的值，确定所述升级版本的评估结果之前，获取所述服务的原始版本对应的多个第二会话的会话信息；
    所述确定单元还用于根据所述获取单元获取的所述多个第二会话的会话信息，确定所述原始版本的版本测试参数的值；
    所述确定单元用于根据所述原始版本的版本测试参数的值和所述升级版本的版本测试参数的值，确定所述升级版本的评估结果。
  10. 根据权利要求9所述的评估装置，其特征在于，所述评估装置还包括：
    日志生成单元，用于生成升级版本评估日志，其中，所述升级版本评估日志包括所述确定单元确定的所述原始版本与所述升级版本的版本测试参数的值的对比结果。
  11. 根据权利要求9或10所述的评估装置，其特征在于，所述确定单元还用于确定所述升级版本的版本测试参数的值是否满足干预事件触发条件；
    所述评估装置还包括：
    触发单元，用于若所述确定单元确定所述升级版本的版本测试参数的值满足所述干预事件触发条件，触发干预事件。
  12. 根据权利要求11所述灰度发布的评估装置，其特征在于，所述干预事件包括下列中的至少一项：
    发送警报消息；
    将至少一个用户的路由配置由访问所述升级版本更改为访问所述原始版本。
  13. 根据权利要求11或12所述的评估装置，其特征在于，所述干预事件触发条件包括：所述升级版本的版本测试参数的值与所述原始版本的版本测试参数的值的比较结果高于或低于预设门限值；或者
    所述干预事件触发条件包括：所述升级版本的版本测试参数的值高于或低于门限值，其中，所述门限值是根据所述原始版本的版本测试参数的值确定的。
  14. 根据权利要求8至13中任一项所述的评估装置，其特征在于，所述获取单元具体用于接收服务器发送的与所述升级版本对应的多个第一会话的会话信息，其中，所述服务器用于提供所述服务的升级版本。
  15. 一种灰度发布的评估系统，其特征在于，包括：第一服务器、至少一个第二服务器和至少一个第三服务器，其中，所述至少一个第二服务器提供服务的升级版本，所述至少一个第三服务器提供所述服务的原始版本；
    所述至少一个第二服务器用于向所述第一服务器发送与所述服务的升级版本对应的多个第一会话的会话信息；
    所述至少一个第三服务器用于向所述第一服务器发送与所述服务的原始版本对应的多个第二会话的会话信息；
    所述第一服务器用于根据所述多个第一会话的会话信息，确定所述升级版本的版本测试参数的值，并且根据所述多个第二会话的会话信息，确定所述原始版本的版本测试参数的值，其中，所述版本测试参数包括下列参数中的至少一种：用户行为参数和版本质量参数；
    所述第一服务器还用于根据所述原始版本的版本测试参数的值和所述升级版本的版本测试参数的值，确定所述升级版本的评估结果。
PCT/CN2016/080252 2015-06-01 2016-04-26 灰度发布的评估方法、装置和系统 WO2016192488A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510293522.X 2015-06-01
CN201510293522.XA CN104993946B (zh) 2015-06-01 2015-06-01 灰度发布的评估方法、装置和系统

Publications (1)

Publication Number Publication Date
WO2016192488A1 true WO2016192488A1 (zh) 2016-12-08

Family

ID=54305702

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/080252 WO2016192488A1 (zh) 2015-06-01 2016-04-26 灰度发布的评估方法、装置和系统

Country Status (2)

Country Link
CN (1) CN104993946B (zh)
WO (1) WO2016192488A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112636954A (zh) * 2020-12-11 2021-04-09 车主邦（北京）科技有限公司 一种服务器升级方法及装置

Families Citing this family (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104993946B (zh) * 2015-06-01 2019-04-05 华为技术有限公司 灰度发布的评估方法、装置和系统
CN106708787B (zh) * 2015-11-13 2019-09-13 阿里巴巴集团控股有限公司 业务功能模块正确性的确定方法及装置
CN105897457A (zh) * 2015-12-09 2016-08-24 乐视云计算有限公司 服务器群组的服务升级方法及系统
CN105955761A (zh) * 2016-06-30 2016-09-21 乐视控股（北京）有限公司 基于docker的灰度发布装置及方法
CN106789202A (zh) * 2016-12-08 2017-05-31 努比亚技术有限公司 业务管理装置及方法
CN106933618B (zh) 2017-01-25 2020-03-27 上海蔚来汽车有限公司 基于系统参数相关系数的系统升级评估方法
CN108989267A (zh) * 2017-05-31 2018-12-11 中兴通讯股份有限公司 基于sip的灰度发布方法、系统、设备和存储介质
CN107341008A (zh) * 2017-06-22 2017-11-10 上海艾融软件股份有限公司 微服务灰度发布方法
CN108093075A (zh) * 2017-12-29 2018-05-29 浪潮软件股份有限公司 一种应用系统灰度发布的实现方法
CN108156038B (zh) * 2017-12-29 2019-07-02 Oppo广东移动通信有限公司 请求分发方法、装置、接入网关及存储介质
CN108768875B (zh) * 2018-05-31 2023-04-07 康键信息技术（深圳）有限公司 应用的灰度发布方法、装置及计算机可读存储介质
CN109347695A (zh) * 2018-09-18 2019-02-15 上海移远通信技术股份有限公司 一种升级测试系统及方法
CN109787830B (zh) * 2019-01-21 2023-03-07 北京京东尚科信息技术有限公司 灰度发布控制方法、装置、系统、电子设备及存储介质
CN109871232A (zh) * 2019-02-11 2019-06-11 南威软件股份有限公司 一种APP Android新版本灰度发布的方法
CN110543317A (zh) * 2019-08-29 2019-12-06 上海艾融软件股份有限公司 交易请求处理方法、装置、网关及存储介质
CN111309614B (zh) * 2020-02-17 2022-10-18 支付宝(杭州)信息技术有限公司 A/b测试方法、装置及电子设备
CN112527611A (zh) * 2020-09-24 2021-03-19 上海趣蕴网络科技有限公司 一种产品健康度评估方法及系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7543282B2 (en) * 2006-03-24 2009-06-02 Sun Microsystems, Inc. Method and apparatus for selectively executing different executable code versions which are optimized in different ways
CN103324566A (zh) * 2012-03-20 2013-09-25 阿里巴巴集团控股有限公司 一种网页产品多版本测试方法及装置
CN103455938A (zh) * 2013-09-03 2013-12-18 小米科技有限责任公司 一种数据处理方法、装置及服务器设备
CN104993946A (zh) * 2015-06-01 2015-10-21 华为技术有限公司 灰度发布的评估方法、装置和系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7543282B2 (en) * 2006-03-24 2009-06-02 Sun Microsystems, Inc. Method and apparatus for selectively executing different executable code versions which are optimized in different ways
CN103324566A (zh) * 2012-03-20 2013-09-25 阿里巴巴集团控股有限公司 一种网页产品多版本测试方法及装置
CN103455938A (zh) * 2013-09-03 2013-12-18 小米科技有限责任公司 一种数据处理方法、装置及服务器设备
CN104993946A (zh) * 2015-06-01 2015-10-21 华为技术有限公司 灰度发布的评估方法、装置和系统

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112636954A (zh) * 2020-12-11 2021-04-09 车主邦（北京）科技有限公司 一种服务器升级方法及装置
CN112636954B (zh) * 2020-12-11 2023-10-31 车主邦（北京）科技有限公司 一种服务器升级方法及装置

Also Published As

Publication number Publication date
CN104993946B (zh) 2019-04-05
CN104993946A (zh) 2015-10-21

Similar Documents

Publication Publication Date Title
WO2016192488A1 (zh) 灰度发布的评估方法、装置和系统
JP6220440B2 (ja) モバイル・デバイスのためのキャリアの検出
US8261362B2 (en) Online privacy management
JP5686815B2 (ja) 事前設定条件に基づくウェブサイトユーザのためのデータ更新
US11184223B2 (en) Implementation of compliance settings by a mobile device for compliance with a configuration scenario
CN108073506B (zh) 测试方法和装置
US9619773B2 (en) Systems and methods for code instrumentation for analytics
CN111522711B (zh) 一种数据监控处理系统、方法、执行端、监控端及电子设备
CN109688280A (zh) 请求处理方法、请求处理设备、浏览器及存储介质
US9654580B2 (en) Proxy-based web application monitoring through script instrumentation
US20200034882A1 (en) Systems, methods, and apparatuses for maintaining data granularity while performing dynamic group level multi-variate testing in a group-based communication system
US11062350B2 (en) Method, apparatus, and device for monitoring promotion status data, and non-volatile computer storage medium
US20170346909A1 (en) Client-side bottleneck analysis using real user monitoring data
US20170250890A1 (en) Asynchronous transactions reported as critical path
US20220147431A1 (en) Distributed website load testing system running on mobile devices
CN106598821B (zh) 网页加载速度的测试方法及装置
US10432490B2 (en) Monitoring single content page application transitions
CN111107118A (zh) 一种图片访问加速方法、装置、设备、系统及存储介质
US9537807B2 (en) Automatically transitioning a user from a call to action to an enrollment interface
US20080162687A1 (en) Data acquisition system and method
CN117040799A (zh) 页面拦截规则生成、页面访问控制方法、装置及电子设备
CN103634348A (zh) 终端设备以及发布信息的方法
US20230053592A1 (en) Performance profiler of third party automations
CN105491064A (zh) 一种推送方法和设备
CN107317723B (zh) 一种数据处理方法及服务器

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16802411

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16802411

Country of ref document: EP

Kind code of ref document: A1