WO2023134312A1

WO2023134312A1 - Content charging test method, management device, terminal device and storage medium

Info

Publication number: WO2023134312A1
Application number: PCT/CN2022/134370
Authority: WO
Inventors: 居彦超; 郑军; 张学良
Original assignee: 中兴通讯股份有限公司
Priority date: 2022-01-14
Filing date: 2022-11-25
Publication date: 2023-07-20
Also published as: CN114071391A; CN114071391B

Abstract

Disclosed in the present application are a content charging test method, a management device, a terminal device and a storage medium, which are applied to a content charging test management device. The method comprises: acquiring work order information, the work order information comprising a charging rule and a first service code (S610); generating a test case according to the charging rule (S620); sending the test case to a terminal device, so that the terminal device performs a charging test according to the test case (S630); acquiring test signaling generated by a network element management system in the charging test process for the terminal device (S640); and generating a charging test result according to the test signaling and the first service code (S650).

Description

Content billing test method, management equipment, terminal equipment and storage medium

Cross References to Related Applications

This application is based on a Chinese patent application with application number 202210042819.9 and a filing date of January 14, 2022, and claims the priority of this Chinese patent application. The entire content of this Chinese patent application is hereby incorporated by reference into this application.

technical field

The present application relates to the technical field of communications, and in particular to a content charging test method, management equipment, terminal equipment and storage media.

Background technique

Content billing means that the operator performs billing in combination with different information about user access services and rules configured by core network elements. Operators manage user access rights and charge fees according to the different services accessed by mobile phone users. If the billing rules are incorrectly configured, it is likely to cause errors in the operator's business, and even cause disputes with users. Therefore, the accuracy of content billing rule matching is of great significance. With the continuous development of mobile phone services and core network services, the content billing strategy is becoming more and more complex, and the number of services and rules is also increasing. It is often necessary to upgrade or adjust the billing strategy. Billing rules and policies are often manually configured by operation and maintenance personnel, which is difficult to maintain and may lead to configuration errors. Therefore, after upgrading and adjusting the billing policy, engineers need to conduct tests to ensure the normal operation of the business.

In the related technology, rely on manual use of mobile phone terminals for dial testing, and then manually compare the bills generated by the core network; or use software simulation to simulate access to the core network for testing. The test time used by the manual test method is long, it is difficult to cover a large number of billing rules, and the work efficiency is low; the simulation method does not use the real business process, and cannot simulate the content billing generated when users use real mobile phones to access data services , it is easy to produce test errors, and cannot adapt to unused rules, the flexibility of use is poor, and the reliability of test results is low. Therefore, how to conduct a more efficient, accurate and reliable test for content billing is an urgent problem to be solved.

Contents of the invention

The following is an overview of the topics described in detail in this article. This summary is not intended to limit the scope of the claims.

The embodiment of the present application provides a content charging test method.

In the first aspect, the embodiment of the present application provides a content charging test method, which is applied to a content charging test management device, and the content charging test method includes: obtaining work order information, and the work order information includes charging rules and the first service code; generate a test case according to the charging rule; send the test case to the terminal device, so that the terminal device performs a charging test according to the test case; obtain the network element management system for the A test signaling generated by the terminal device during the charging test; generating a charging test result according to the test signaling and the first service code.

In the second aspect, the embodiment of the present application also provides a content charging test method, which is applied to a terminal device. The content charging test method includes: obtaining a test case from a content charging test management device, and the test case is determined by the The content charging test management device is generated according to the charging rules in the work order information; and the charging test is performed according to the test cases.

In the third aspect, the embodiment of the present application also provides a content billing test management device, including: a memory, a processor, and a computer program stored on the memory and operable on the processor, and the processor executes the computer program. The program implements the content charging test method as described in the first aspect above.

In the fourth aspect, the embodiment of the present application also provides a terminal device, including: a memory, a processor, and a computer program stored on the memory and operable on the processor, and the processor implements the above when executing the computer program. The content charging test method described in the second aspect.

In a fifth aspect, the embodiment of the present application further provides a storage medium storing computer-executable instructions, and the computer-executable instructions are used to execute the content charging test method as described above.

Other features and advantages of the technical solution of the present application will be set forth in the following description, and partly become obvious from the description, or can be understood by implementing the technical solution of the present application. The objectives and other advantages of the application will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Description of drawings

The accompanying drawings are used to provide a further understanding of the technical solution of the present application, and constitute a part of the specification, and are used together with the embodiments of the present application to explain the technical solution of the present application, and do not constitute a limitation to the technical solution of the present application.

Fig. 1 is a network architecture diagram for implementing a content charging test method provided by an embodiment of the present application;

FIG. 2 is a software product architecture diagram for implementing a content charging test provided by an embodiment of the present application;

FIG. 3 is a schematic diagram of functional modules of a content charging test management device provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of functional modules of a terminal device provided by an embodiment of the present application;

FIG. 5 is a schematic diagram of a test flow when no test proxy device is configured according to an embodiment of the present application;

FIG. 6 is a schematic flowchart of a content charging test method applied to a content charging test management device provided by an embodiment of the present application;

FIG. 7 is a schematic diagram of a signaling comparison analysis process provided by an embodiment of the present application;

FIG. 8 is a schematic flowchart of a specific method of step S620 in FIG. 6 provided by an embodiment of the present application;

FIG. 9 is a schematic flow diagram of use case generation in the content charging test management device provided by an embodiment of the present application;

FIG. 10 is a schematic diagram of a test script generation rule provided by an embodiment of the present application;

FIG. 11 is a schematic flowchart of a content charging test method applied to a terminal device provided by an embodiment of the present application;

FIG. 12 is a schematic flowchart of a specific method of step S1120 in FIG. 11 provided by an embodiment of the present application;

13 is a schematic diagram of the processing flow of the mobile phone execution test case provided by an embodiment of the present application;

Fig. 14 is the home scene interface of iTest-Cloud provided by an embodiment of the present application;

Fig. 15 is the sub-scene interface of iTest-Cloud provided by an embodiment of the present application;

Fig. 16 is the APP interface of the mobile terminal provided by an embodiment of the present application;

FIG. 17 is a content charging test management device provided by an embodiment of the present application;

FIG. 18 is a terminal device provided by an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the technical solution of the present application, and are not intended to limit the technical solution of the present application.

It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than in the flowchart. The terms "first", "second" and the like in the specification and claims and the above drawings are used to distinguish similar objects, and not necessarily used to describe a specific sequence or sequence.

It should be noted that in the description of this application, several means one or more, and multiple means two or more. Greater than, less than, exceeding, etc. are understood as excluding the original number, and above, below, within, etc. are understood as Include this number.

The present application provides a content charging test method, management equipment, terminal equipment, and storage media. First, the content charging test management equipment is used to obtain work order information, wherein the work order information includes the first business code of the charging rule; according to The billing rules automatically generate test cases, and send the test cases to the terminal equipment, so that the terminal equipment performs billing test processing according to the test cases after receiving the test cases. The content charging test management device obtains the test signaling generated during the tracking charging test process of the network element management system, generates the charging test result according to the test signaling and the first service code, and obtains the updated charging rule configuration.

This application is used to verify the correctness of the configuration of core network content charging rules, especially the configuration of the 5G core network. Before the operator's core network goes online, or after major operations such as expansion, upgrade, and data configuration changes on the core network, use the content charging test management device and terminal equipment to cooperate with each other to configure the content charging rules of the core network Carry out automated testing to improve the accuracy and efficiency of testing for billing rules.

Embodiments of the present application will be described below in conjunction with the accompanying drawings.

Referring to FIG. 1 , FIG. 1 is a network architecture diagram for implementing a content charging test method provided by an embodiment of the present application. The network architecture diagram shown in Figure 1 includes five parts, namely the terminal device part, the content billing test management device part, the test proxy device part, the network element management system part and the 5G operator network part.

Referring to Figure 1, iTest-APP is deployed on the terminal equipment part. iTest-APP is a mobile phone APP software used to test content charging rules and supports the Android system. Part of the content billing test management equipment is deployed with iTest-Cloud, which is a cloud system for testing content billing rules and supports Linux and Windows operating systems. iTest-Station is deployed in part of the test proxy equipment, and iTest-Station is a station-side system software for testing content charging rules. The terminal equipment, the content billing test management equipment and the test proxy equipment are all connected to the Internet, can communicate and connect, can transmit and exchange receipts with each other, and cooperate with each other to complete the content billing test for the 5G core network.

It should be noted that the content billing test management device can be a server or a virtual machine, and the test proxy device can be a notebook computer or a traditional desktop computer. There are no specific restrictions on the management device and the test proxy device, as long as they can cooperate with each other to realize the function of billing test management. The terminal device can be a mobile phone or other electronic devices capable of running iTest-APP.

EMS (Element Management System, Network Element Management System) is a system that manages one or more telecommunications NE (Network Element, Network Element) of a specific type. EMS is the network management system of the core network. It has functions such as accounting, topology management, and system monitoring, and can track the signaling data of Internet access services on user terminal equipment. The test proxy device is connected to the EMS management plane network where the tested 5G core network is located. The test proxy device deployed with iTest-Station can automatically obtain the signaling tracking data of the Internet service of the user terminal equipment from the EMS, and upload the signaling tracking data to the content charging test management device, so that the content charging test management The device performs data comparison and analysis to obtain the configuration of charging rules. In addition, after the iTest-APP of the terminal device (that is, the APP for dialing test) executes the test script, the engineering personnel can manually download the signaling tracking data from the EMS and import it to the iTest-Cloud to complete the signaling comparison Analyze the charging rule configuration.

Referring to Figure 1, the 5G operator network includes but is not limited to: (wireless) access network ((radio) Access Network, (R) AN), user plane function (User Plane Function, UPF) network elements, access and mobility Management function (Access and Mobility Management Function, AMF) network element, session management function (Session Management Function, SMF) network element and charging function network element (Charging Function, HCF). In the above-mentioned 5G operator network, the part other than the (wireless) access network is called the 5G core network.

The 5G network structure used in this application uses a representation based on reference points to represent the interaction between network functions. There are reference points based on service interface and reference points based on traditional point-to-point communication.

In FIG. 1 , interfaces such as N1 , N2 , N3 , N6 , N11 , and N40 are interfaces provided by the carrier network and have interface serial numbers. The functions of these interfaces and the specific meanings of the serial numbers of the interfaces can refer to the definition and interpretation in the 3GPP standard protocol, and will not be repeated here. In order to facilitate the understanding of the content of the present application, some functional network elements involved in the network architecture of the present application are explained below.

The access network (Access Network, AN) is a sub-network of the operator's network and an implementation system between service nodes and terminal devices in the operator's network. (Wireless) Access Network ((radio) Access Network, (R) AN) is a kind of AN equipment, which can provide wireless communication functions for terminal equipment. (R) AN refers to all or part of fixed users accessing the switch in a wireless manner.

It should be noted that the radio access network is the ground infrastructure, and the access network equipment includes but not limited to: next-generation base stations (gNodeB, gNB), radio network controllers, base station controllers, and mobile switching centers in 5G, etc. wait.

The data network is a network outside the operator's network. The operator's network can access multiple DNs, and various services can be deployed on the DNs, which can provide data and/or voice services for terminal devices.

The user plane function network element is a gateway provided by the operator. It is the gateway for communication between the operator network and the DN. It can implement packet routing and forwarding, policy implementation, service usage reporting, uplink packet detection, downlink data packet storage and other user plane related functions. .

The access and mobility management function network element is a control plane network element provided by the operator's network. It has access and mobility management functions and can complete the access and mobility management of terminal equipment accessing the operator's network. For example, it includes functions such as mobile status management, allocation of user temporary identities, access authentication and access authorization.

The session management function network element is responsible for managing the protocol data unit (Protocol Data Unit, PDU) session of the terminal equipment, that is, processing the user's business. The SMF network element is responsible for establishing, maintaining and deleting PDU sessions, and can also manage sessions, such as session establishment, modification and release. In addition, SMF network elements also have session-related functions such as selecting and controlling UPF network elements, and forwarding session information to an appropriate destination network.

The CHF network element has a charging function and supports online charging, offline charging and converged charging. Among them, converged charging is a new charging mode in the 5G specification, which can integrate online charging and offline charging. In this mode, in both online and offline modes, network elements transmit charging messages with CHF network elements through the Nchf service interface.

In this network architecture, user terminal equipment accesses the core network through the base station (R)AN to perform various services. Each network element in the 5G core network, such as the SMF network element and the short message network element, reports the billing events involved in the user's Internet access to the CHF network element through the service interface Nchf. The CHF network element is responsible for completing the billing and quota authorization, and generates The bill file is used for accounting processing in the accounting domain. The network element management system tracks the signaling data of the terminal equipment's online service and transmits it to the test agent equipment. The test proxy device automatically transmits the signaling trace data to the content charging test management device. The content billing test management equipment conducts data comparison and analysis, obtains the configuration of billing rules, and completes the billing rule test for the 5G core network.

Referring to FIG. 2 , it is an architecture diagram of a software product for implementing a content charging test provided by an embodiment of the present application. In the example in Figure 2, the software product architecture diagram includes the iTest-APP at the terminal device side, the iTest-Cloud at the billing test management side, the iTest-Station at the test proxy device side, and the network element management system. iTest-APP, iTest-Cloud and iTest-Station are all connected to the Internet and can communicate with each other. In one example, the LAN IP of the terminal device is 192.168.101.x, the LAN IP of the test proxy device is 192.168.51.x, the management plane IP is 192.168.2.x, and the public network IP of the billing test management side is 58.213 .91.x. The terminal device can access each other through the billing test management terminal and the test proxy device. The agent test terminal and the network element management system access each other through the management plane to realize data transmission.

It should be noted that the test proxy device is optional, and the software product may or may not be configured with a test proxy device. When the software product has a test proxy device, the test proxy device automatically obtains the signaling tracking data from the core network when the mobile phone executes the test case, and then automatically uploads the data to the billing test management end, and the billing test management end Let the data be compared and analyzed. The automated testing process effectively improves the efficiency of billing testing and reduces the workload of operation and maintenance personnel.

When the test proxy device is not configured in the software product, or the test proxy device fails to work normally, you can manually download the signaling tracking data from the core network after the iTest-APP of the terminal device executes the test script and import it into iTest After -Cloud, perform data comparison and analysis, and the test result analysis report can be exported after iTest-Cloud analysis. Therefore, the present application does not impose specific restrictions on whether a test proxy device is configured in the software product.

Referring to FIG. 3 , FIG. 3 is a schematic diagram of functional modules of a content charging test management device provided by an embodiment of the present application. The content billing test management device deploys iTest-Cloud. The main functional modules of iTest-Cloud include:

The work order importing module is used to obtain the operator's content billing work order, process and analyze the content billing work order, and obtain work order information, the work order information includes: charging rules and first service codes. The content billing work order of the operator is generally in Excel format, and the form filling of the content billing work order must meet the specified format. In addition to the billing rule and the first service code, the work order information includes main information such as the service name.

The use case processing module is used to obtain billing rules from the work order import module, and generate test cases according to the billing rules, and the test cases include test scripts and test rules. In some embodiments, the use case processing module includes an experience library module, a URL library module, a rule transformation module and a use case generation module. Among them, the experience library module, the URL library module and the rule conversion module can all be used to obtain the billing rule and the first network address corresponding to the billing rule from the work order import module, and generate a matching available address according to the address type of the first network address. Accessed test address.

The experience library module is used to obtain the first network address corresponding to the billing rule from the work order import module, and determine the matching second network address as the test address according to the first network address, or to store the billing rule for the successful test The first network address of . URL addresses that pass the billing rule test will be automatically stored in the experience database. One charging rule corresponds to one network address, and this network address has been tested and accessed successfully. After learning, the experience database can have a large number of network addresses with successful test access. When the content billing work order is imported, it will automatically match the data in the experience database to improve the test success rate and test efficiency.

The URL library module is configured to import the first network address corresponding to the billing rule from the work order, and determine a matching third network address as the test address according to the first network address. The URL library can match billing rules to accessible URL addresses through the big data system and domain name crawler system and store them in the URL library. Billing rules can be matched from the URL library to precise URL addresses for testing, which also improves Test success rate and test efficiency.

The rule conversion module is used to obtain the first network address corresponding to the charging rule from the work order import module, and determine the fourth network address as the test address according to the test script generation rule and the first network address. In the actual test, there will be a test address that cannot be obtained from the experience library module and the URL library module that matches the first network address. At this time, the rule conversion module can be used to convert each rule in the content billing work order into Accessible test address. Effectively guarantee the normal progress and success rate of the billing test.

The use case generation module is used to obtain the test address from the experience library module or the URL library module or the rule conversion module, and generate a test case according to the test address, and the test case includes a test script and a test rule.

The scenario management module is used to obtain test cases from the use case generation module to create test scenarios. The scenario management module can meet multiple testing requirements, and the operation and maintenance personnel can select the appropriate test scenario for testing, improving the credibility and accuracy of the test results.

The task management module is used to obtain test cases from the use case generation module, generate test tasks according to the test cases, and send the test tasks to the terminal devices, so that the terminal devices can perform billing tests according to the test scripts and test rules in the test tasks. After the task management module creates a test task, the task is synchronized to the mobile phone through the API interface of the docking mobile phone, and the tester can execute the task on the iTest-APP of the mobile phone.

The signaling analysis module is configured to obtain the first service code and the test signaling sent by the network element management system from the work order import module. During the charging test of the terminal equipment, the network element management system performs signaling tracking to generate test signaling. The signaling analysis module can generate a charging test result according to the test signaling and the first service code. The signaling analysis module can import test signaling or connect to the station-side API interface from the station side, that is, the test agent device side, to automatically obtain signaling, then perform signaling analysis, and compare the rate group (Rating Group, RG) or service code (Service id, SI) message is the same as the service code of the operator's work order, and judges whether each charging rule is configured correctly.

In addition, iTest-Cloud also has a report export module, which is used to obtain the charging test results from the signaling analysis module, and automatically generate a test report for the content charging test. The export format of the test report is Excel format.

Referring to FIG. 4 , FIG. 4 is a schematic diagram of functional modules of a terminal device provided by an embodiment of the present application. The terminal device deploys iTest-APP, and the main functional modules of iTest-APP include:

The API interface module is connected with the content billing test management device end, and is used for receiving the message from the content billing test management device end, and sending data to the content billing test management device end. The task management module is used to synchronize the tasks of the content billing test management device side, so that engineers can perform test tasks on the terminal device, view task details, etc.; the use case management module is used to manage test cases under the test task, you can choose Or execute several use cases repeatedly, view the execution results of the use cases, etc.

iTest-APP also has a use case execution module, which adopts different use case execution methods according to the IP address or URL address in the test case, and is used for billing the test content. In the use case execution of some embodiments, two modules of sending GET request and getting GET response are used. In some embodiments, the use case execution module can obtain the test task, and according to the test rules in the test task, adopt different corresponding use case execution methods to perform the test task; send the GET request module to communicate with the use case execution module for constructing Use case execution request information, and send the use case execution request information to the core network; obtain the GET response module, communicate with the use case execution module, and receive the use case execution response information returned by the core network, and the use case execution response information responds to the use case execution request The information is used to confirm that the core network has tested the network address corresponding to the charging rule.

The network architecture and application scenarios described in the embodiments of the present application are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute limitations on the technical solutions provided by the embodiments of the present application. Those skilled in the art know that with the network architecture The evolution of the technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

Those skilled in the art can understand that the network architecture shown in Figure 1 does not constitute a limitation to the embodiment of the present application, and may include more or less components than those shown in the illustration, or combine certain components, or different components layout.

Based on the architectures and functional modules in FIGS. 1 to 4 above, various embodiments of the content charging testing method of the present application are proposed below.

For example, in the following example, when the test proxy device is not configured in the software product, or the test proxy device fails to work normally, you can execute the test script after the iTest-APP (that is, the dial-up test APP) of the terminal device , engineers can manually download signaling trace data from the core network, import it into iTest-Cloud, and make iTest-Cloud perform data comparison and analysis to obtain test results.

Referring to FIG. 5 , FIG. 5 is a schematic diagram of a test flow provided by an embodiment of the present application when no test proxy device is configured. The operator's work order processing end screens out valid URL addresses to generate a work order, and imports the work order to the content billing test management device side. After the content billing test management device receives the work order imported by the operator's work order processing end, it generates a test case for the billing test. It should be noted that there may be errors or deviations when generating the test case, so the generated The test case can be manually modified to make it more in line with the test requirements. After the test case is generated, the content accounting test management device creates a scenario package according to the test case, and then creates a test task according to the scenario package. After the test task is created, the content charging test management device sends the test task to the mobile terminal. Both the content billing test management device and the mobile terminal can manage test tasks and view task details. After receiving the test task sent by the content charging test management device, the mobile terminal executes the test case in the test task. When the mobile terminal executes the test task, the EMS tracks the test process, and generates a signaling trace file according to the result of the test task performed by the mobile terminal. Engineers manually export the signaling file with the interface name N40, and import the signaling file to the cloud. After the content billing test management device receives the imported signaling file, it parses the signaling file, compares the SI information in the work order, and draws an analysis report on the configuration of billing rules, displaying the configuration results. So far, the charging test under the condition that the test proxy device is not configured or fails is completed.

It should be noted that the content charging test management device side creates a scenario package according to the test case, and creates a corresponding test task according to the scenario package. The content billing test management device side can generate a test task carrying a test case when the scenario package is selected, and send the test task to the terminal device, so that the terminal device executes the test case in the test task; the content billing test management device The terminal can also directly send the test case to the terminal device without creating a scenario package and a test task, so that the terminal device executes the test case to complete the billing test. This application does not impose specific restrictions on the specific scenarios and methods in which the content charging test management device sends the test case to the terminal device, as long as the terminal device can execute the test case to complete the charging test.

Testing billing rules can improve network quality and customer satisfaction to a certain extent, and effectively reduce complaints about billing errors. This application can conveniently test the configuration state of the content charging rule or the situation of no configuration. Avoiding call charge disputes caused by incorrect billing rules can bring good economic and social benefits to operators and improve user experience.

Referring to FIG. 6 , FIG. 6 is a schematic flowchart of a content charging test method applied to a content charging test management device provided by an embodiment of the present application. The content charging test method may include but not limited to step S610, step S620, step S630, step S640 and step S650.

Step S610: Obtain work order information, where the work order information includes charging rules and a first service code.

In this step, the work order information is obtained from the operator. It can be understood that the work order information includes but is not limited to charging rules, business codes, and also includes main information such as business names and billing unit prices. In order to facilitate processing, the work order information is in Excel format, and the main information in the work order is filled in the specified format.

Step S620: Generate a test case according to the charging rule.

In some embodiments, in this step, the content charging test management device first obtains an accessible test address according to the charging rules, secondly obtains a test script according to the test address, and finally generates a test case according to the test script and preset test rules. The preset test rule is a preset charging rule to be tested. Generated test cases have test scripts and test rules.

It should be noted that the content charging test management device generates a test case for each charging rule, and the test case includes a test script and a test rule. It can be understood that there are many billing rules to be tested, and after obtaining the test case for each billing rule, an automated test case set is automatically created.

Step S630: Send the test case to the terminal device, so that the terminal device performs a charging test according to the test case.

In this step, the content charging test management device sends the test case to the terminal device. After receiving the test case, the terminal device performs the charging test according to the test case. It should be noted that, before sending the test case to the terminal device, the engineer can also select the corresponding test scenario package on the content charging test management device side according to the test scenario. The content billing test management device selects the required test case according to the selected test scenario, creates a test task, and sends the test task to the terminal device for distribution to testers. The terminal device starts to perform the billing test on the test script and test rules in the test task according to the tester's specific instructions. The test scene is close to the real scene, which can improve the credibility and reliability of the test results. This application does not impose specific restrictions on whether to select a test scenario at the content charging test management device.

Step S640: Obtain the test signaling generated by the network element management system during the charging test for the terminal equipment.

In this step, the content charging test management device uses the EMS connected to the 5G core network, and enables signaling tracking for the test mobile phone through the EMS, so as to obtain the test generated by the network element management system during the charging test of the terminal device signaling.

Step S650: Generate a charging test result according to the test signaling and the first service code.

In this step, the content charging test management device obtains the charging session response information according to the test signaling, and obtains the second service code from the charging session response information. Comparing the second business code with the first business code, if the second business code is the same as the first business code, the billing test result indicates that the charging rule configuration is correct; if the second business code is different from the first business code In the case of , the accounting test result shows that the accounting rule configuration is incorrect.

Referring to FIG. 7 , FIG. 7 is a schematic flow chart of signaling comparison analysis provided by an embodiment of the present application. The content charging test management device receives the EMS signaling file sent by the EMS during the execution of the use case, and first performs the signaling search process to obtain the detailed data of the signaling. Then, the field comparison process is performed according to the detailed data of the signaling, the test result is obtained, and the test report is finally generated. In the signaling search process, the test management device screens out the signaling with the interface name N40 from the received EMS signaling, and then screens out the information name Nchf_ConvergedCharging_Release_Req from the signaling with the interface name N40 within the execution time of the use case (Release Charging Session Response Information) or Nchf_ConvergedCharging_Update_Req (Update Charging Session Response Information) signaling. Then extract the service code value from the two signaling messages, and the fields in the test signaling are rating group (ratingGroup, RG) and service code (serviceid, SI). The content charging test management device compares whether the service code in the test signaling is the same as the service code in the work order provided by the operator. When the RG and SI in the test signaling have at least any value and the service in the work order When the codes are equal, it means that the test signaling analysis is successful, indicating that the content charging rules are configured correctly; otherwise, it means that the content charging rules are configured incorrectly.

All operators are paying great attention to the digitization of operation and maintenance, including test automation. This application can greatly save labor costs, and can automatically compare the business code of the work order and the N40 port signaling file in the test signaling, replacing the manual test process of content billing. Using the application can realize automatic testing, is easy to operate, effectively improves the efficiency of billing testing, has low requirements on users' technical capabilities, and is easier to popularize.

In one embodiment, referring to FIG. 8, FIG. 8 is a schematic flowchart of a specific method of step S620 in FIG. Step S820.

Step S810: Obtain the first network address corresponding to the charging rule according to the charging rule.

In this step, the first network address may be an IP address or a URL address or a combined address of an IP address and a URL address. It should be noted that different address types in this application correspond to different methods of obtaining test addresses.

Step S820: Determine an accessible test address according to the address type of the first network address.

In this step, when the first network address is a URL address, determine the accessible test address according to the address type of the first network address, including at least the following three situations:

The first case is: if there is a second network address matching the first network address recorded in the experience database, then the second network address is determined as the test address in the test script, and the experience database stores the billing for successful testing. The network address corresponding to the rule and the charging rule. The second situation is: when the second network address is not recorded in the experience database, the third network address matching the first network address is obtained from the URL database as the test address in the test script. The third situation is: when the first network address cannot be obtained from the experience database and the URL database, the fourth network address is determined as the test address according to the preset test script generation rules and the first network address.

And when the first network address is an IP address, or when it is a combined address of an IP address and a URL address, then according to the first network address and a preset test script generation rule, determine the fifth network address corresponding to the first network address as an accessible test address.

Referring to FIG. 9 , FIG. 9 is a schematic flowchart of use case generation in the content charging test management device provided by an embodiment of the present application.

It should be noted that the content charging test management device includes, but is not limited to, the functional modules shown in FIG. 3 , and may also include a use case management module, which is used to manage test cases. As shown in Figure 9, the use case management module first adds a deep packet inspection (Deep Packet Inspection, DPI) use case set. Judging whether the test address of the test case is a seven-layer rule, that is, judging whether the test address is a URL address.

When the test address is a URL address, first check whether there is a matching test address in the experience database. If there is a record of the billing rule in the experience library, use the network address corresponding to the rule recorded in the experience library as the test address for mobile phone access in the test script.

If the test address cannot be matched in the experience library, then it will be matched in the URL library. The data recorded in the URL library is a large number of accurate URLs found in advance through big data or crawler technology. Based on this, iTest-Cloud can search the URL database for an exact address that can be matched by a certain rule, and if there is a match, use the matched address as the test address for mobile phone access in the test script.

If the test case cannot find a matching recorded URL address or IP address in both the experience library and the URL library. Then, according to the corresponding rules, that is, the test script generation rules, processing is performed to obtain the test address accessed by the mobile phone in the test script. After getting the test address, generate a mobile phone script, and finally generate a use case.

It should be noted that the rules corresponding to the network addresses recorded in the experience database are charging rules that have been successfully tested. In some embodiments, the use case management module manages the test cases, the scenario management module uses the test cases in the use case management module to create test scenarios, creates test tasks according to the test scenarios, and sends the test tasks to the mobile phone APP. The mobile APP executes the test task, and iTest-Cloud receives the imported signaling and performs signaling analysis. When the access URL address executes the task successfully and the signaling analysis is successful, the URL address is automatically entered into the experience database. The test method provided in this application can be self-learning, and provides training samples for the realization of the test robot. For content billing rules that have been successfully tested, they are stored in the experience database. As long as the rules in the operator work order are recorded in the experience database, the original rules can be replaced with the accessible domain names in the library to improve the test success rate.

iTest-Cloud can automatically convert the network address of the charging rule into an accessible test address in the rule conversion module for the network address of different charging rules. In some embodiments, refer to FIG. 10 , which is a schematic diagram of test script generation rules provided by an embodiment of the present application. Taking a DPI work order as an example, the processing rules are as follows:

For only the three-layer rule (IP address), if it is a specific address, the address is IPV4, such as 49.4.42.14, then direct access. If it is an IPV6 address, access after removing the 128 mask. If the network address only has an address range, for IPV4, remove the first and last available addresses as the accessible test address; for the IPV6 address, calculate the address range according to the mask, and take the first available address as the accessible test address . For the network address of the designated port and designated address, the test address is obtained by adding the designated port to the address. For the network address of the specified address and port range, take the smallest port in the port range and concatenate the specified address to obtain an accessible test address. When the network address has an address range and a port range, for IPV4, the smallest port in the port range will be spliced to the available addresses in the address range to obtain an accessible test address; for IPV6, the smallest port in the port range will be used The port splicing of the masked address is removed to obtain an accessible test address. Then generate a mobile phone test script for charging rule testing.

For the seven-layer rule (URL address), if there is no wildcard, it will be directly accessed according to the precise address. If there is a wildcard, replace the wildcard of the port with the actual port of 8080, and remove the others. Then generate a mobile phone test script for charging rule testing. For the combined rules of the third layer and the seventh layer, the third layer rule and the seventh layer rule are also processed respectively according to the above rules.

Referring to FIG. 11 , FIG. 11 is a schematic flowchart of a content charging test method applied to a terminal device provided by an embodiment of the present application. The content charging test method may include but not limited to step S1110 and step S1120.

Step S1110: the terminal device acquires a test case from the content charging test management device.

In this step, the terminal device obtains a test case from the content charging test management device, and the test case is generated by the content charging test management device according to the charging rules in the work order information. The terminal device can directly obtain the test case from the content charging test management device, or obtain the test task from the content charging test management device, and then obtain the test case from the test task. When the terminal device acquires a test task from the content charging test management device, the terminal device can automatically synchronize the test task created by the content charging test management device. Testers can log in to the iTest-APP of the terminal device, view the synchronized test tasks, and then execute the test cases in the test tasks. Users can operate on the mobile phone, perform test tasks or view task details, etc.

Step S1120: Perform a billing test according to the test case.

In this step, the terminal device performs a charging test on the charging rule according to the test case.

In one embodiment, referring to FIG. 12, FIG. 12 is a schematic flowchart of a specific method of step S1120 in FIG. 11 provided by an embodiment of the present application, and describes step S1120. This step S1120 may include but not limited to steps S1210, Step S1220, step S1230 and step S1240.

Step S1210: the terminal device acquires test rules according to the test cases.

In this step, the terminal device obtains a test case from the content charging test management device, and the test case includes a test script and a test rule. Wherein, the test rule is a preset charging rule to be tested. Terminal devices can obtain test rules from test cases.

Step S1220: Obtain a test rule to determine a test address.

Step S1230: Determine the corresponding use case execution rule according to the address type of the test address.

In this step, the address type of the test address includes at least one of: IP address and URL address. It can be understood that the address type of the test address may be an IP address, a URL address, or a combined address of an IP address and a URL address. Different address types correspond to different use case execution rules to ensure that test cases can be executed smoothly and improve the success rate of billing tests.

Step S1240: Carry out a billing test according to the use case execution rules.

In this step, it also includes: the terminal device sends the use case execution request information according to the use case execution rules, and then, after receiving the use case execution response information, executes the test script according to the use case execution rules to perform the charging test. The terminal device ensures that all test cases can be successfully tested through the use case test processing flow.

Referring to FIG. 13 , FIG. 13 is a schematic diagram of a processing flow of a mobile phone executing a test case provided by an embodiment of the present application. In some embodiments, charging rules can be divided into three-tier rules, seven-tier rules and combined rules, where the combined rule value is a combination of three-tier rules and seven-tier rules. The mobile terminal starts the test through keywords to determine whether the test rule is a combination rule. When the test rule is a combined rule, establish a TCP link and determine whether it is an HTTPS protocol. When the TCP link is successfully established and the protocol is not HTTPS, the mobile phone constructs a GET message to send, and obtains the GET message to ensure that the network address corresponding to the rule has been tested. When the TCP link is successfully established and the HTTPS protocol is used, the mobile phone sends a ClientHello message with a matching domain name. If the authentication is successful, it sends a GET request and receives a GET response to ensure that the network address corresponding to the rule has been tested. If If the authentication is unsuccessful, the test ends. When the TCP link establishment fails and the HTTPS protocol is used, a ClientHello message is sent to complete the test. The destination address in the message is the layer-3 address in the combination address, that is, the IP address, and servername is the domain name in the combination rule. In the case that the TCP link establishment fails and the protocol is not HTTPS, construct and send a GET message to complete the test. The destination address in the GET message is the three-layer rule in the combination rule, and the Host is the domain name in the combination rule.

When the test rule is not a combination rule, but a three-layer rule (IP address), then establish a link according to the TCP specified port in the combination rule. If the link is established successfully, the billing rule will be triggered, and the mobile terminal will send a GET message to obtain a GET response to ensure that the test reaches the network address corresponding to the test rule, and the test is completed. If the link building fails, the test ends.

When the test rule is not a combination rule, but a seven-layer rule (URL address). For the test rules that require DNS domain name resolution, if the DNS resolution is successful and the link is established successfully, send a GET request directly for HTTP and get the GET response to complete the test; for HTTPS, send a ClientHello message with the matching domain name, and the authentication is successful Then send a GET request to complete the test, and end the test if the authentication is unsuccessful. If the DNS resolution is successful and the link building fails, for HTTP, a GET request is sent directly to complete the test. The destination address in the GET request is the IP address of the link building, and the Host is the domain name of the seventh layer; for HTTPS, it is sent with the matching domain name ClientHello message, the test is completed, the destination address in the ClientHello message is the IP address of the link, and the servername is the domain name of the seventh layer. If the test address resolution fails or does not require DNS resolution, send a GET message directly for HTTP, and send a ClientHello message for HTTPS to complete the test. At this time, the destination address of the GET message and the ClientHello message are the same, and the ClientHello message The servername in it is the domain name of the seventh layer, and the Host in the GET message is the domain name of the seventh layer. If there is a port, the GET message also needs to carry the port.

The test address may not exist or may be virtual. Constructing and sending a GET message can ensure that the network address corresponding to the charging rule is tested, which improves the success rate of the charging test and saves test resources.

Referring to Fig. 14, Fig. 14 is the home scene interface of iTest-Cloud provided by an embodiment of the present application. There is a test scene selection in the scene interface, and the user can select a scene package on the popular scene on the home page, and click Follow.

The scene package that the user pays attention to will be displayed in the sub-page of "My scene package". Refer to Figure 15, which is the sub-scene interface of iTest-Cloud provided by an embodiment of the present application. The user can select the concerned scene package creation task in the sub-scene interface as shown in FIG. 15 . After users pay attention to the scene package, they can create tasks according to the scene package. After the task is created, both the mobile terminal and the iTest cloud will display the task, and the task status on both sides is the same. In the sub-scene interface shown in Figure 15, the user can click to create a task, and then modify the task name in another sub-page. After creating a task, you can see the task you created on the "My Tasks" page.

Referring to FIG. 16 , FIG. 16 is an APP interface of a mobile terminal provided by an embodiment of the present application. After the task is created, the task will be displayed on both the mobile terminal and the iTest cloud, and the status of the tasks on both sides is the same. The mobile APP page is shown in Figure 16. The user can select the DPI test task to be performed on the mobile APP page, click execute, download the use case, click continue, and wait for the mobile phone to complete the DPI test.

After the mobile phone is tested, the user logs in to iTest-Cloud and checks the task. At this time, the task status will be updated from created to completed. Users can log in to the EMS management system to export signaling tracking files, enter the task page, find the tasks they have completed, and import the signaling files generated after the task execution to iTest-Cloud. After the signaling file is successfully imported, use the data analysis function of iTest-Cloud for data analysis. After the analysis is over, the user can click on the task to view the task details, the execution result of each use case, that is, whether each URL address test is successful, and click on the analysis result to view the log, that is, the RG number comparison result. If the mobile APP test URL address test is successful but the RG number does not exist, the analysis results also reflect that the log will show that the RG number does not exist.

The embodiment of the present application provides a content charging test management device and a terminal device to cooperate with each other to complete the charging test. It replaces the manual test process of content billing, automatically compares the service code of the work order with the N40 in the signaling, and obtains the test status of the billing rule. Engineers can grasp the progress of test tasks through APP and iTest-Cloud, and can conveniently check and verify the test status of billing rules, saving labor costs and improving the work efficiency of engineers. Verifying the configuration of billing rules will help improve network quality and customer satisfaction, and effectively reduce similar complaints. In addition, the present application can conveniently test the situation that the content charging rule is not configured. Avoid call charge disputes caused by incorrect billing rules, and bring good economic and social benefits to operators.

FIG. 17 is a content charging test management device 1700 provided by an embodiment of the present application. As shown in Figure 17, the content charging test management device 1700 includes but is not limited to:

The memory 1720 is used to store programs; the processor 1710 is used to execute the programs stored in the memory 1720. When the processor 1710 executes the programs stored in the memory 1720, the processor 1710 is used to execute the above-mentioned content charging test method. The processor 1710 and the memory 1720 may be connected through a bus or in other ways.

FIG. 18 is a terminal device 1800 provided by an embodiment of the present application. As shown in Figure 18, the terminal device 1800 includes but is not limited to: a memory 1820 for storing programs; a processor 1810 for executing the programs stored in the memory 1820, when the processor 1810 executes the programs stored in the memory 1820, the processor Step 1810 is used to execute the above content charging test method. The processor 1810 and the memory 1820 may be connected through a bus or in other ways.

The memory 1720 and the memory 1820, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs, such as the content charging test method described in the embodiment of the present application. The processor 1710 and the processor 1810 respectively execute the non-transitory software programs and instructions stored in the memory 1720 and the memory 1820 to implement the above-mentioned content charging test method.

The memory 1720 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application program required by a function; the data storage area may store and execute the content charging test method described above. In addition, the memory 1720 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some implementations, the memory 1720 may optionally include memory located remotely from the processor 1710, and these remote memories may be connected to the processor 1710 through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The non-transitory software programs and instructions required to realize the above-mentioned content billing test method are stored in the memory 1720, and when executed by one or more processors 1710, the above-mentioned content billing test method is executed, for example, as shown in FIG. Method steps S610 to S650 in 6 and method steps S810 to S820 in FIG. 8 .

In one embodiment, the storage medium stores computer-executable instructions, and the computer-executable instructions are executed by one or more control processors 1710, for example, by one of the processors 1710 in the above-mentioned content charging test management device 1700 , the above-mentioned one or more processors 1710 may be made to execute the above-mentioned content charging test method, for example, execute method steps S610 to S650 in FIG. 6 and method steps S810 to S820 in FIG. 8 .

The non-transitory software programs and instructions required to realize the above-mentioned content billing test method are stored in the memory 1820, and when executed by one or more processors 1810, the above-mentioned content billing test method is executed, for example, the implementation of FIG. 11 The method steps S1110 to S1120 in and the method steps S1210 to S1240 in FIG. 12 .

In one embodiment, the storage medium stores computer-executable instructions, and the computer-executable instructions are executed by one or more control processors 1810, for example, executed by one of the processors 1810 in the terminal device 1800, so that the above-mentioned One or more processors 1810 execute the above-mentioned content charging test method, for example, execute method steps S1110 to S1120 in FIG. 11 and method steps S1210 to S1240 in FIG. 12 .

The embodiment of the present application includes: when it is necessary to test the configuration of the updated content charging rule, the content charging test management device acquires work order information, wherein the work order information includes the charging rule and the first service code. The content charging test management device generates a test case according to the charging rule, and sends the test case to the terminal device. After receiving the test case, the terminal device performs test processing according to the test case. During the charging test process of the terminal equipment, the network element management system tracks the charging test process to generate test signaling and sends it to the content charging test management device. The content charging test management device generates a charging test result according to the test signaling and the first service code, and obtains the updated charging rule configuration. According to the solution of the embodiment of the present application, by using the content billing test management device to receive work order information, and using the content billing test management device to automatically generate test cases and analyze test signaling according to the work order information, the updated Configuration of content charging rules. The solution of the embodiment of the present application can use the content charging test management device and the real terminal device to perform the test task when the content charging rule is updated and the configuration is unknown. The two devices cooperate with each other to test the updated charging Whether the rules are configured correctly can effectively improve the accuracy and test efficiency of content billing tests, provide operation and maintenance engineers with more reliable configuration reference results, and improve the work efficiency of engineering personnel.

The embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

Those skilled in the art can understand that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware and an appropriate combination thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit . Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those of ordinary skill in the art, the term computer storage media includes both volatile and nonvolatile media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. permanent, removable and non-removable media. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cartridges, magnetic tape, magnetic disk storage or other magnetic storage devices, or can Any other medium used to store desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embody computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media .

Claims

A content charging test method applied to a content charging test management device, comprising:

Obtain work order information, where the work order information includes charging rules and a first business code;

Generate test cases according to the billing rules;

Sending the test case to the terminal device, so that the terminal device performs a charging test according to the test case;

Obtaining test signaling generated by the network element management system for the terminal device during the charging test;

Generate a charging test result according to the test signaling and the first service code.
The content charging test method according to claim 1, wherein said generating test cases according to said charging rules comprises:

Obtain an accessible test address according to the billing rules;

Obtain a test script according to the test address;

The test case is generated according to the test script and preset test rules.
The content charging test method according to claim 2, wherein said obtaining an accessible test address according to said charging rule comprises:

Acquire a first network address corresponding to the charging rule according to the charging rule;

The accessible test address is determined according to the address type of the first network address.
The content charging test method according to claim 3, wherein the first network address is a URL address, and determining the accessible test address according to the address type of the first network address includes:

In the case that a second network address matching the first network address is recorded in the experience library, the second network address is determined as the test address in the test script, and the experience library stores the test A successful charging rule and a network address corresponding to the charging rule;

or,

In the case where the second network address is not recorded in the experience database, a third network address matching the first network address is obtained from the URL library as the test address in the test script;

or,

In the case that the first network address cannot be obtained from the experience library and the URL library, a fourth network address is determined as the test address according to a preset test script generation rule and the first network address .
The content billing test method according to claim 3, wherein the first network address is an IP address, or a combined address of an IP address and a URL address, and according to the address type of the first network address, determine Accessible test addresses include:

A fifth network address corresponding to the first network address is determined as the accessible test address according to the first network address and a preset test script generation rule.
The content charging test method according to claim 1, wherein said generating a charging test result according to said test signaling and said first service code comprises:

Acquiring charging session response information according to the test signaling;

obtaining a second service code according to the charging session response information;

In the case where the second service code is the same as the first service code, the charging test result indicates that the charging rule configuration is correct,

or,

In a case where the second service code is different from the first service code, the result of the charging test is that the charging rule is configured incorrectly.
A content charging test method applied to a terminal device, comprising:

Obtaining a test case from the content charging test management device, the test case is generated by the content charging test management device according to the charging rules in the work order information;

Carry out billing test according to the test case.
The content charging test method according to claim 7, wherein said performing the charging test according to the test case includes:

Obtain test rules according to the test case;

Determine the test address according to the test rules;

determining a corresponding use case execution rule according to the address type of the test address;

Carry out billing test according to the use case execution rules.
The content charging test method according to claim 8, wherein, performing the charging test according to the use case execution rules includes:

sending use case execution request information according to the use case execution rules;

In the case of receiving the use case execution response information, perform the charging test according to the use case execution rules.
The content charging test method according to claim 8, wherein the address type of the test address includes at least one of: IP address; URL address.
A content billing test management device, comprising: a memory, a processor, and a computer program stored on the memory and operable on the processor, wherein, when the processor executes the computer program, it implements claims 1 to 6 The content billing test method described in any one.
A terminal device, comprising: a memory, a processor, and a computer program stored on the memory and operable on the processor, wherein, when the processor executes the computer program, any one of claims 7 to 10 is implemented The content billing test method described above.
A storage medium, wherein computer-executable instructions are stored, and the computer-executable instructions are used to execute the content charging test method according to any one of claims 1 to 6, or any one of claims 7 to 10 A content billing test method as described.