CN114071391B

CN114071391B - Content charging test method, management device, terminal device and storage medium

Info

Publication number: CN114071391B
Application number: CN202210042819.9A
Authority: CN
Inventors: 居彦超; 郑军; 张学良
Original assignee: ZTE Corp
Current assignee: ZTE Corp
Priority date: 2022-01-14
Filing date: 2022-01-14
Publication date: 2022-05-17
Anticipated expiration: 2042-01-14
Also published as: CN114071391A; WO2023134312A1

Abstract

The invention discloses a content charging test method, a management device, a terminal device and a storage medium, which are applied to the content charging test management device, and the method comprises the following steps: acquiring work order information, wherein the work order information comprises a charging rule and a first service code; generating a test case according to the charging rule; sending the test case to terminal equipment so that the terminal equipment performs charging test according to the test case; and acquiring a test signaling generated by the network element management system in the charging test process of the terminal equipment, and generating a charging test result according to the test signaling and the first service code. According to the scheme of the embodiment of the invention, under the condition that the updating configuration condition of the content charging rule is unknown, the management equipment and the terminal equipment are matched with each other to perform the test task, the configuration condition of the updated charging rule is tested, and the accuracy and the test efficiency of the content charging test are effectively improved.

Description

Content charging test method, management device, terminal device and storage medium

Technical Field

The present invention relates to the field of communications technologies, and in particular, to a content charging test method, a management device, a terminal device, and a storage medium.

Background

Content charging refers to that an operator charges according to different information of user access service and rules configured by a network element of a core network. The operator manages the user access rights and charges according to different services accessed by the mobile phone user. If the charging rule is configured incorrectly, it is likely to cause the service of the operator to be wrong, and even generate charge dispute with the user, so the accuracy of matching the content charging rule has important significance. With the continuous development of mobile phone services and core network services, content charging policies are increasingly complex, the number of services and rules is also increasing, and charging policies need to be updated or adjusted frequently. The charging rules and policies are often configured manually by operation and maintenance personnel, and the maintenance difficulty is high, which may cause configuration errors. Therefore, after the charging policy is updated and adjusted, the engineer needs to perform a test to ensure normal operation of the service.

In the related technology, a mobile phone terminal is used manually for dialing and testing, and then telephone bills generated by a core network are compared manually; or testing by using software to simulate accessing the core network. The manual testing method has the advantages that the testing time is long, the large number of charging rules are difficult to cover, and the working efficiency is low; the simulation method does not use a real service flow, cannot simulate content charging generated when a user uses a real mobile phone to access data services, is easy to generate test errors, cannot adapt to unused rules, and has poor use flexibility and low reliability of test results. Therefore, how to perform a more efficient, accurate and reliable test on content charging is an urgent problem to be solved.

Disclosure of Invention

The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

The embodiment of the invention provides a content charging test method, which can cover various test scenes under the condition that an operator updates content charging rules, and effectively improves the accuracy and the test efficiency of content charging tests.

In a first aspect, an embodiment of the present invention provides a content charging test method, which is applied to a content charging test management device, where the content charging test method includes:

acquiring work order information, wherein the work order information comprises a charging rule and a first service code;

generating a test case according to the charging rule;

sending the test case to terminal equipment so that the terminal equipment performs charging test according to the test case;

acquiring a test signaling generated by a network element management system to the terminal equipment in the charging test process;

and generating a charging test result according to the test signaling and the first service code.

In a second aspect, an embodiment of the present invention further provides a content charging test method, which is applied to a terminal device, where the content charging test method includes:

acquiring a test case from content charging test management equipment, wherein the test case is generated by the content charging test management equipment according to a charging rule in work order information;

and carrying out charging test according to the test case.

In a third aspect, an embodiment of the present invention further provides a content charging test management apparatus, including: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the content rating test method as described above in the first aspect when executing the computer program.

In a fourth aspect, an embodiment of the present invention further provides a terminal device, including: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the content rating test method as described above in the second aspect when executing the computer program.

In a fifth aspect, an embodiment of the present invention further provides a storage medium storing computer-executable instructions, where the computer-executable instructions are configured to execute the content billing test method described above.

The embodiment of the invention comprises the following steps: when the configuration condition of the updated content charging rule needs to be tested, the content charging test management equipment acquires work order information, wherein the work order information comprises the charging rule and a first service code. And the content charging test management equipment generates a test case according to the charging rule and sends the test case to the terminal equipment. And after the terminal equipment receives the test case, performing test processing according to the test case. In the process of carrying out the charging test by the terminal equipment, the network element management system tracks the charging test process to generate a test signaling and sends the test signaling to the content charging test management equipment. And the content charging test management equipment generates a charging test result according to the test signaling and the first service code, and obtains the configuration condition of the updated charging rule. According to the scheme of the embodiment of the invention, the content charging test management equipment is used for receiving the work order information, and the content charging test management equipment is used for carrying out processing such as automatic generation of a test case, analysis of a test signaling and the like according to the work order information, so that the configuration condition of the updated content charging rule is obtained. The scheme of the embodiment of the invention can use the content charging test management equipment and the real terminal equipment to carry out the test task under the conditions that the content charging rule is updated and the configuration condition is unknown, the two equipment are mutually matched, and whether the updated charging rule is configured correctly or not is tested, thereby effectively improving the accuracy and the test efficiency of the content charging test, providing a more reliable configuration reference result for operation and maintenance engineers, and improving the working efficiency of engineers.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Drawings

The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the example serve to explain the principles of the invention and not to limit the invention.

Fig. 1 is a diagram of a network architecture for implementing a content charging test method according to an embodiment of the present invention;

FIG. 2 is a diagram of a software product architecture for implementing content billing tests according to an embodiment of the present invention;

fig. 3 is a functional block diagram of a content billing test management device according to an embodiment of the present invention;

fig. 4 is a schematic functional module diagram of a terminal device according to an embodiment of the present invention;

fig. 5 is a schematic diagram illustrating a testing process when a testing agent device is not configured according to an embodiment of the present invention;

fig. 6 is a flowchart illustrating a content charging test method applied to a content charging test management device according to an embodiment of the present invention;

fig. 7 is a schematic diagram illustrating a signaling comparison analysis process according to an embodiment of the present invention;

fig. 8 is a flowchart illustrating a specific method of step S620 in fig. 6 according to an embodiment of the present invention;

fig. 9 is a schematic flow chart illustrating the generation of use cases in the content billing test management device according to an embodiment of the present invention;

FIG. 10 is a diagram illustrating test script generation rules, according to an embodiment of the present invention;

fig. 11 is a flowchart illustrating a content charging testing method applied to a terminal device according to an embodiment of the present invention;

fig. 12 is a flowchart illustrating a specific method of step S1120 in fig. 11 according to an embodiment of the present invention;

fig. 13 is a schematic processing flow diagram of a mobile phone executing a test case according to an embodiment of the present invention;

FIG. 14 is a top view scene interface of iTest-Cloud according to an embodiment of the present invention;

FIG. 15 is a view interface of iTest-Cloud according to an embodiment of the present invention;

fig. 16 is a mobile phone end APP interface provided in an embodiment of the present invention;

fig. 17 is a content charging test management apparatus according to an embodiment of the present invention;

fig. 18 is a terminal device according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

It should be noted that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different from that in the flowcharts. The terms first, second and the like in the description and in the claims, and the drawings described above, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.

In the description of the present invention, the meaning of a plurality of the terms is one or more, the meaning of a plurality of the terms is two or more, and the terms larger, smaller, larger, and the like are understood as excluding the present number, and the terms larger, smaller, and the like are understood as including the present number.

The invention provides a content charging test method, a management device, a terminal device and a storage medium, wherein the content charging test management device is used for acquiring work order information, wherein the work order information comprises a first service code of a charging rule; and automatically generating a test case according to the charging rule, and sending the test case to the terminal equipment so that the terminal equipment performs charging test processing according to the test case after receiving the test case. The content charging test management equipment acquires a test signaling generated in the tracking charging test process of the network element management system, generates a charging test result according to the test signaling and the first service code, and acquires the configuration condition of the updated charging rule.

The invention is used for verifying the correctness of the core network content charging rule configuration, in particular to the configuration of a 5G core network. Before the operation of the operator core network on line or after the significant operations such as capacity expansion, upgrading, data configuration change and the like are carried out on the core network, the content charging test management equipment and the terminal equipment are matched with each other, and the configuration of the content charging rule of the core network is automatically tested, so that the accuracy and the efficiency of the test aiming at the charging rule are improved.

The embodiments of the present invention will be further explained with reference to the drawings.

Referring to fig. 1, fig. 1 is a network architecture diagram for implementing a content charging test method according to an embodiment of the present invention. The network architecture diagram shown in fig. 1 includes five parts, which are a terminal device part, a content billing test management device part, a station side part, a network element management system part, and a 5G operator network part, respectively.

Referring to fig. 1, an iTest-APP is deployed in a terminal device part, and the iTest-APP is mobile phone APP software for testing content charging rules and supports an android system. The content charging test management device part is deployed with iTest-Cloud, which is a Cloud system for testing content charging rules and supports Linux and Windows operating systems. The Station part is deployed with an iTest-Station, which is Station system software for testing the content charging rules. The terminal device, the content charging test management device and the station end are all connected to the Internet and can be in communication connection, exchange receipts can be transmitted among the terminal device, the content charging test management device and the station end, and the terminal device, the content charging test management device and the station end are matched with each other to jointly complete the content charging test aiming at the 5G core network.

It should be noted that the content charging test management device may be a server or a virtual machine, and the test agent device may be a notebook computer or a conventional desktop computer. The terminal equipment can be a mobile phone or other electronic equipment capable of running the iTest-APP.

An EMS (Element Management System) is a System that manages one or more telecommunication NEs (Network elements) of a specific type. The EMS is a network management system of a core network, has functions of accounting, topology management, system monitoring and the like, and can track signaling data of internet access service on user terminal equipment. The test agent equipment is communicated with an EMS management plane network where the tested 5G core network is located. The test agent device with the iTest-Station can automatically acquire the signaling tracking data of the internet access service of the user terminal device from the EMS, and upload the signaling tracking data to the content charging test management device, so that the content charging test management device performs data comparison and analysis to obtain the configuration condition of the charging rule. In addition, an engineer can also manually download signaling tracking data from the EMS after the iTest-APP (i.e. the APP for dial testing) of the terminal device executes the test script, and import the signaling tracking data into the iTest-Cloud, so that the signaling tracking data is compared and analyzed to obtain the charging rule configuration condition.

Referring to fig. 1, a 5G operator network includes, but is not limited to: a (radio) Access Network (R) AN, a User Plane Function (UPF) Network element, AN Access and Mobility Management Function (AMF) Network element, a Session Management Function (SMF) Network element, and a Charging Function (HCF) Network element. In the above-mentioned 5G operator network, the part other than the (radio) access network is referred to as a 5G core network.

As used in this application, the 5G network structure employs a reference point-based representation to represent interactions between network functions. The reference points are a reference point based on a service interface and a reference point based on traditional point-to-point communication.

In fig. 1, interfaces such as N1, N2, N3, N6, N11, and N40 provide interfaces for the operator network with interface serial numbers. The functions of these interfaces and the specific meanings of the interface serial numbers can be referred to the definitions and explanations in the 3GPP standard protocol, and are not described herein again. For the purpose of understanding the present invention, some functional network elements involved in the network architecture of the present application are explained below.

AN Access Network (AN) is a sub-Network of AN operator Network, and is AN implementation system between a service node and a terminal device in the operator Network. The (radio) Access Network (R) AN is one of the AN devices, and is capable of providing a terminal device with a radio communication function. The (R) AN means that all or part of the fixed users are wirelessly accessed to the switch.

It should be noted that the radio access network is a terrestrial infrastructure, and the access network devices include, but are not limited to: next generation base stations (G NodeB, G nb) in 5G, radio network controllers, base station controllers, and mobile switching centers, etc.

The data network is a network outside the operator network, the operator network can access a plurality of DNs, and the DNs can be used for deploying various services and providing services such as data and/or voice for terminal equipment.

The user plane functional network element is a gateway provided by an operator, is a gateway for communication between the operator network and the DN, and can realize functions related to the user plane, such as packet routing forwarding, policy implementation, service usage reporting, uplink packet detection, downlink data packet storage and the like.

The access and mobility management function network element is a control plane network element provided by an operator network, has access and mobility management functions, and can complete access and mobility management of terminal equipment accessing the operator network. For example, the functions include mobility state management, user temporary identity allocation, access authentication and access authorization.

The session management function network element is responsible for managing a Protocol Data Unit (PDU) session of the terminal device, that is, processing a service of a user. The SMF network elements are responsible for setting up, maintaining and deleting PDU sessions and are also capable of managing sessions, such as session setup, modification and release. The SMF network element also has session-related functions of selecting and controlling the UPF network element, forwarding session information to an appropriate destination network, etc.

The CHF network element has a charging function and supports online charging, offline charging and fusion charging. The integrated charging is a charging mode newly added in the 5G specification, and can integrate two charging modes of online charging and offline charging. In this mode, both online and offline, the network element communicates charging messages with CHF network elements over the Nchf services interface.

In the network architecture, user terminal equipment accesses a core network through a base station (R) AN to perform various services. Each network element in the 5G core network, such as an SMF network element and a short message network element, reports a charging event related to user internet access to a CHF network element through a service interface Nchf, and the CHF network element is responsible for completing charging and quota authorization and generating a ticket file to perform accounting processing in an accounting domain. The network element management system tracks the signaling data of the internet access service of the terminal equipment and transmits the signaling data to the test agent equipment. The test agent device automatically transmits the signaling trace data to the content billing test management device. And the content charging test management equipment performs data comparison and analysis to obtain the configuration condition of the charging rule, and completes the charging rule test aiming at the 5G core network.

Referring to fig. 2, a software product architecture diagram for implementing content billing test according to an embodiment of the present invention is provided. In the example of fig. 2, the software product architecture diagram includes an iTest-APP at the terminal device side, an iTest-Cloud at the charging test management side, an iTest-Station at the test agent device side, and an element management system. The iTest-APP, the iTest-Cloud and the iTest-Station are all connected to the Internet, and can be in communication connection with each other. In a specific example, the local area network IP of the terminal device end is 192.168.101.x, the local area network IP of the test agent device end is 192.168.51.x, the management plane IP is 192.168.2.x, and the public network IP of the charging test management end is 58.213.91. x. The terminal equipment end can access the test agent equipment end through the charging test management end. The agent test end and the network element management system access each other through the management plane to realize data transmission.

It should be noted that, the test agent device end is optional, and the software product may be configured with the test agent device end or may not be configured with the test agent device end. When the software product is provided with the test agent equipment terminal, the test agent equipment terminal automatically acquires signaling tracking data when the mobile phone executes a test case from the core network, and then automatically uploads the data to the charging test management terminal, and the charging test management terminal compares and analyzes the signaling data. The automatic testing process effectively improves the efficiency of the charging test and reduces the workload of operation and maintenance personnel.

When the test agent equipment end is not configured in the software product or the test agent equipment end fails to be normally used, after the iTest-APP of the terminal equipment executes the test script, the signaling tracking data can be manually downloaded from the core network, after the iTest-Cloud is introduced, the data comparison analysis is performed, and after the iTest-Cloud is analyzed, a test result analysis report can be derived. Therefore, the present invention does not specifically limit whether or not the test agent device is configured in the software product.

Referring to fig. 3, fig. 3 is a functional module diagram of a content billing test management device according to an embodiment of the present invention. The main functional modules of the content charging test management device deployment iTest-Cloud, iTest-Cloud include:

the work order import module is used for obtaining the content charging work order of the operator, processing and analyzing the content charging work order and obtaining the work order information, wherein the work order information comprises: a charging rule and a first service code. The content charging work order of the operator is generally in an Excel format, the form filling of the content charging work order needs to meet the specified format, and the work order information also comprises main information such as a service name and the like besides a charging rule and a first service code.

And the case processing module is used for acquiring the charging rule from the work order import module and generating a test case according to the charging rule, wherein the test case comprises a test script and a test rule. Specifically, the use case processing module comprises an experience library module, a URL library module, a rule conversion module and a use case generation module. The experience library module, the URL library module and the rule conversion module can be used for obtaining the charging rule and the first network address corresponding to the charging rule from the work order import module and generating a matched accessible test address according to the address type of the first network address.

And the experience library module is used for acquiring a first network address corresponding to the charging rule from the work order import module, and determining a matched second network address as a test address according to the first network address, or storing the first network address of the charging rule which is tested successfully. The URL address successfully tested by the charging rule can be automatically stored in an experience library. One charging rule corresponds to one network address, and the network address is tested and successfully accessed. The experience base can have a large number of network addresses which are successfully tested and accessed through learning, data in the experience base can be automatically matched when a content expense bill is imported, and the testing success rate and the testing efficiency are improved.

And the URL library module is used for importing a first network address corresponding to the charging rule from the work order import module and determining a matched third network address as a test address according to the first network address. The URL library can match the charging rule with the accessible URL address through the big data system and the domain name crawler system and store the URL address into the URL library, the charging rule can be matched with the accurate URL address from the URL library for testing, and the testing success rate and the testing efficiency are improved.

And the rule conversion module is used for acquiring the first network address corresponding to the charging rule from the work order import module and determining a fourth network address as a test address according to the test script generation rule and the first network address. In actual testing, a testing address matched with the first network address cannot be obtained from the experience library module and the URL library module, and at this time, each rule in the content billing worksheet can be converted into an accessible testing address by using the rule conversion module. The normal operation and success rate of the charging test are effectively ensured.

And the case generation module is used for acquiring the test address from the experience library module or the URL library module or the rule conversion module and generating a test case according to the test address, wherein the test case comprises a test script and a test rule.

And the scene management module is used for acquiring the test case from the case generation module to create the test scene. The scene management module can meet diversified test requirements, and operation and maintenance personnel can select a proper test scene to test, so that the reliability and accuracy of a test result are improved.

And the task management module is used for acquiring the test case from the case generation module, generating a test task according to the test case and sending the test task to the terminal equipment so that the terminal equipment performs charging test according to the test script and the test rule in the test task. After the task management module creates a test task, the task is synchronized to the mobile phone through an API (application program interface) of the butt-joint mobile phone, and a tester can execute the task on the iTest-APP of the mobile phone.

And the signaling analysis module is used for acquiring the first service code from the work order import module and the test signaling sent by the network element management system. And in the process of carrying out the charging test by the terminal equipment, the network element management system carries out signaling tracking to generate a test signaling. The signaling analysis module can generate a charging test result according to the test signaling and the first service code. The signaling analysis module can import a test signaling or automatically acquire the signaling from a station end, namely a test agent device end, by interfacing with a station end API, and then perform signaling analysis, compare whether a Rate Group (RG) or Service Id (SI) message in the signaling is the same as a Service code of an operator work order, and determine whether each charging rule is configured correctly.

In addition, the iTest-Cloud also has a report export module, which is used for obtaining the charging test result from the signaling analysis module and automatically generating a test report of the content charging test, wherein the export format of the test report is an Excel format.

Referring to fig. 4, fig. 4 is a schematic diagram of functional modules of a terminal device according to an embodiment of the present invention. Terminal equipment deploys iTest-APP, and main functional modules of the iTest-APP comprise:

and the API module is connected with the content charging test management equipment terminal and is used for receiving the message of the content charging test management equipment terminal and sending data to the content charging test management equipment terminal. The task management module is used for synchronizing tasks of the content charging test management equipment terminal, so that engineering personnel can execute the test tasks on the terminal equipment, check detailed information of the tasks and the like; the case management module is used for managing test cases under the test tasks, selecting or repeatedly executing some cases, checking case execution results and the like.

The iTest-APP also has a case execution module, and different case execution methods are adopted for testing content charging according to the IP address or URL address in the test case. In the specific use case execution, two modules of sending a GET request and acquiring a GET response are used. Specifically, the case execution module can obtain the test task, and perform the test task by adopting different corresponding case execution methods according to the test rule in the test task; the GET sending request module is in communication connection with the use case execution module and is used for constructing use case execution request information and sending the use case execution request information to a core network; the GET response module is in communication connection with the case execution module and is used for receiving case execution response information returned by the core network, and the case execution response information is used for responding to the case execution request information and confirming that the core network tests the network address corresponding to the charging rule.

The network architecture and the application scenario described in the embodiment of the present invention are for more clearly illustrating the technical solution of the embodiment of the present invention, and do not form a limitation on the technical solution provided in the embodiment of the present invention, and it is known to those skilled in the art that the technical solution provided in the embodiment of the present invention is also applicable to similar technical problems along with the evolution of the network architecture and the appearance of new application scenarios.

Those skilled in the art will appreciate that the network architecture shown in fig. 1 is not intended to limit embodiments of the present invention and may include more or fewer components than shown, or some of the components may be combined, or a different arrangement of components.

Based on the architecture and the functional modules of fig. 1 to fig. 4, the following provides various embodiments of the content charging test method of the present invention.

Take a specific example:

when the test agent equipment end is not configured in the software product or the test agent equipment end fails to be normally used, after the iTest-APP (i.e. the dial-test APP) of the terminal equipment executes the test script, an engineer can manually download signaling tracking data from the core network, and import the iTest-Cloud, so that the iTest-Cloud performs data comparison and analysis to obtain a test result.

Referring to fig. 5, fig. 5 is a schematic diagram illustrating a testing process when a testing agent device is not configured according to an embodiment of the present invention. And the work order processing terminal of the operator screens out the effective URL address to generate a work order and introduces the work order into the content charging test management equipment terminal. After receiving the work order imported by the work order processing end of the operator, the content charging test management device end generates a test case for charging test, which needs to be explained that errors or deviations may exist when the test case is generated, so that the generated test case can be manually modified to better meet the test requirements. After the test case is generated, the content charging test management equipment end creates a scene packet according to the test case, and then creates a test task according to the scene packet. And after the test task is created, the content charging test management equipment terminal sends the test task to the mobile phone terminal. The content charging test management equipment terminal and the mobile phone terminal can manage the test tasks and check the detailed information of the tasks. And after receiving the test task sent by the content charging test management equipment terminal, the mobile phone terminal executes the test case in the test task. In the process of executing the test task by the mobile phone terminal, the EMS tracks the test process, and generates a signaling tracking file according to the result of executing the test task by the mobile phone terminal. The engineer manually exports a signaling file with an interface name of N40 and imports the signaling file into the cloud. After receiving the imported signaling file, the content charging test management equipment terminal analyzes the signaling file, compares SI information in the work order, obtains an analysis report about the charging rule configuration condition, and displays the configuration condition result. Up to this point, the billing test is completed in the event that the test agent device is not configured or fails.

It should be noted that the content charging test management device creates a scenario packet according to the test case, and creates a corresponding test task according to the scenario packet. The content charging test management equipment terminal can generate a test task carrying a test case under the condition of selecting the scene packet, and sends the test task to the terminal equipment, so that the terminal equipment executes the test case in the test task; the content charging test management device end can also directly send the test case to the terminal device under the condition of not creating a scene packet and a test task, so that the terminal device executes the test case to complete the charging test. The method and the device have no specific limitation on the specific scene and the specific mode of sending the test case to the terminal device by the content charging test management device end, and the terminal device can execute the test case to complete the charging test.

The charging rules are tested, so that the network quality and the customer satisfaction can be improved to a certain extent, and complaints similar to charging errors are effectively reduced. The invention can conveniently test the configuration state or the non-configuration condition of the content charging rule. The telephone charge dispute caused by wrong charging rules is avoided, good economic and social benefits can be brought to operators, and the use experience of users is improved.

Referring to fig. 6, fig. 6 is a flowchart illustrating a content billing test method applied to a content billing test management device according to an embodiment of the present invention. The content billing test method may include, but is not limited to, step S610, step S620, step S630, step S640, and step S650.

Step S610: and acquiring work order information, wherein the work order information comprises a charging rule and a first service code.

In this step, the work order information is obtained from the operator. It is understood that the work order information includes, but is not limited to, a charging rule, a service code, and further includes main information such as a service name and a charging unit price. For convenience of processing, the work order information adopts an Excel format, and the filling of the main information in the work order meets the specified format.

Step S620: and generating a test case according to the charging rule.

In this step, specifically, the content charging test management device obtains an accessible test address according to the charging rule, obtains a test script according to the test address, and generates a test case according to the test script and a preset test rule. The preset test rule is a preset charging rule to be tested. The generated test case has a test script and a test rule.

It should be noted that, the content charging test management device generates a test case for each charging rule, where the test case includes a test script and a test rule. It can be understood that the number of the charging rules to be tested is large, and after the test case for each charging rule is obtained, an automatic test case set is automatically created.

Step S630: and sending the test case to the terminal equipment so that the terminal equipment performs the charging test according to the test case.

In this step, the content billing test management device sends the test case to the terminal device. And after the terminal equipment receives the test case, carrying out charging test according to the test case. It should be noted that before sending the test case to the terminal device, the engineer may also select a corresponding test scenario package at the content billing test management device according to the test scenario. And the content charging test management equipment selects a required test case according to the selected test scene, creates a test task, and sends the test task to the terminal equipment to be distributed to the testers. And the terminal equipment starts to carry out charging test on the test script and the test rule in the test task according to the specific instruction of the tester. The test scene is close to the real scene, so that the reliability and reliability of the test result can be improved. The method and the device do not specifically limit whether the content charging test management device end performs the test scene selection.

Step S640: and acquiring a test signaling generated by the network element management system to the terminal equipment in the charging test process.

In this step, the content billing test management device uses the EMS accessed to the 5G core network, and starts signaling tracing for the test handset through the EMS, thereby obtaining the test signaling generated by the network element management system in the process of performing the billing test on the terminal device.

Step S650: and generating a charging test result according to the test signaling and the first service code.

In this step, the content charging test management device obtains the charging session response information according to the test signaling, and obtains the second service code from the charging session response information. Comparing the second service code with the first service code, and under the condition that the second service code is the same as the first service code, the charging test result is that the charging rule is correctly configured; and under the condition that the second service code is different from the first service code, the charging test result indicates that the charging rule configuration is wrong.

Specifically, referring to fig. 7, fig. 7 is a schematic diagram illustrating a signaling comparison analysis flow according to an embodiment of the present invention. The content charging test management device receives an EMS signaling file during the case execution period sent by the EMS, and firstly carries out a signaling searching process to obtain detailed signaling data. And then, carrying out a field comparison process according to the detailed data of the signaling to obtain a test result, and finally generating a test report. In the signaling lookup procedure, the test management device screens out a signaling with an interface name of N40 from the received EMS signaling, and then screens out a signaling with an information name of Nchf _ convertgedchargg _ Release _ Req (Release charging session response information) or Nchf _ convertgdchargg _ Update _ Req (Update charging session response information) from the signaling with the interface name of N40 within the case execution time. The value of the service code is then extracted from the two signalling messages, the fields in the test signalling being the Rating Group (RG) and the service code (SI). The content charging test management equipment compares whether the service code in the test signaling is the same as the service code in the work order provided by the operator, when at least one of the RG and SI in the test signaling is equal to the service code in the work order, the analysis of the test signaling is successful, the configuration of the content charging rule is correct, otherwise, the configuration of the content charging rule is wrong.

Operators have paid great attention to operation and maintenance digitization, including test automation. The invention can greatly save the labor cost, can automatically compare the service code of the work order with the N40 port signaling file in the test signaling, and replaces the manual test flow of content charging. The invention can realize automatic test, has simple operation, effectively improves the efficiency of the charging test, has low requirement on the technical ability of the user and is easier to popularize.

In an embodiment, referring to fig. 8, fig. 8 is a flowchart illustrating a specific method of step S620 in fig. 6 according to an embodiment of the present invention, and the step S620 is further described, where the step S620 may include, but is not limited to, step S810 and step S820.

Step S810: and acquiring a first network address corresponding to the charging rule according to the charging rule.

In this step, the first network address may be an IP address or a URL address or a combination of the IP address and the URL address. It should be noted that, in the present application, the test address obtaining modes corresponding to different address types are different.

Step S820: an accessible test address is determined based on the address type of the first network address.

In this step, when the first network address is a URL address, determining an accessible test address according to an address type of the first network address includes at least the following three cases:

the first case is: and under the condition that a second network address matched with the first network address is recorded in the experience base, determining the second network address as a test address in the test script, and storing the charging rule which is tested successfully and the network address corresponding to the charging rule in the experience base. The second case is: and under the condition that the second network address is not recorded in the experience library, acquiring a third network address matched with the first network address from the URL library as a test address in the test script. The third case is: and under the condition that the first network address cannot be acquired from the experience library and the URL library, determining a fourth network address as a test address according to a preset test script generation rule and the first network address.

And when the first network address is an IP address or a combined address of the IP address and the URL address, determining a fifth network address corresponding to the first network address as an accessible test address according to the first network address and a preset test script generation rule.

As a specific example, referring to fig. 9, fig. 9 is a schematic flow diagram illustrating a process of generating a use case in a content billing test management device according to an embodiment of the present invention.

It should be noted that the content billing test management device includes, but is not limited to, the functional modules shown in fig. 3, and may further include a use case management module, where the use case management module is used to manage test cases. As shown in fig. 9, the use case management module first adds a Deep Packet Inspection (DPI) use case set. And judging whether the test address of the test case is a seven-layer rule, namely judging whether the test address is a URL (uniform resource locator) address.

When the test address is the URL address, whether a matched test address exists or not is searched in the experience base. If the experience base has the record of the charging rule, the network address corresponding to the rule recorded in the experience base is used as the test address accessed by the mobile phone in the test script.

If the test address is not matched in the experience library, then matching is performed in the URL library. The data recorded in the URL library is mass accurate URLs found in advance through big data or a crawler technology. Based on the method, the iTest-Cloud can search a URL library for an accurate address matched with a rule, and if the rule is matched with the address, the matched address is used as a test address accessed by a mobile phone in a test script.

If the test case can not find the URL address of the matched record in the experience library and the URL library, or the URL address is the IP address. Then according to the corresponding rule, namely the test script generation rule, processing is carried out to obtain the test address accessed by the mobile phone in the test script. And generating a mobile phone script after the test address is obtained, and finally generating a use case.

It should be noted that the rule corresponding to the network address recorded in the experience base is a charging rule that has been tested successfully. Specifically, the use case management module manages the test cases, the scene management module creates test scenes by using the test cases in the use case management module, creates test tasks according to the test scenes, and sends the test tasks to the mobile phone APP. And the mobile phone APP executes the test task, and the iTest-Cloud receives the imported signaling and analyzes the signaling. And when the access URL address is successful in executing the task and the signaling analysis is successful, automatically inputting the URL address into an experience library. The testing method provided by the invention can be self-learned, and provides training samples for the realization of the testing robot. And storing the successfully tested content charging rules into an experience library, wherein the accessible domain name in the library can be used for replacing the original rules as long as the rules in the operator work order have records in the experience library, so that the testing success rate is improved.

The iTest-Cloud may automatically translate the network address of the charging rule into an accessible test address in the rule translation module for the network address of the different charging rule. Specifically, referring to fig. 10, fig. 10 is a schematic diagram of a test script generation rule according to an embodiment of the present invention. Taking a certain DPI work order as an example, the processing rule is as follows:

for three-tier only rules (IP address), if it is a specific address, this address is IPV4, e.g. 49.4.42.14, then access is direct. If it is an IPV6 address, the 128 masked access is removed. If the network address is address range only, for IPV4, removing the head and tail available addresses as accessible test addresses; for IPV6 addresses, an address range is calculated based on the mask, taking the first available address as the accessible test address. And adding the specified port to the network address of the specified port and the specified address to obtain the test address. For the network address of the designated address and the port range, a smallest port in the port range is taken to be spliced after the designated address to obtain an accessible test address. When the network address has an address range and a port range, for the IPV4, splicing a minimum port in the port range into an available address in the address range to obtain an accessible test address; for IPV6, a test address accessible after removing masked addresses is spliced to the smallest port in the port range. And then generating a mobile phone test script to test the charging rule.

For the seven-layer rule (URL address), if there is no wildcard, then direct access is based on the exact address. If there is a wildcard, the wildcard for the port is replaced with the 8080 actual port, and the others are removed. And then generating a mobile phone test script to test the charging rule. For the three-layer and seven-layer combination rules, the three-layer rule and the seven-layer rule are respectively processed according to the above rules.

Referring to fig. 11, fig. 11 is a flowchart illustrating a content charging testing method applied to a terminal device according to an embodiment of the present invention. The content billing test method may include, but is not limited to, step S1110 and step S1120.

Step S1110: and the terminal equipment acquires the test case from the content charging test management equipment.

In this step, the terminal device obtains a test case from the content billing test management device, and the test case is generated by the content billing test management device according to the billing rule in the work order information. The terminal device may directly obtain the test case from the content charging test management device, or obtain the test task from the content charging test management device, and then obtain the test case from the test task. When the terminal device obtains the test task from the content charging test management device, the terminal device can automatically synchronize the test task created by the content charging test management device. The tester can log in the iTest-APP of the terminal equipment, check the synchronized test task and then execute the test cases in the test task. The user can operate on the mobile phone, perform testing tasks or view task details, and the like.

Step S1120: and carrying out charging test according to the test case.

In this step, the terminal device performs a charging test on the charging rule according to the test case.

In an embodiment, referring to fig. 12, fig. 12 is a flowchart illustrating a specific method of step S1120 in fig. 11 according to an embodiment of the present invention, and the step S1120 is further described, where the step S1120 may include, but is not limited to, step S1210, step S1220, step S1230, and step S1240.

Step 1210: and the terminal equipment acquires the test rule according to the test case.

In this step, the terminal device obtains a test case from the content charging test management device, where the test case includes a test script and a test rule. Wherein, the test rule is a preset charging rule to be tested. The terminal device may obtain the test rule from the test case.

Step S1220: and acquiring a test rule to determine a test address.

Step S1230: and determining a corresponding use case execution rule according to the address type of the test address.

In this step, the address types of the test addresses at least include one of: IP address, URL address. It is understood that the address type of the test address may be an IP address, a URL address, or a combination of an IP address and a URL address. Different address types correspond to different case execution rules to ensure that the test case can be executed smoothly, and the success rate of the charging test is improved.

Step S1240: and carrying out charging test according to the case execution rule.

In this step, still include: the terminal equipment sends the case execution request information according to the case execution rule, and then executes the test script according to the case execution rule under the condition of receiving the case execution response information so as to perform the charging test. The terminal equipment ensures that the test cases can be successfully tested through the case test processing flow.

Specifically, referring to fig. 13, fig. 13 is a schematic view of a processing flow of the mobile phone executing the test case according to an embodiment of the present invention. Specifically, the charging rules may be divided into three-layer rules, seven-layer rules, and combination rules, where a combination rule worth a combination of three-layer rules and seven-layer rules. And the mobile phone end starts testing through the keywords and judges whether the testing rule is a combination rule or not. And when the test rule is the combination rule, performing TCP link establishment and further judging whether the test rule is an HTTPS protocol. And under the condition that the TCP link establishment is successful and is not an HTTPS protocol, the mobile phone end constructs a GET message to be sent, and the network address corresponding to the rule is ensured to be tested by obtaining the GET message. And under the condition that the TCP link establishment is successful and the TCP link establishment is an HTTPS protocol, the mobile phone end sends a ClientHello message carrying the matched domain name, sends a GET request if the authentication is successful, receives a GET response to ensure that the network address corresponding to the rule is tested, and ends the test if the authentication is unsuccessful. And under the condition that the TCP link establishment fails and the TCP link establishment is an HTTPS protocol, constructing and sending a ClientHello message to finish the test, wherein the destination address in the message is a three-layer address in the combined address, namely an IP address, and the servername is a domain name in the combined rule. Under the condition that TCP link establishment fails and is not an HTTPS protocol, constructing and sending a GET message to finish testing, wherein the destination address in the GET message is a three-layer rule in a combination rule, and Host is a domain name in the combination rule.

When the test rule is not a combination rule and is a three-layer rule (IP address), the link is established according to the TCP specified port in the combination rule. If the link is successfully established, the charging rule is triggered, the mobile phone end sends a GET message to obtain a GET response, and the network address corresponding to the test rule is ensured to be tested, so that the test is completed. If the link establishment fails, the test is ended.

When the test rule is not a combination rule, and is a seven-layer rule (URL address). For a test rule needing DNS domain name resolution, if DNS resolution is successful and link establishment is successful, directly sending a GET request to HTTP, and acquiring a GET response to complete the test; and for HTTPS, sending a ClientHello message carrying the matched domain name, sending a GET request if the authentication is successful, completing the test, and ending the test if the authentication is unsuccessful. Under the conditions that DNS analysis is successful and link establishment is failed, a GET request is directly sent to HTTP to complete the test, the target address in the GET request is an IP address of link establishment, and Host is a seven-layer domain name; and for the HTTPS, sending a ClientHello message carrying the matched domain name to finish the test, wherein the destination address in the ClientHello message is the IP address of the established chain, and the servername is the domain name of seven layers. If the test address analysis fails or DNS analysis is not needed, directly sending a GET message to HTTP, and constructing and sending a ClientHello message to HTTPS to finish the test, wherein the destination addresses of the GET message and the ClientHello message are the same, the servername in the ClientHello message is a seven-layer domain name, the Host in the GET message is a seven-layer domain name, and if the GET message has a port, the GET message also needs to carry the port.

The test address may not exist or may be virtual, and constructing and sending the GET message can ensure that the network address corresponding to the charging rule is tested, improve the success rate of the charging test, and save the test resources.

Referring to fig. 14, fig. 14 is a front page scene interface of iTest-Cloud according to an embodiment of the present invention. The scene interface has test scene selection, and a user can select a scene package in a hot scene on a home page and click to pay attention to the scene package.

The scene package focused by the user is displayed in the sub-page of the my scene package, referring to fig. 15, fig. 15 is a sub-scene interface of iTest-Cloud provided by an embodiment of the present invention. The user may select a scene package creation task of interest in a sub-scene interface as shown in fig. 15. After the user focuses on the scene package, a task can be created according to the scene package. After the task is established, the mobile phone terminal and the iTest cloud end can display the task, and the task states of the two sides are consistent. In the sub-scenario interface shown in FIG. 15, after the user can click on the create task again, the task name can be modified in another sub-page. The tasks are created, and the tasks created by the user can be seen on the My tasks page.

Referring to fig. 16, fig. 16 is an APP interface at a mobile phone end according to an embodiment of the present invention. After the task is established, the mobile phone end and the iTest cloud end can display the task, and the task states of the two sides are consistent. The mobile phone APP page is shown in fig. 16. The user can select the DPI test task to be executed in the APP page of the mobile phone, click to execute, download the case, click to continue, and wait for the mobile phone to finish the DPI test.

After the mobile phone completes the test, the user logs in the iTest-Cloud to check the task, and the task state is updated from the created state to the completed state. The user can log in the EMS management system to export the signaling tracking file, enter a task page, find the task which is executed by the user, and import the signaling file generated after the task is executed into the iTest-Cloud. And after the signaling file is successfully imported, carrying out data analysis by using the iTest-Cloud data analysis function. After the analysis is finished, the user can click the task to check the task details, can check whether the execution result of each use case, namely the test of each URL address is successful, and can check the log, namely the comparison result of the RG number by clicking the analysis result. If the URL address test of the mobile phone APP is successful but the RG number does not exist, the analysis result is reflected, and the log shows that the RG number does not exist.

The embodiment of the invention provides a content charging test management device and a terminal device to complete charging test in a mutual matching way. The method replaces the manual testing process of content charging, and automatically compares the service code of the work order with the N40 in the signaling to obtain the testing condition of the charging rule. The progress of the testing task can be mastered by engineering personnel through the APP and the iTest-Cloud, the testing condition of the charging rule can be conveniently checked and verified, the labor cost is saved, and meanwhile the working efficiency of the engineering personnel is improved. The configuration condition of the charging rule is verified, so that the network quality and the customer satisfaction are improved, and similar complaints are effectively reduced. In addition, the invention can conveniently test the condition that the content charging rule is not configured. Telephone charge dispute caused by wrong charging rules is avoided, and good economic and social benefits are brought to operators.

Fig. 17 is a content billing test management apparatus 1700 according to an embodiment of the present invention. As shown in fig. 17, the content billing test management apparatus 1700 includes, but is not limited to:

a memory 1720 for storing a program; a processor 1710 for executing the program stored in the memory 1720, wherein when the processor 1710 executes the program stored in the memory 1720, the processor 1710 is configured to execute the content billing test method. The processor 1710 and memory 1720 may be connected by a bus or other means.

Fig. 18 is a terminal device 1800 according to an embodiment of the present invention. As shown in fig. 18, the terminal device 1800 includes but is not limited to: a memory 1820 for storing programs; the processor 1810 is configured to execute the program stored in the memory 1820, and when the processor 1810 executes the program stored in the memory 1820, the processor 1810 is configured to perform the content billing test method described above. The processor 1810 and the memory 1820 may be connected by a bus or other means.

Memory 1720 and memory 1820 serve as a non-transitory computer readable storage medium that may be used to store non-transitory software programs and non-transitory computer executable programs, such as the content billing test methods described in embodiments of the present invention. The processor 1710 and the processor 1810 implement the content billing test method described above by running non-transitory software programs and instructions stored in the memory 1720 and the memory 1820, respectively.

The memory 1720 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store the content billing test method described above. Further, memory 1720 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some implementations, the memory 1720 may optionally include memory located remotely from the processor 1710, which may be connected to the processor 1710 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

Non-transitory software programs and instructions necessary to implement the content billing test methods described above are stored in the memory 1720 and, when executed by the one or more processors 1710, perform the content billing test methods described above, e.g., performing method steps S610-S650 in fig. 6 and method steps S810-S820 in fig. 8.

In one embodiment, the storage medium stores computer-executable instructions that are executed by one or more control processors 1710, for example, by one of the processors 1710 in the content billing test management apparatus 1700, and cause the one or more processors 1710 to perform the content billing test method described above, for example, the method steps S610 through S650 in fig. 6 and the method steps S810 through S820 in fig. 8.

Non-transitory software programs and instructions necessary to implement the content billing test method described above are stored in the memory 1820 and, when executed by the one or more processors 1810, perform the content billing test method described above, e.g., perform method steps S1110 to S1120 in fig. 11 and method steps S1210 to S1240 in fig. 12.

In an embodiment, the storage medium stores computer-executable instructions, which are executed by one or more control processors 1810, for example, by one processor 1810 in the terminal device 1800, and may cause the one or more processors 1810 to execute the content charging test method, for example, the method steps S1110 to S1120 in fig. 11 and the method steps S1210 to S1240 in fig. 12.

The above described embodiments are merely illustrative, wherein elements illustrated as separate components may or may not be physically separate, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.

One of ordinary skill in the art will appreciate that all or some of the steps, systems, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.

Claims

1. A content charging test method is applied to content charging test management equipment and is characterized by comprising the following steps:

generating a test case according to the charging rule;

generating a charging test result according to the test signaling and the first service code;

wherein, the generating of the test case according to the charging rule comprises: obtaining an accessible test address according to the charging rule, obtaining a test script according to the test address, and generating the test case according to the test script and a preset test rule;

the obtaining the accessible test address according to the charging rule comprises: acquiring a first network address corresponding to the charging rule according to the charging rule, and determining the accessible test address according to the address type of the first network address;

when the first network address is a URL address, the determining the accessible test address according to the address type of the first network address includes:

determining a second network address matched with the first network address as the test address in the test script under the condition that the experience base records the second network address, wherein the experience base stores the charging rule which is tested successfully and the network address corresponding to the charging rule,

or,

acquiring a third network address matching the first network address from a URL library as the test address in the test script in the case that the second network address is not recorded in the experience library,

or,

and under the condition that the first network address cannot be acquired from the experience library and the URL library, determining a fourth network address as the test address according to a preset test script generation rule and the first network address.

2. The content billing method of claim 1, wherein the first network address is an IP address or a combination of an IP address and a URL address, and the determining the accessible test address according to the address type of the first network address comprises:

and determining a fifth network address corresponding to the first network address as the accessible test address according to the first network address and a preset test script generation rule.

3. The content charging test method according to claim 1, wherein the generating a charging test result according to the test signaling and the first service code comprises:

acquiring charging session response information according to the test signaling;

obtaining a second service code according to the charging session response information;

under the condition that the second service code is the same as the first service code, the charging test result is that the charging rule is configured correctly,

or,

and under the condition that the second service code is different from the first service code, the charging test result indicates that the charging rule configuration is wrong.

4. A content charging test method is applied to terminal equipment and is characterized by comprising the following steps:

carrying out charging test according to the test case;

the test case comprises a test script and a preset test rule;

the test script is obtained by the content charging test management equipment according to a test address;

the test address is determined by the content charging test management equipment according to the address type of the first network address corresponding to the charging rule, which is acquired by the charging rule;

when the first network address is a URL address, the content billing test management device determines that the accessible test address includes, according to the address type of the first network address:

in the case that a second network address matching the first network address is recorded in an experience base, the test address in the test script is determined as the second network address by the test management device, the experience base stores a charging rule which is successfully tested and a network address corresponding to the charging rule,

or,

in the case that the second network address is not recorded in the experience library, the test address in the test script is a third network address which is acquired by the test management device from a URL library and matches the first network address,

or,

and under the condition that the first network address cannot be acquired from the experience library and the URL library, the test address is a fourth network address determined by the content charging test management equipment according to a preset test script generation rule and the first network address.

5. The content charging test method according to claim 4, wherein the performing the charging test according to the test case comprises:

acquiring the test rule and the test script according to the test case;

determining the test address in the test script according to the test rule;

determining a corresponding use case execution rule according to the address type of the test address;

and carrying out charging test according to the use case execution rule.

6. The content billing method according to claim 5, wherein the billing test according to the use case execution rule includes:

sending use case execution request information according to the use case execution rule;

and executing the charging test according to the case execution rule under the condition of receiving the case execution response information.

7. The content billing method of claim 5, comprising: the address types of the test address at least comprise one of the following types: an IP address; the URL address.

8. A content charging test management apparatus comprising: memory, processor and computer program stored on the memory and executable on the processor, characterized in that the processor implements the content rating test method according to any of claims 1 to 3 when executing the computer program.

9. A terminal device, comprising: memory, processor and computer program stored on the memory and executable on the processor, characterized in that the processor implements the content rating test method according to any of claims 4 to 7 when executing the computer program.

10. A storage medium having stored thereon computer-executable instructions for performing the content rating test method of any of claims 1 to 3, or the content rating test method of any of claims 4 to 7.