CN114339864B

CN114339864B - Performance evaluation method, device, equipment and medium for network product

Info

Publication number: CN114339864B
Application number: CN202111624949.5A
Authority: CN
Inventors: 陈平; 陈可楠; 石振中; 高星
Original assignee: Ruijie Networks Co Ltd
Current assignee: Ruijie Networks Co Ltd
Priority date: 2021-12-28
Filing date: 2021-12-28
Publication date: 2024-04-23
Anticipated expiration: 2041-12-28
Also published as: CN114339864A

Abstract

The embodiment of the application provides a performance evaluation method, device, equipment and medium for network products, which are used for evaluating the performance of the network products before the network products are deployed on a commercial network. In the application, the electronic equipment can determine the running scene of the network product according to the running information of the network product in the laboratory simulation environment, and then determine the index value of the performance evaluation index of the network product in the scene, so that the performance evaluation result of the network product deployed in the commercial network is predicted according to the index value of the performance evaluation index, and the application is suitable for the performance evaluation of the network product which is not deployed in the commercial network. And moreover, the running quality of the network product can be obtained before the network product is deployed on a commercial network, so that more accurate performance evaluation and maintenance are performed on the network product, and better use experience is provided for users.

Description

Performance evaluation method, device, equipment and medium for network product

Technical Field

The present application relates to the field of wireless communications technologies, and in particular, to a method, an apparatus, a device, and a medium for evaluating performance of a network product.

Background

The key performance indicators (key performance indicator, KPI) directly reflect the quality of the mobile wireless network, so how to determine network quality and locate network problems based on KPIs, and thus network optimization is the focus of attention of operators.

In the traditional mode, a base station is mainly used for collecting and reporting performance files, and a network equipment management system analyzes and obtains KPIs (key performance indicators) under each dimension to represent corresponding network quality according to each dimension in a performance evaluation index system based on the performance files.

The conventional method is applicable to mature networks already deployed on commercial networks, can evaluate the network quality of the current network to a certain extent, but is not applicable to network products not deployed on commercial networks. Because the running quality of the network product cannot be known before the network product is not deployed, the running quality of the network product after the network product is deployed to the commercial network has overlarge deviation from the expected running quality, and good use experience cannot be provided for users.

Content of the application

The embodiment of the application provides a performance evaluation method, device, equipment and medium for network products, which are used for evaluating the performance of the network products before the network products are deployed on a commercial network.

The application provides a performance evaluation method of a network product, which comprises the following steps:

acquiring operation information of a network product, and determining a scene of the operation of the network product;

According to the running information, determining an index value of a performance evaluation index of the network product in the scene;

and predicting a performance evaluation result of the network product deployed under the commercial network according to the index value of the performance evaluation index.

Further, the determining the running scene of the network product includes:

And determining the commercial network scene corresponding to the first network behavior as the running scene of the network product according to the first network behavior in the running information of the network product and the corresponding relation between the second network behavior in the commercial network and the commercial network scene.

Further, the scene includes a wireless scene, an air interface scene, a load scene, a service scene, a network networking scene, a transmission scene, or an operation and maintenance scene.

Further, the wireless scenario includes one or more of the following: urban area scene, suburban area scene, rural area scene, and high-speed moving scene;

The air interface scene includes one or more of the following: a stay strategy scene, a coverage strategy scene and a network roaming scene;

The load scenario includes one or more of the following: urban heavy load scene, urban medium load scene, urban light load scene, office park heavy load scene, office park medium load scene, industrial factory building heavy load scene, industrial factory building medium load scene, industrial factory building light load scene;

the business scenario includes one or more of the following: a Transmission Control Protocol (TCP) scene, a user data packet protocol (UDP) scene, a voice scene and an instant messaging scene;

the network networking scenario includes one or more of the following: network sharing scene, PLMN scene of multiple public land mobile networks, core network resource pool scene, multimode co-station scene, network backup scene and first network evolution scene;

The transmission scenario includes one or more of the following: interface type scene, logic interface scene, back-pass interface scene, forward-pass interface scene, base station cascade scene, radio frequency cascade scene, multi-mode back-pass scene, transmission backup scene, and clock source scene;

The operation and maintenance scene comprises one or more of the following: network fault recovery scene, start-up scene, upgrade scene, hardware operation and maintenance scene, characteristic operation and maintenance scene, network operation and maintenance scene and second network evolution scene.

Further, the performance evaluation index comprises an interference index and/or a channel quality indication index.

In another aspect, the present application provides a performance evaluation apparatus for a network product, the apparatus comprising:

The first determining unit is used for acquiring the operation information of the network product and determining the operation scene of the network product;

The second determining unit is used for determining an index value of the performance evaluation index of the network product under the scene according to the running information;

and the prediction unit is used for predicting the performance evaluation result of the network product deployed under the commercial network according to the index value of the performance evaluation index.

Further, the first determining unit is specifically configured to determine, according to a first network behavior in the operation information of the network product and a correspondence between a second network behavior in the commercial network and a commercial network scenario, the commercial network scenario corresponding to the first network behavior as a scenario in which the network product operates.

the transmission scenario includes one or more of the following: interface type scene, logic interface scene, back-pass interface scene, forward interface scene, base station cascade scene, radio frequency cascade scene, multimode back-pass scene, transmission backup scene, and clock source scene.

In another aspect, the present application provides an electronic device comprising a processor and a memory, the processor being configured to implement the steps of any of the above-described methods of evaluating the performance of a network product when executing a computer program stored in the memory.

In another aspect, the present application provides a computer readable storage medium storing a computer program which, when executed by a processor, implements the steps of the performance evaluation method of any one of the above-described network products.

In the application, the electronic equipment can determine the running scene of the network product according to the running information of the network product in the laboratory simulation environment, and then determine the index value of the performance evaluation index of the network product in the scene, so that the performance evaluation result of the network product deployed in the commercial network is predicted according to the index value of the performance evaluation index, and the application is suitable for the performance evaluation of the network product which is not deployed in the commercial network. And moreover, the running quality of the network product can be obtained before the network product is deployed on a commercial network, so that more accurate performance evaluation and maintenance are performed on the network product, and better use experience is provided for users.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram illustrating a performance evaluation process of a network product according to some embodiments of the present application;

FIG. 2 is a schematic diagram illustrating a performance evaluation process of a network product according to some embodiments of the present application;

FIG. 3 is a schematic diagram illustrating a performance evaluation apparatus for a network product according to some embodiments of the present application;

Fig. 4 is a schematic structural diagram of an electronic device according to some embodiments of the present application.

Detailed Description

The present application will be described in further detail below with reference to the attached drawings, wherein it is apparent that the embodiments described are only some, but not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

In order to perform performance evaluation on a network product before deployment to a commercial network, the embodiment of the application provides a performance evaluation method, device, equipment and medium of the network product.

Example 1:

fig. 1 is a schematic diagram of a process for evaluating performance of a network product according to an embodiment of the present application, where the process includes the following steps:

s101: and acquiring the operation information of the network product, and determining the operation scene of the network product.

The performance evaluation method of the network product provided by the embodiment of the application is applied to the electronic equipment. The electronic device may be a server, or a computer, a mobile phone, etc. The electronic device is provided with a platform system which is used for evaluating the performance of the network product when the network product is simulated to run under a laboratory, and the electronic device adopts the platform system to evaluate the performance of the network product before the network product is deployed on a commercial network.

The network product may be a software product, a hardware product, or a combination of software and hardware.

When the network product is running on the platform system, the electronic device may obtain running information of the network product, where the running information includes the first network behavior and/or information related to the communication process. Optionally, the network behavior comprises user behavior and/or operator behavior, and the information related to the communication process is used to determine a performance assessment indicator of the network product.

In one possible implementation manner, the electronic device may obtain network behaviors in different scenes in the commercial network in advance, so as to determine, according to the network behaviors matched with the network product, a scene of the commercial network in which the network product conforms. Wherein, the description of the "scene" is referred to the subsequent embodiments.

S102: and according to the running information, determining an index value of the performance evaluation index of the network product in the scene.

The performance evaluation index can be one or more, and the performance evaluation index can be a performance evaluation index in the related technology or a newly added performance evaluation index. The performance evaluation index in the related art may be a performance evaluation index specified in a third generation partnership project (3rd generation partnership project,3GPP) protocol, and/or a performance evaluation index commonly used by those skilled in the art. The newly added performance evaluation index may be a network environment quality index, and/or other newly added index. For a description of the "performance evaluation index" in particular, reference is made to the following examples.

The scenes have corresponding performance evaluation indexes, and the performance evaluation indexes of the network product under different scenes can be completely the same, or can be partially the same, or can be completely different. For example, the performance evaluation index in the wireless scene includes an interference index, and the performance evaluation index in the service scene includes an interference index, a utilization index, and the like. As another example, each scene may correspond to all performance evaluation indicators. It can be understood that the correspondence between the scene and the performance evaluation index is only an example, and the correspondence between the scene and the performance evaluation index is not limited in practical application.

The operation information may further include evaluation information for determining relevant information of the performance evaluation index.

In one possible implementation, the electronic device may obtain, in the operation information, evaluation information for determining a performance evaluation index in the scenario. For example, for some scenarios that relate to a user communication process, the electronic device may obtain evaluation information related to the communication process, and determine performance evaluation metrics for the network product under the scenario.

In another possible implementation manner, the electronic device may acquire, in the operation information, evaluation information for determining all performance evaluation indexes, so that, for a scenario in which the network product is operated, the evaluation information for determining all performance evaluation indexes may be screened out from the evaluation information for determining the performance evaluation indexes in the scenario.

Each performance evaluation index has a corresponding index evaluation algorithm, for example, the utilization index may include a ratio of the number of occupied downlink physical resource blocks to the number of available downlink physical resource blocks, and/or a ratio of the number of occupied uplink physical resource blocks to the number of available uplink physical resource blocks. Therefore, the electronic equipment can calculate and obtain the index value of the performance evaluation index in the scene according to the evaluation information of the performance evaluation index in the scene and the corresponding index evaluation algorithm.

In the platform system, the electronic equipment can determine the index value corresponding to the performance evaluation index of the network product in each scene aiming at each scene in which the network product possibly operates, and can more comprehensively and accurately predict the performance evaluation result of the network product in the commercial network.

S103: and predicting a performance evaluation result of the network product deployed under the commercial network according to the index value of the performance evaluation index.

In one example, the electronic device may directly determine an index value of the performance evaluation index as a performance evaluation result of the network product deployed under the commercial network.

In another example, the electronic device may predict a performance evaluation result of the network product deployment under the commercial network based on the index value of the performance evaluation index and an algorithm for predicting the performance evaluation result. For example, an algorithm for predicting a performance evaluation result may be provided in the electronic device (or a platform system of the electronic device), and an index value of the performance evaluation index may be an input value of the algorithm for predicting the performance evaluation result. The algorithm in the embodiment of the application can be an existing algorithm, a newly designed algorithm or an algorithm improved from the existing algorithm, and is not limited herein.

Alternatively, the algorithm may be provided in the form of a formula in the electronic device.

In yet another example, the electronic device may predict a performance evaluation result of the network product deployment under the commercial network based on the index value of the performance evaluation index and the network name used to predict the performance evaluation result. For example, a network model for predicting the performance evaluation result may be provided in the electronic device (or platform coefficient of the electronic device), and an index value of the performance evaluation index may be used as an input value of the network model for predicting the performance evaluation result. Wherein the network model for predicting the performance evaluation result is a trained network model.

In the application, the electronic equipment can determine the running scene of the network product according to the running information of the network product in the laboratory simulation environment, and then determine the performance evaluation index of the network product in the scene, so that the performance evaluation result of the network product deployed in the commercial network is predicted according to the performance evaluation index, and the application is suitable for the performance evaluation of the network product which is not deployed in the commercial network. And moreover, the running quality of the network product can be obtained before the network product is deployed on a commercial network, so that more accurate performance evaluation and maintenance are performed on the network product, and better use experience is provided for users.

Example 2:

in order to perform performance evaluation on a network product that is not deployed to a commercial network, in the embodiment of the present application, the determining a scenario in which the network product operates includes:

The correspondence between the second network behavior in the commercial network and the commercial network scene may be stored in the electronic device.

In one example, in the platform system, the correspondence between the second network behavior in the commercial network and the commercial network scenario may be used as an input to the platform system. When the network product runs on the platform system, the platform system can search a second network behavior matched with the first network behavior in the input corresponding relation according to the first network behavior of the network product, wherein a commercial network scene corresponding to the second network behavior is the commercial network scene corresponding to the first network behavior, namely, the network product running scene.

Example 3:

in order to further improve the comprehensiveness and accuracy of the performance evaluation of the network product, in the embodiments of the present application, the scenario includes a wireless scenario, an air interface scenario, a load scenario, a service scenario, a network networking scenario, a transmission scenario, or an operation and maintenance scenario.

The embodiment of the application provides seven scene dimensions for evaluating the performance of the network product, and can further improve the comprehensiveness and accuracy of the performance evaluation of the network product.

Scene one: a wireless scenario.

The wireless scenario includes one or more of the following: urban, suburban, rural, and high-speed mobile scenarios.

By way of example, urban scenes may also include one or more of the following: urban walking street scene, urban street scene, indoor driving outdoor scene, outdoor driving indoor scene, subway tunnel scene, elevator scene.

Suburban scenes may also include one or more of the following: medium inter-station distance scene, medium and long distance building shielding scene.

The rural scenario may also include one or more of the following scenarios: a stadia zoom-out scene, a bush blocking scene.

The high speed movement scenario may also include one or more of the following scenarios: a motor train high-speed rail scene, an inter-city expressway scene and an inter-city expressway scene.

Scene II: air interface scene.

The air interface scene includes one or more of the following: a stay strategy scene, an overlay strategy scene and a network roaming scene.

Scene III: load scenario.

Scene four: business scenarios.

The business scenario includes one or more of the following: a transmission control protocol (transmission control protocol, TCP) scenario, a user packet protocol (user datagram protocol, UDP) scenario, a voice scenario, an instant messaging scenario, and a heartbeat probe scenario.

Scene five: network networking scenarios.

The network networking scenario includes one or more of the following: a network sharing scenario, a multi-public land mobile network (public land mobile network, PLMN) scenario, a core network resource pool scenario, a multimode co-sited scenario, a network backup scenario, a first network evolution scenario.

Scene six: the scene is transmitted.

Scene seven: and (5) an operation and maintenance scene.

Accordingly, the commercial network scenario may also include one or more of the scenarios one through seven described above.

The commercial network scene and the second network behavior have a corresponding relation, namely, through carrying out inductive analysis on the operation information of the network product, the corresponding network behavior can be obtained through inductive analysis, so that the scene (or the commercial network scene) corresponding to the network product can be obtained.

The network behavior corresponding to the commercial network scenario is illustrated below.

Commercial network scenario one: a wireless scenario.

1. Urban scenes.

1) Urban walking street scenes.

The scenario includes the following network behaviors:

Street normal fading: the user switching point of the street scene is near the midpoint of the signal quality, the user moving speed is less than 5 kilometers (Km)/hour (h), and multipath channels are arranged on the periphery. The periphery means a range within a set distance centered on the user, the set distance being a positive value.

Right angle turning: the signal variation at the corner of the street is large, and the user is far from the near midpoint abrupt change. Wherein the user is closer to the base station in the case of the near midpoint of the channel quality than the far midpoint of the channel quality, and is farther from the base station in the case of the far midpoint of the channel quality.

2) Urban street scenes.

The scenario includes the following network behaviors:

Street normal fading: the switching point is near the midpoint of the signal quality in the moving process of the user, the moving speed of the user is more than 5Km/h and less than 60Km/h, and multipath channels are arranged on the periphery.

Right angle turning: the signal quality changes by more than 20 decibel milliwatts (decibel relative to one milliwatt, dBm) from near to far point in the course of user movement.

3) And an indoor and outdoor scene is played, namely, a user dials a call of an outdoor user indoors.

The scenario includes the following network behaviors:

blind spot scenario: under the scenes of office buildings and the like, the station cannot be built outdoors, and operators can supplement blind spots.

4) And (3) playing an indoor scene outdoors, namely, a user dials a call of an indoor user outdoors.

The scenario includes the following network behaviors:

the signal position of the user is at the middle and far point of a typical office or residential building, and the signal attenuation of glass or wall-removed is about 20 dBm.

5) Indoor scene.

The scenario includes the following network behaviors:

Indoor room: typically the inter-site distance (i.e., base station distance) is around 20 meters and the user signal quality is at a near midpoint.

Indoor public area: the distance between stations is about 50 meters, and the user is near the middle point in the moving process.

6) Subway tunnel scene.

The scenario includes the following network behaviors:

and a tunnel model, wherein key scene points under the tunnel model are complex multipath channel models.

7) Elevator scenarios.

The scenario includes the following network behaviors:

the user signal quality typically changes from near midpoint to far point, near the drop point, by about 30dBm or more.

2. Suburban scenes.

1) Medium inter-station distance scenarios.

The scenario includes the following network behaviors:

user scene behavior starts moving from a near mid-point under a station and moves to a far point of signal quality, or from a far point to a near mid-point under a station, and the distance between stations is usually more than 1000 meters.

2) The middle-long distance building shields the scene.

The scenario includes the following network behaviors:

user scene behavior starts zooming from near midpoint to midpoint switching.

3. Rural scenarios.

1) The stadia zoom out scene.

The scenario includes the following network behaviors:

user scene behavior switches from a point in the site that is zoomed out to a far point.

2) The shrubs occlude the scene.

The scenario includes the following network behaviors:

The signal is transmitted through the shielding object, the middle and far point burr signal change exists, and the corresponding wireless scene is a rapid mutation with the signal quality of about 10dBm when moving to the middle and far point.

4. The scene is moved at a high speed.

1) A motor train high-speed rail scene.

The scenario includes the following network behaviors:

the moving speed is greater than 200km/h, and the number of peripheral multipath channels is small.

2) Inter-urban expressway scenes.

The scenario includes the following network behaviors:

The moving speed is less than 120km/h, and the number of peripheral multipath channels is small.

3) Inter-city expressway scenes.

The scenario includes the following network behaviors:

the moving speed is 70km/h, and the number of peripheral multipath channels is small.

Commercial network scene two: air interface scene.

1. A stay policy scenario.

1) The rule tends to reside inside.

The scenario includes the following network behaviors:

based on service layering, an operator in the wireless network covers the network in layers at different frequency bands according to service types so as to achieve the purpose of special network application.

Based on the frequency point priority, operators in the wireless network configure different residence and mobile switching priorities based on the frequency points for partial frequency points in order to achieve continuous coverage, prevent coverage holes or network hierarchical management.

Based on the terminal priority, operators in the wireless network configure different residence policies for the terminal type.

2) Inter-system trends stay.

The network behavior in the scene is 1) continuous coverage in the system, and the description of the similarities is omitted.

2. Covering the policy scenario.

1) And (5) continuously covering in a standard mode.

The scenario includes the following network behaviors:

The same frequency continuous coverage and/or the different frequency continuous coverage.

2) And (5) continuously covering the systems.

The scenario includes the following network behaviors:

the 4G network serves as a base floor overlay and the 5G network serves as a hot spot area to absorb high traffic.

4G and 5G cross coverage, wherein the air interface scene belongs to a scene which is not continuously covered in the initial stage of 5G network construction; or the 5G network is used as a private network part, and the boundary area is an air interface scene for maintaining mobility.

4G and 5G layered continuous coverage: typical scene after network construction, the common air interface networking form in the network development process.

3) Macro-micro continuous coverage.

The scenario includes the following network behaviors:

and the macro station continuously covers and builds a 5G network long-term air interface form after the network.

The macro station is covered by the base layer, and the micro station is used as a hot spot to absorb traffic.

3. Network roaming scenario.

The air interface scene of the network roaming comprises the network selection of the roaming terminal and/or the resource control residence strategy of the network for the roaming user.

Commercial network scenario three: load scenario.

1. Urban heavy load scene.

1) Venue event heavy load scene.

Large stadium events, such as sports games and concerts, show dense fluctuating loads, with instantaneous loads exceeding 70%.

2) Commercial centers reload scenes.

Typical scenes in urban areas, such as mall shopping centers, markets and the like, and indoor stations continuously and stably load more than 50%.

3) Walking streets heavily loaded scenes.

Urban outdoor typical scenes such as pedestrian streets, snack streets and the like, and outdoor stations (including outdoor small stations and macro stations) continuously and stably load more than 50%.

2. Urban medium-load scenes.

Common streets are loaded, typical common urban streets, non-people flow concentrated commercial bodies and stable load is 30-50%.

3. Urban light load scene.

The light load scene is when the urban area is idle, such as when the business center is idle, various stadiums are idle, and the service load is less than 30%.

4. And (5) an office park heavy-duty scene.

Indoor heavy load and outdoor peak heavy load in the morning, in the middle and at night, and the network user types are mostly enhanced mobile broadband (enhanced mobile broadband, eMBB) users, typically high-rise office parks, with a stable load of more than 50%.

5. And carrying a scene in the office park.

Indoor medium load and outdoor medium load, the network user type is eMBB users, and the stable load is 30-50%.

6. Industrial factory building heavy load scene.

1) The stable user number of the mass equipment is overloaded, and belongs to a overloading scene of a large number of user numbers, for example, a large number of equipment in a factory building continuously uploads data with fixed bandwidth to a control center and the like, and most network user types are large-scale machine type communication (MASSIVE MACHINE TYPE communication, mMTC) users, and the stable load is more than 50%.

2) The equipment stabilizes the heavy load of service, the network user type is mMTC users, the data with large bandwidth is continuously and stably uploaded to a control center and the like, and the stability load is more than 50%.

7. And carrying a scene in the industrial factory building.

1) The mass equipment stabilizes the user number and carries in the middle load scene that belongs to a large amount of user numbers, for example, the medium equipment in the factory building continuously uploads the data with fixed bandwidth to a control center and the like, the network user type is mMTC users, and the stable load is 30% -50%.

2) The equipment stabilizes the heavy load of service, the network user type is mMTC users mostly, and the stable load is 30% -50%.

8. Industrial factory building light load scene.

1) The stable user number light load of a small amount of equipment belongs to the light load scene of a small amount of user numbers, for example, the small amount of equipment in a factory continuously uploads data with fixed bandwidth to a control center and the like, the type of network user is mMTC users, and the stable load is less than 30%.

2) The equipment stabilizes the heavy load of service, the network user type is mMTC users mostly, and the stable load is less than 30%.

Commercial network scenario four: business scenarios.

1. TCP scene, TCP is the basic network protocol in the current network, and based on the various application layer protocols of TCP evolution, can be decomposed according to the following typical service model:

1) The file transfer protocol (FILE TRANSFER protocol, FTP) big packet full buffer (Fullbuffer) is downloaded for a long time.

2) FTP big packet Fullbuffer is uploaded for a long time.

3) FTP package Fullbuffer is downloaded simultaneously for a long time.

4) FTP package Fullbuffer is downloaded instantaneously.

5) FTP package Fullbuffer is downloaded instantaneously.

6) FTP package Fullbuffer is downloaded instantaneously and simultaneously.

7) The FTP packet uploads the short call.

8) The FTP packet downloads the short call.

9) The hypertext transfer protocol (hyper text transfer protocol, HTTP) is upstream.

10 HTTP downstream.

11 High definition video on demand.

2. UDP scene, UDP is the basic network protocol in the current network, based on UDP evolution multiple application layer protocol, can be decomposed according to the following typical service model:

1) The UDP big packet Fullbuffer is downloaded for a long time.

2) The UDP big packet Fullbuffer is uploaded for a long time.

3) The UDP big packet Fullbuffer is downloaded simultaneously for a long time.

4) The UDP big packet Fullbuffer is downloaded instantaneously.

5) The UDP big packet Fullbuffer is downloaded instantaneously.

6) The UDP big packet Fullbuffer is downloaded instantaneously and simultaneously.

7) And uploading UDP packets.

8) And downloading UDP packets.

3. A speech scene.

1) A voice short call.

2) Voice call is long.

3) Video.

4. Instant messaging scenario.

1) Short-call instant messaging.

2) Long-calling instant messaging.

3) Video instant messaging.

4) Instant messaging.

5. The heartbeat detects the scene.

1) An internet packet detector (PACKET INTERNET groper, ping) packet is used to communicate the continuous heartbeat of both parties.

2) And the Ping big packet is used for continuous communication of both communication parties.

Commercial network scenario five: network networking scenarios.

1. Network sharing scenarios.

The scenario includes the following network behaviors:

1) Wireless devices share, only share wireless device building stations, but do not share wireless spectrum.

2) Full spectrum sharing, wireless base station and full wireless spectrum sharing.

3) Partial spectrum sharing, radio base station and partial radio spectrum sharing.

2. Multi-PLMN scenario.

The scenario includes the following network behaviors:

1) An operator core network contains a plurality of PLMNs.

2) One operator core network contains a plurality of PLMNs and another operator lower single PLMN to form network sharing.

3. Core network resource pool scenario.

4. Multimode co-sited scenario.

The scenario includes the following network behaviors:

1) The 5G independent site and the 5G completely newly built site, and other wireless network related modules are not arranged in the base station.

2) 5G and 4G share a base station, the 5G completely establishes a station or expands a 5G network on the basis of the 4G base station, and other wireless network related modules are not arranged in the base station.

5. Network backup scenarios.

The scenario includes the following network behaviors:

1) Primary and backup between stations: inter-station logical resource active-standby mode.

2) Master and slave in the station: in-station logical resource active-standby mode.

6. A first network evolution scenario.

1) The independent site evolves to the shared site.

2) The independent site evolves to a multi-PLMN site.

3) The shared site evolves to an independent site.

4) The shared site evolves to a multi-PLMN site.

5) Multiple PLMN sites evolve to independent sites.

6) The multi-PLMN site evolves to a multi-PLMN + shared site.

Commercial network scenario six: a transmission scenario, i.e. a transmission mode scenario.

1. Interface type scenario.

1) An optical port, a port for remote data transmission using an optical fiber.

2) Electrical ports, RJ45 based ports.

3) Photoelectric mixing port: and a photoelectric mixing port.

2. Logical interface scenario.

1) Layer 2 (L2) transport networks are based on the L2 paging scheme.

2) L3 transport network, based on route addressing mode.

3. And returning the interface scene.

1) An Ethernet network.

2) An optical fiber.

3) And (5) microwave.

4) A wireless cellular network.

5) And (3) a satellite.

4. And forwarding an interface scene.

1) An Ethernet network.

2) An optical fiber.

5. The base stations concatenate the scenes.

1) Star networking.

2) And (5) chain networking.

3) And (5) loop networking.

6. A radio frequency cascading scenario.

1) And (5) chain networking.

2) And (5) independent networking.

7. Multimode backhaul scenarios.

1) 5G and 4G physical co-transmission.

2) 5G and 4G are transmitted independently.

8. And transmitting the backup scene.

1) Cold backup, failure manually decides link switching.

2) Hot backup, automatic link switching in case of failure.

3) Load balancing, poll (POOL) load balancing approach.

9. A clock source scenario.

1) Global positioning system (global positioning system, GPS).

2) Beidou satellite.

3) Global satellite navigation system (global navigation SATELLITE SYSTEM, GLONASS).

4) Galileo (Galileo).

5)1588V2。

Commercial network scenario seven: and (5) an operation and maintenance scene.

1. Network failure recovery scenario.

1) The recovery of the internal hardware fault of the base station comprises the following steps: base station single board fault recovery, radio frequency module fault recovery and clock fault recovery.

2) The logic fault recovery of the internal software of the base station comprises the following steps: and recovering the service-exiting fault of the base station cell.

3) An interface transmission failure recovery comprising: interface transmission failure recovery, 5G core network (5G core,5 gc) core network equipment failure recovery, network equipment management module link failure recovery.

2. A start scene.

1) A proximal docking station comprising: the base station is directly connected to a station, a palm Application (APP), a client, or the like.

2) And (5) network management remotely opens a station.

3) And (5) starting the station in batches.

3. And upgrading the scene.

1) Proximal upgrades.

2) And upgrading the network management.

3) And upgrading in batches.

4. Hardware operation and maintenance scene.

1) And (5) replacing hardware.

2) Firmware upgrades for hardware bill of materials (BOM), field programmable gate arrays (field programmable GATE ARRAY, FPGA), and the like.

5. A property operation and maintenance scene.

1) The characteristic is on.

2) And (5) characteristic adjustment.

3) The characteristic is deactivated.

4) And upgrading the characteristics.

6. Network operation and maintenance scene.

1) A new network comprising: and (5) creating a hot spot and an area.

2) Network relocation comprising: flower arrangement relocation, hot spot relocation and area relocation.

3) A network extension comprising: hardware capacity expansion, capacity expansion on the original site, and capacity expansion of the new site building site.

7. And a second network evolution scene.

1) The sharing station evolves to the independent station.

2) The independent station evolves to the sharing station.

3) Sharing spectrum to independent spectrum.

4) Independent spectrum to shared spectrum.

5) The shared site evolves to a multi-PLMN site.

6) The independent site evolves to a multi-PLMN site.

7) Multiple PLMN sites evolve to independent sites.

8) The multi-PLMN site evolves to a multi-PLMN + shared site.

9) The core network is moved.

As shown in fig. 2, the network behavior obtained by analysis in the commercial network scene is used as the input of the platform system, wherein the commercial network scene comprises a wireless scene, an air interface scene, a load scene, a service scene, a network networking scene, a transmission scene and an operation and maintenance scene, after the network product operates in the platform system, KPI indexes are monitored according to the operation information of the network product, that is, the performance evaluation of the network product in the scene is represented by the KPI indexes, and finally, the product evaluation conclusion of the network product can be output, that is, the performance evaluation result of the network product is obtained.

Example 4:

in order to further improve accuracy of performance evaluation of the network product, in the embodiments of the present application, the performance evaluation index includes an interference index and/or a channel quality indicator.

The embodiment of the application provides the network environment quality index, and when the performance of the network product is evaluated, the non-network environment quality problem can be distinguished, and the performance evaluation of the network product is more accurate.

The network environment quality indicator includes an interference indicator, and/or a channel quality indicator. Optionally, the interference indicator is characterized by a Resource Block (RB) interference KPI, and the channel quality indicator may be characterized by a channel quality indicator (channel quality indicator, CQI) duty cycle KPI, e.g. a CQI duty cycle comprising a CQI 0-CQI 15 duty cycle.

Optionally, the performance evaluation index may further include a performance evaluation index in the related art, such as a performance evaluation index specified in the 3GPP protocol.

In a specific embodiment, the performance evaluation indicators include access class KPIs, hold class KPIs, mobile class KPIs, service integrity KPIs, utilization class KPIs, availability KPIs, traffic class KPIs. As shown in table 1, the access class KPI includes: radio resource control (radio resource control, RRC) setup success rate (RRC Setup Success Rate), next Generation (NG) interface signaling (NG INTERFACE SIGNALLING, NGSIG) connection setup success rate (NGSIG Connection Setup Success Rate), wherein the interface between the wireless node and the core network is referred to as NG interface, quality of service (quality of service, qoS) flow setup success rate (Qos Flow Setup Success Rate). Maintaining class KPIs includes: service call Drop Rate (SERVICE CALL Drop Rate), RRC connection reestablishment number (RRC AttConnReestab Times), RRC connection reestablishment success Rate (RRC AttConnReestab Success Rate). The mobile KPI comprises: handover out success rate (Handover Out Success Rate), intra-radio access technology handover in success rate (Intra-RAT Handover In Success Rate), intra-radio access technology handover out success rate (Inter-RAT Handover Out Success Rate) (generally referred to as handover from NR to LTE), evolved PACKET SYSTEM Fallback (EPSFB) handover success rate (EPSFB Success Rate Based Handover) (generally referred to as handover from NR to E-UTRAN). The service integrity KPI includes: cell downlink average throughput (Cell Downlink Average Throughput), cell uplink average throughput (Cell Uplink Average Throughput). The utilization class KPIs include: downlink resource block utilization (Downlink Resource Block Utilizing Rate), uplink resource block utilization (Uplink Resource Block Utilizing Rate). Availability KPIs include: radio resource availability (Radio Network Availability Rate). The business class KPI comprises: downlink traffic (Downlink Traffic Volume), uplink traffic (Uplink Traffic Volume), average number of users (Average User Number), maximum number of users (Maximum User Number), RB interference, CQI 0-15.

TABLE 1

/>

In order to perform performance evaluation on a network product before being deployed to a commercial network, the application further provides a performance evaluation device of the network product based on the same technical conception as the method embodiment on the basis of the above embodiments. Fig. 3 is a schematic structural diagram of a performance evaluation device 300 for a network product according to the present application, and as shown in fig. 3, the performance evaluation device 300 for a network product includes:

a first determining unit 301, configured to obtain operation information of a network product, and determine a scenario in which the network product operates;

a second determining unit 302, configured to determine, according to the operation information, an index value of a performance evaluation index of the network product in the scenario;

And the predicting unit 303 is configured to predict a performance evaluation result of the network product deployed under the commercial network according to the index value of the performance evaluation index.

In a possible implementation manner, the first determining unit 301 is specifically configured to determine, according to a first network behavior in the operation information of the network product and a correspondence between a second network behavior in a commercial network and a commercial network scenario, the commercial network scenario corresponding to the first network behavior as the scenario in which the network product is operated.

In one possible implementation, the scenario includes a wireless scenario, an air interface scenario, a load scenario, a traffic scenario, a network networking scenario, a transmission scenario, or an operation and maintenance scenario.

In one possible implementation, the wireless scenario includes one or more of the following: urban area scene, suburban area scene, rural area scene, and high-speed moving scene;

In a possible embodiment, the performance evaluation index comprises an interference index and/or a channel quality indicator.

Fig. 4 is a schematic structural diagram of an electronic device according to the present application, and on the basis of the foregoing embodiments, an embodiment of the present application further provides an electronic device, as shown in fig. 4, including: the processor 401, the communication interface 402, the memory 403 and the communication bus 404, wherein the processor 401, the communication interface 402 and the memory 403 complete communication with each other through the communication bus 404;

the memory 403 has stored therein a computer program which, when executed by the processor 401, causes the processor 401 to perform the steps of:

Further, the processor 401 is specifically configured to determine, according to a first network behavior in the operation information of the network product and a correspondence between a second network behavior in the commercial network and a commercial network scenario, the commercial network scenario corresponding to the first network behavior as the scenario in which the network product operates.

The communication interface 402 is used for communication between the electronic device and other devices described above.

The Memory may include random access Memory (Random Access Memory, RAM) or may include Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.

The processor may be a general-purpose processor, including a central processing unit, a network processor (Network Processor, NP), etc.; but also digital instruction processors (DIGITAL SIGNAL Processing units, DSPs), application specific integrated circuits, field programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

On the basis of the above embodiments, the embodiments of the present application provide a computer readable storage medium having stored therein a computer program executable by a processor, which when run on the processor, causes the processor to perform the steps of:

In a possible implementation manner, the determining the running scene of the network product includes:

Since the principle of solving the problem by using the computer readable medium is similar to that of the performance evaluation method of the network product, the steps implemented after the processor executes the computer program in the computer readable medium can be referred to the other embodiments, and the repetition is omitted.

It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims

1. A method of evaluating performance of a network product, the method comprising:

Acquiring operation information of a network product and determining a scene of operation of the network product, wherein the method comprises the following steps: according to a first network behavior in the operation information of the network product and a corresponding relation between a second network behavior in a commercial network and commercial network scenes, determining the commercial network scenes corresponding to the first network behavior as the scenes in which the network product operates; the network product is a network product before being deployed to a commercial network, and the scene comprises a wireless scene, an air interface scene, a load scene, a service scene, a network networking scene, a transmission scene or an operation and maintenance scene;

2. The method of claim 1, wherein the wireless scene comprises one or more of: urban area scene, suburban area scene, rural area scene, and high-speed moving scene;

3. The method of claim 1, wherein the performance assessment indicator comprises an interference indicator and/or a channel quality indicator.

4. A performance evaluation device for a network product, the device comprising:

The first determining unit is used for acquiring the operation information of the network product, determining the operation scene of the network product, and particularly determining the commercial network scene corresponding to the first network behavior as the operation scene of the network product according to the first network behavior in the operation information of the network product and the corresponding relation between the second network behavior in the commercial network and the commercial network scene; the network product is a network product before being deployed to a commercial network, and the scene comprises a wireless scene, an air interface scene, a load scene, a service scene, a network networking scene, a transmission scene or an operation and maintenance scene;

5. The apparatus of claim 4, wherein the performance evaluation indicator comprises an interference indicator and/or a channel quality indicator.

6. An electronic device comprising at least a processor and a memory, the processor being adapted to perform the steps of the method for evaluating the performance of a network product according to any of claims 1-3 when executing a computer program stored in the memory.

7. A computer-readable storage medium, characterized in that it stores a computer program which, when executed by a processor, performs the steps of the performance evaluation method of a network product according to any one of claims 1-3.