WO2023202067A1 - 对象评估方法、装置、存储介质及电子设备 - Google Patents

对象评估方法、装置、存储介质及电子设备 Download PDF

Info

Publication number
WO2023202067A1
WO2023202067A1 PCT/CN2022/133448 CN2022133448W WO2023202067A1 WO 2023202067 A1 WO2023202067 A1 WO 2023202067A1 CN 2022133448 W CN2022133448 W CN 2022133448W WO 2023202067 A1 WO2023202067 A1 WO 2023202067A1
Authority
WO
WIPO (PCT)
Prior art keywords
evaluation
indicators
level
index
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/133448
Other languages
English (en)
French (fr)
Inventor
张乐
吴艳芹
吕田田
郭蓉蓉
袁晶晶
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Telecom Corp Ltd
Original Assignee
China Telecom Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Telecom Corp Ltd filed Critical China Telecom Corp Ltd
Publication of WO2023202067A1 publication Critical patent/WO2023202067A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063—Operations research, analysis or management
    • G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
    • G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063—Operations research, analysis or management
    • G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/40—Business processes related to the transportation industry

Definitions

  • the present disclosure relates to the field of computer applications, and in particular, to an object evaluation method, an object evaluation device, a storage medium and an electronic device.
  • AI artificial intelligence
  • the purpose of this disclosure is to provide an object evaluation method, object evaluation device, storage medium and electronic equipment, aiming to intelligently evaluate the current telecommunications network management system to provide guidance for network operation and maintenance evolution planning.
  • an object evaluation method including: in response to an evaluation instruction of a target evaluation object, acquiring an initial evaluation model and characteristic information of the target evaluation object; determining the said target evaluation object based on the characteristic information.
  • the final evaluation model is used to perform layer-by-layer aggregation according to the indicator score and the weight information to determine the evaluation result of the target evaluation object.
  • the initial evaluation model includes one or more of the secondary evaluation indicators among intention indicators, acquisition indicators, analysis indicators, decision indicators and execution indicators.
  • the second-level evaluation index includes at least one third-level evaluation index; wherein the intention index includes an intention acquisition index, an intention identification index, an intention analysis index, an intention classification index, an intention
  • the collection indicators include data collection indicators and/or data processing indicators;
  • the analysis indicators include one or more of resource analysis indicators, business analysis indicators, and customer analysis indicators;
  • the decision-making indicators include policy generation indicators and/or execution judgment indicators;
  • the execution indicators include system execution indicators and/or network element execution indicators.
  • the evaluation indicators at the target level are four-level evaluation indicators; wherein the three-level evaluation indicators include at least one of the four-level evaluation indicators.
  • the characteristic information includes: product business information, operation scenario information, and system scope information; wherein the product business information includes government and enterprise oriented categories, family oriented categories, and public individual oriented categories. One or more of categories and internal customer facing categories; the operation scenario information includes one or more of planning and construction, operation preparation and support, implementation delivery, guarantee and billing; the system scope information includes functions One or more of modules, feature sets, and entire systems.
  • the three-level evaluation indicators include at least one of the The four-level evaluation indicators include: the intention acquisition indicators include order acquisition indicators and/or product arrangement package acquisition indicators; the intention identification indicators include order identification indicators; the intention analysis indicators include order decomposition indicators; and the intention classification
  • the indicators include capability mapping indicators; the intention back-translation indicators include completion feedback indicators; the data collection indicators include network element data collection indicators, bandwidth data collection indicators, business performance data collection indicators, pipeline resource data collection indicators, and process data collection One or more indicators and customer satisfaction collection indicators; the data processing indicators include data cleaning indicators and/or data verification indicators; the resource analysis indicators include idle port analysis indicators and/or pipeline resource analysis indicators; so
  • the business analysis indicators include one or more of path calculation indicators, path analysis indicators, and completion test analysis indicators; the customer analysis indicators include activation experience analysis indicators; the strategy generation indicators include optimal path recommendation indicators, completion test analysis indicators, One or more
  • using the final evaluation model to perform layer-by-layer aggregation according to the indicator score and the weight information to determine the evaluation result of the target evaluation object includes: according to the The indicator score determines the fourth-level evaluation results corresponding to each of the four-level evaluation indicators; a weighted sum is performed based on the fourth-level evaluation results and the weight information of each of the four-level evaluation indicators to obtain the third-level evaluation corresponding to each third-level evaluation indicator.
  • the second-level evaluation results corresponding to each second-level evaluation index are obtained; based on the second-level evaluation results and each second-level evaluation index
  • the weight information of the evaluation indicators is weighted and summed to obtain the evaluation result of the target evaluation object.
  • an object evaluation device including: a response module, configured to obtain an initial evaluation model and characteristic information of the target evaluation object in response to an evaluation instruction of the target evaluation object; a determination module , used to determine the evaluation indicators of the target level in the initial evaluation model based on the characteristic information to obtain the final evaluation model, and to determine the weight information of the evaluation indicators at each level in the final evaluation model; a scoring module, used to obtain the The index score corresponding to the evaluation index of the target evaluation object at the target level; an evaluation module, used to use the evaluation model to perform layer-by-layer aggregation according to the index score and the weight information to determine the evaluation of the target evaluation object result.
  • a computer-readable storage medium on which a computer program is stored.
  • the program is executed by a processor, the object evaluation method in the above embodiment is implemented.
  • an electronic device which is characterized in that it includes: one or more processors; a storage device for storing one or more programs.
  • the one or more processors When executed by the one or more processors, the one or more processors are caused to implement the object evaluation method as in the above embodiment.
  • a computer program including: instructions that, when executed by a processor, cause the processor to perform the object evaluation method in the above embodiment.
  • Figure 1 schematically shows a flow chart of an object evaluation method in an exemplary embodiment of the present disclosure
  • Figure 2 schematically shows a composition diagram of an initial evaluation model in an exemplary embodiment of the present disclosure
  • Figure 3 schematically shows a three-dimensional coordinate diagram of a type of feature information in an exemplary embodiment of the present disclosure
  • Figure 4 schematically illustrates a flow chart of a method for determining an evaluation result in an exemplary embodiment of the present disclosure
  • Figure 5 schematically shows the composition of an object evaluation device in an exemplary embodiment of the present disclosure
  • Figure 6 schematically shows a schematic diagram of a computer-readable storage medium in an exemplary embodiment of the present disclosure
  • FIG. 7 schematically shows a structural diagram of a computer system of an electronic device in an exemplary embodiment of the present disclosure.
  • Example embodiments will now be described more fully with reference to the accompanying drawings.
  • Example embodiments may, however, be embodied in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concepts of the example embodiments. To those skilled in the art.
  • AI artificial intelligence
  • ITU-T released a standard recommendation in 2020 on a framework for evaluating the intelligence level of future networks including IMT (IEMeeting, Emmett (Conference Equipment)-2020). It applies to the entire network and its operation and maintenance systems. However, , it only provides the theory of the method and does not propose how to evaluate the content of the telecom operation management system level.
  • TMF TeleManagement Forum, Telecommunications Management Forum
  • IG1218 Autonomous Network Business Requirements and Architecture
  • the present disclosure provides an object evaluation method, which can construct an analytic hierarchy process evaluation model based on the analytic hierarchy process model for the evaluation objects of the telecommunications operation management system in the IP virtual private network activation scenario, and Develop an indicator weight determination method to obtain the intelligent level classification (Level 0-Level 5) of the evaluation object at each level as the evaluation result.
  • Figure 1 schematically shows a flow chart of an object evaluation method in an exemplary embodiment of the present disclosure.
  • the object evaluation method includes steps S101 to S104:
  • Step S101 in response to the evaluation instruction of the target evaluation object, obtain the initial evaluation model and the characteristic information of the target evaluation object;
  • Step S102 Determine the evaluation indicators of the target level in the initial evaluation model based on the characteristic information to obtain the final evaluation model, and determine the weight information of the evaluation indicators at each level in the final evaluation model;
  • Step S103 Obtain the index score corresponding to the evaluation index of the target evaluation object at the target level
  • Step S104 Use the evaluation model to perform layer-by-layer aggregation based on the indicator score and the weight information to determine the evaluation result of the target evaluation object.
  • the indicator scores in the previous level are calculated; the previous level is determined as the current level, and the above steps are repeated until the indicators in the highest level of the evaluation model are calculated. Scoring; determine the evaluation result of the target evaluation object based on the indicator score in the highest level.
  • the target level can be determined as the initial current level, and the above-mentioned layer-by-layer aggregation process can be performed.
  • an initial evaluation model is first created, and after responding to the evaluation instruction, the characteristic information of the target evaluation object is automatically obtained; and then the target level in the evaluation model is determined based on the characteristic information of the target evaluation object.
  • Corresponding evaluation indicators are used to obtain a complete final evaluation model, and the weight information of each evaluation indicator in the final evaluation model is determined; finally, the final evaluation model is used to gather the evaluation results layer by layer, so that a suitable and accurate evaluation can be carried out. Optimize the evaluation results to provide more accurate guidance for system optimization.
  • step S101 in response to an evaluation instruction of a target evaluation object, an initial evaluation model and characteristic information of the target evaluation object are obtained.
  • an initial evaluation model that meets the evaluation needs of most evaluation objects can be created in advance.
  • the initial evaluation model can include multiple levels of evaluation indicators, which are recorded as second-level, third-level and other evaluation indicators in turn.
  • Each second-level evaluation indicator includes At least one third-level evaluation indicator, and so on for the others.
  • FIG. 2 schematically shows a composition diagram of an initial evaluation model in an exemplary embodiment of the present disclosure.
  • the analytic hierarchy process evaluation model can be divided into four levels.
  • the first layer is used to store the characteristic information and evaluation results of the target evaluation object.
  • the evaluation results can be obtained by aggregating and calculating the evaluation results corresponding to the secondary evaluation indicators, and the evaluation results are to be determined before the evaluation.
  • the second layer includes secondary evaluation indicators
  • the second-level evaluation index is the evaluation index of the intent-driven workflow closed loop, including the five-dimensional closed-loop workflow from intention to execution, which can be obtained by aggregating and calculating the evaluation results corresponding to the third-level evaluation index.
  • the analytic hierarchy process evaluation model includes one or more of the secondary evaluation indicators among intention indicators, acquisition indicators, analysis indicators, decision-making indicators and execution indicators.
  • the third layer includes three-level evaluation indicators
  • the third-level evaluation index is 14 evaluation sub-dimensions with general characteristics described based on the second-level evaluation index, which can be obtained by aggregating and calculating the evaluation results corresponding to the fourth-level evaluation index.
  • the intention indicators include one or more of intention acquisition indicators, intention identification indicators, intention analysis indicators, intention classification indicators, and intention back-translation indicators;
  • the collection indicators include data collection indicators and/or data processing indicators;
  • the analysis indicators include one or more of resource analysis indicators, business analysis indicators, and customer analysis indicators;
  • the decision-making indicators include strategy generation indicators and/or execution judgment indicators;
  • the execution indicators include system execution indicators and/or Network element execution indicators.
  • the fourth level includes to-be-determined four-level evaluation indicators
  • four-level evaluation indicators need to be defined based on the evaluation needs corresponding to different characteristic information of the evaluation object, and attached with rating rules. Different target evaluation objects have different characteristic information, so the four-level evaluation indicators here are to be determined and can only be confirmed after obtaining the characteristic information of the target evaluation object.
  • the target-level evaluation index is a fourth-level evaluation index; for example, the third-level evaluation index includes at least one of the fourth-level evaluation index. That is to say, after responding to the evaluation instruction of a certain target evaluation object, it is necessary to determine which fourth-level evaluation indicators are included in the third-level evaluation indicators based on the characteristic information of the target evaluation object.
  • the characteristic information includes: product business information, operation scenario information, and system scope information.
  • FIG. 3 schematically shows a three-dimensional coordinate diagram of a type of feature information in an exemplary embodiment of the present disclosure.
  • the characteristic information of the target evaluation object mainly consists of three parts, namely product business, operation scenario, and system scope, which correspond to the x-axis, y-axis, and z-axis in Figure 3 respectively. Therefore, three-point focusing on the three axes of x, y, and z can uniquely determine an evaluation object, for example, marked with point coordinates, which can be recorded as (1, 1, 1).
  • the product business information includes one or more of the categories for government and enterprises, for families, for the public and for individuals, and for internal customers. Below, it is introduced in detail:
  • VPN Virtual Private Network
  • 5G slicing IDC (Internet Data Center, Internet Data Center), etc.
  • CDN Content Delivery Network
  • broadband access etc.
  • Businesses for internal customers This type of business provides convenient daily operations and work solutions for internal customers, such as corporate offices, etc.
  • the operation scenario information includes one or more of planning and construction, operation preparation and support, implementation delivery, guarantee and billing, which are described in detail below:
  • Plan and Build is focused on the development and delivery of systems, computing resources, and network capabilities, including: defining, planning, and implementing all necessary infrastructure (such as applications, computing, and networks), as well as all other supporting infrastructure and business capabilities (such as partnerships, etc.). Plan and build products that identify needs, capabilities, and design and develop new or enhanced infrastructure to support operations
  • Operational preparation and support responsible for providing human resource allocation, management process and other management support for operations, and making operational preparations for the implementation of delivery, guarantee and billing benefits.
  • the call center directly faces customers, the configuration of back-end operation and maintenance personnel, front-line real-time response, and back-end operation processes, etc.
  • SLA Service-Level Agreement, service level agreement
  • QoS Quality of Service, providing service capabilities for designated network communications
  • Billing Responsible for collecting customer product usage, determining charging and billing information, generating timely and accurate bills, providing customers with pre-billing usage information and billing, and processing their payments. Able to provide customer statements, provide bill inquiry status, and solve billing problems in a timely manner to satisfy customers. Prepayment is also supported.
  • the system-wide information includes one or more of functional modules, functional sets, and the entire system.
  • the system includes multiple function sets, and each function set is composed of different functional modules.
  • each function set is composed of different functional modules.
  • step S102 the evaluation indicators of the target level in the initial evaluation model are determined based on the characteristic information to obtain the final evaluation model, and the weight information of the evaluation indicators of each level in the final evaluation model is determined.
  • the evaluation indicators at the target level in the initial evaluation model are determined, that is, the four-level evaluation indicators included in each three-level evaluation indicator are determined.
  • the characteristic information of the target evaluation object is that the product business information is an IP virtual private network for government and enterprises
  • the operation scenario information is the service provisioning sub-scenario in implementation delivery
  • the system scope information is the entire system
  • the three-level evaluation indicators include the four-level evaluation indicators:
  • the intention acquisition indicators include order acquisition indicators and/or product orchestration package acquisition indicators;
  • the intention identification indicators include order identification indicators;
  • the intention analysis indicators include order decomposition indicators;
  • the intention classification indicators include capability mapping indicators;
  • Intention back-translation indicators include as-built feedback indicators;
  • the data collection indicators include one or more of network element data collection indicators, bandwidth data collection indicators, business performance data collection indicators, pipeline resource data collection indicators, process data collection indicators, and customer satisfaction collection indicators;
  • Data processing indicators include data cleaning indicators and/or data verification indicators;
  • the resource analysis indicators include idle port analysis indicators and/or pipeline resource analysis indicators;
  • the business analysis indicators include one or more of path calculation indicators, path analysis indicators, and completion test analysis indicators;
  • the customer analysis indicators Including activation experience analysis indicators;
  • the strategy generation indicators include one or more of optimal path recommendation indicators, completion test plan generation indicators, and strategy simulation verification indicators; the execution judgment indicators include path configuration decision indicators and/or completion test decision indicators;
  • the system execution indicators include configuration command issuance indicators and/or test command issuance indicators; the network element execution indicators include equipment network management execution indicators and/or equipment execution indicators.
  • the evaluation rules for the indicators must also be formulated so that the corresponding indicator scores for the indicators can be obtained later.
  • the analytic hierarchy process is used to determine the weight of each evaluation indicator. Determining the weights involves the following steps.
  • Step 2 After constructing the hierarchically sorted single-layer judgment matrix, calculate the maximum eigenvalue ⁇ max of the judgment matrix F and its corresponding eigenvectors, and then normalize the eigenvectors to obtain the importance of each indicator in the same layer relative to an indicator in the upper layer. In order to obtain the ranking weight of each indicator at the lowest level for the overall goal, the total ranking of the hierarchy is calculated. The total ranking weight combines the single ranking weights of the hierarchy from top to bottom.
  • Step 3 Consistency check.
  • the consistency ratio of the judgment matrix C R is determined by the ratio of the consistency index C 1 and the degree of freedom index R 1.
  • step S103 the index score corresponding to the evaluation index of the target evaluation object at the target level is obtained.
  • the target-level evaluation indicators are four-level evaluation indicators. According to the established evaluation rules, the indicator scores corresponding to each four-level evaluation indicator are determined based on expert experience.
  • step S104 the evaluation model is used to perform layer-by-layer aggregation according to the indicator score and the weight information to determine the evaluation result of the target evaluation object.
  • FIG. 4 schematically illustrates a flow chart of a method for determining an evaluation result in an exemplary embodiment of the present disclosure. As shown in Figure 4, determining the evaluation results includes the following steps:
  • Step S401 Determine the four-level evaluation results corresponding to each of the four-level evaluation indicators based on the indicator scores
  • Step S402 Perform a weighted sum based on the four-level evaluation results and the weight information of each of the four-level evaluation indicators to obtain the three-level evaluation results corresponding to each three-level evaluation indicator;
  • Step S403 Perform a weighted sum based on the third-level evaluation results and the weight information of each of the third-level evaluation indicators to obtain the second-level evaluation results corresponding to each second-level evaluation indicator;
  • Step S404 Perform a weighted sum based on the secondary assessment results and the weight information of each secondary assessment index to obtain an assessment result of the target assessment object.
  • the evaluation results can use the intelligence level of Level 0-Level 5 to describe the evaluation results of the target evaluation object.
  • the scores of the fourth-level evaluation indicators can first be mapped to the level rating results, and then the weighted summation is performed based on the weight information of the evaluation indicators, and the smart level grades of each third-level and second-level evaluation indicators are calculated in turn. , and finally the intelligence level of the target evaluation object is obtained.
  • Table 2 The calculation results are shown in Table 2:
  • the expert scores of the four-level evaluation indicators are mapped to the level rating results, and then the hierarchical model ratings are aggregated layer by layer based on the weight information in the final evaluation model to calculate the IP virtual private network service.
  • the intelligence level evaluation and classification result of the entire system is 1.06.
  • the characteristic information of the target evaluation object is different, and the evaluation results of the final evaluation model and the intelligence level are different.
  • Table 3 shows:
  • the intelligence level is 1.85.
  • an intelligent level evaluation framework is constructed based on the analytic hierarchy process model, the evaluation indicators are defined and decomposed layer by layer, and the weights of indicators at each level are further determined to provide a complete set of system intelligent level evaluation methods and devices. It can promote the intelligent evolution of the entire operation management system and guide the implementation of the intelligent evolution of the system, which has strong guiding significance; on the other hand, based on the method of determining the three-dimensional coordinate evaluation object corresponding to the characteristic information, it can clearly define the telecom operation management system. Intelligence level evaluation is inseparable from the prerequisites of product business and operational scenarios.
  • FIG. 5 schematically shows the composition of an object evaluation device in an exemplary embodiment of the present disclosure.
  • the object evaluation device 500 may include a response module 501 , a determination module 502 , a scoring module 503 and an evaluation module 504 . in:
  • the response module 501 is configured to respond to the evaluation instruction of the target evaluation object and obtain the initial evaluation model and the characteristic information of the target evaluation object;
  • Determining module 502 configured to determine the evaluation indicators of the target level in the initial evaluation model based on the characteristic information to obtain the final evaluation model, and determine the weight information of the evaluation indicators at each level in the final evaluation model;
  • the scoring module 503 is used to obtain the index score corresponding to the evaluation index of the target evaluation object at the target level;
  • the evaluation module 504 is configured to use the evaluation model to perform layer-by-layer aggregation according to the indicator score and the weight information to determine the evaluation result of the target evaluation object.
  • the initial evaluation model includes one or more of the secondary evaluation indicators among intent indicators, collection indicators, analysis indicators, decision indicators and execution indicators.
  • the second-level evaluation index includes at least one third-level evaluation index; for example, the intent index includes an intent acquisition index, an intent identification index, an intent parsing index, an intent classification index, and an intent back-translation index.
  • the collection indicators include data collection indicators and/or data processing indicators
  • the analysis indicators include one or more of resource analysis indicators, business analysis indicators, and customer analysis indicators
  • the decision-making The indicators include policy generation indicators and/or execution judgment indicators
  • the execution indicators include system execution indicators and/or network element execution indicators.
  • the target-level evaluation index is a fourth-level evaluation index; for example, the third-level evaluation index includes at least one of the fourth-level evaluation index.
  • the characteristic information includes: product business information, operation scenario information, and system scope information; for example, the product business information includes: oriented to government and enterprise categories, oriented to family categories, oriented to public individuals, and oriented to One or more of the internal customer categories; the operation scenario information includes one or more of planning and construction, operation preparation and support, implementation delivery, guarantee and billing; the system-wide information includes functional modules, functions One or more of a collection and an entire system.
  • the three-level evaluation indicators include at least one of the four-level Evaluation indicators include: the intention acquisition indicators include order acquisition indicators and/or product orchestration package acquisition indicators; the intention identification indicators include order identification indicators; the intention analysis indicators include order decomposition indicators; the intention classification indicators include capabilities Mapping indicators; the intention back-translation indicators include completion feedback indicators; the data collection indicators include network element data collection indicators, bandwidth data collection indicators, business performance data collection indicators, pipeline resource data collection indicators, process data collection indicators, customer One or more of the satisfaction collection indicators; the data processing indicators include data cleaning indicators and/or data verification indicators; the resource analysis indicators include idle port analysis indicators and/or pipeline resource analysis indicators; the business analysis The indicators include one or more of path calculation indicators, path analysis indicators, and completion test analysis indicators; the customer analysis indicators include activation experience analysis indicators; the strategy generation indicators include optimal path recommendation indicators, and completion test plan generation One or more of indicators and strategy simulation verification indicators
  • the evaluation module 504 is configured to determine a fourth-level evaluation result corresponding to each of the four-level evaluation indicators according to the indicator score; based on the fourth-level evaluation result and each of the four-level evaluation Perform a weighted summation of the weight information of the indicators to obtain the third-level evaluation results corresponding to each third-level evaluation indicator; perform a weighted sum based on the third-level evaluation results and the weight information of each of the third-level evaluation indicators to obtain the corresponding second-level evaluation indicators.
  • the secondary evaluation results based on the secondary evaluation results and the weight information of each of the secondary evaluation indicators, a weighted sum is performed to obtain the evaluation result of the target evaluation object.
  • a storage medium capable of implementing the above method is also provided.
  • Figure 6 schematically shows a schematic diagram of a computer-readable storage medium in an exemplary embodiment of the present disclosure.
  • a program product 600 for implementing the above method according to an embodiment of the present disclosure is described, which can It adopts a portable compact disk read-only memory (CD-ROM) and includes program code, and can be run on a terminal device such as a mobile phone.
  • CD-ROM compact disk read-only memory
  • the program product of the present disclosure is not limited thereto.
  • a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.
  • FIG. 7 schematically shows a structural diagram of a computer system of an electronic device in an exemplary embodiment of the present disclosure.
  • the computer system 700 includes a central processing unit (Central Processing Unit, CPU) 701, which can be loaded into a random accessory according to a program stored in a read-only memory (Read-Only Memory, ROM) 702 or from a storage part 708. Access the program in the memory (Random Access Memory, RAM) 703 to perform various appropriate actions and processes. In RAM 703, various programs and data required for system operation are also stored.
  • CPU 701, ROM 702 and RAM 703 are connected to each other through bus 704.
  • An input/output (I/O) interface 705 is also connected to bus 704.
  • the following components are connected to the I/O interface 705: an input part 706 including a keyboard, a mouse, etc.; an output part 707 including a cathode ray tube (Cathode Ray Tube, CRT), a liquid crystal display (Liquid Crystal Display, LCD), etc., and a speaker, etc. ; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN (Local Area Network) card, a modem, etc.
  • the communication section 709 performs communication processing via a network such as the Internet.
  • Driver 710 is also connected to I/O interface 705 as needed.
  • Removable media 711 such as magnetic disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage portion 708 as needed.
  • embodiments of the present disclosure include a computer program product including a computer program carried on a computer-readable medium, the computer program containing program code for performing the method illustrated in the flowchart.
  • the computer program may be downloaded and installed from the network via communication portion 709 and/or installed from removable media 711 .
  • this computer program is executed by the central processing unit (CPU) 701, various functions defined in the system of the present disclosure are performed.
  • the computer-readable medium shown in the embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two.
  • the computer-readable storage medium may be, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof.
  • Computer readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard drive, random access memory (RAM), read only memory (ROM), removable Programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), flash memory, optical fiber, portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage device, magnetic storage device, or any of the above suitable The combination.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above.
  • a computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device .
  • Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logic functions that implement the specified executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown one after another may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved.
  • each block in the block diagram or flowchart illustration, and combinations of blocks in the block diagram or flowchart illustration can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or may be implemented by special purpose hardware-based systems that perform the specified functions or operations. Achieved by a combination of specialized hardware and computer instructions.
  • the units involved in the embodiments of the present disclosure can be implemented in software or hardware, and the described units can also be provided in a processor. Among them, the names of these units do not constitute a limitation on the unit itself under certain circumstances.
  • the present disclosure also provides a computer-readable medium.
  • the computer-readable medium may be included in the electronic device described in the above embodiments; it may also exist independently without being assembled into the electronic device. middle.
  • the computer-readable medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiments.
  • the example embodiments described here can be implemented by software, or can be implemented by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, U disk, mobile hard disk, etc.) or on the network , including several instructions to cause a computing device (which may be a personal computer, a server, a touch terminal, a network device, etc.) to execute the method according to the embodiments of the present disclosure.
  • a computing device which may be a personal computer, a server, a touch terminal, a network device, etc.

Landscapes

  • Business, Economics & Management (AREA)
  • Human Resources & Organizations (AREA)
  • Engineering & Computer Science (AREA)
  • Strategic Management (AREA)
  • Economics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Development Economics (AREA)
  • Educational Administration (AREA)
  • General Physics & Mathematics (AREA)
  • Marketing (AREA)
  • Tourism & Hospitality (AREA)
  • Physics & Mathematics (AREA)
  • General Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • Operations Research (AREA)
  • Quality & Reliability (AREA)
  • Game Theory and Decision Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Primary Health Care (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

本公开涉及计算机应用领域，尤其涉及一种对象评估方法、对象评估装置、存储介质及电子设备。该对象评估方法包括响应于目标评估对象的评估指令，获取初始评估模型以及所述目标评估对象的特征信息；基于所述特征信息确定所述初始评估模型中目标层级的评估指标以得到最终评估模型，以及确定所述最终评估模型中各层级评估指标的权重信息；获取所述目标评估对象在所述目标层级的评估指标对应的指标评分；根据所述指标评分和所述权重信息利用所述最终评估模型进行层层汇聚以确定所述目标评估对象的评估结果。

Description

对象评估方法、装置、存储介质及电子设备
相关申请的交叉引用
本申请是以CN申请号为202210411597.3，申请日为2022年4月19日的申请为基础，并主张其优先权，该CN申请的公开内容在此作为整体引入本申请中。
技术领域
本公开涉及计算机应用领域，尤其涉及一种对象评估方法、对象评估装置、存储介质及电子设备。
背景技术
随着网络的不断发展和5G(5th Generation Mobile Communication Technology，第五代移动通信技术)时代的到来，运营商的网络越来越复杂，业务需求逐渐多样化，导致电信5G网络运营管理复杂化不断加重。引入人工智能(AI)技术来增强电信5G网络运营管理系统的智能化和自动化，将对现有的运营和管理系统框架产生重大影响。
评估当前电信网络管理系统的水平有助于运营商评估自身智能水平，为网络运维演进规划提供指导。目前在ITU-T(国际电联电信标准化部门)及其它标准组织中对评估自治或智能网络水平的理论进行了研究。
需要说明的是，在上述背景技术部分公开的信息仅用于加强对本公开的背景的理解，因此可以包括不构成对本领域普通技术人员已知的现有技术的信息。
发明内容
本公开的目的在于提供一种对象评估方法、对象评估装置、存储介质及电子设备，旨在对当前电信网络管理系统进行智能评估，以为网络运维演进规划提供指导。
本公开的其他特性和优点将通过下面的详细描述变得显然，或部分地通过本公开的实践而习得。
根据本公开实施例的一方面，提供了一种对象评估方法，包括：响应于目标评估对象的评估指令，获取初始评估模型以及所述目标评估对象的特征信息；基于所述特征信息确定所述初始评估模型中目标层级的评估指标以得到最终评估模型，以及确定所述最终评估模型中各层级评估指标的权重信息；获取所述目标评估对象在所述目标 层级的评估指标对应的指标评分；根据所述指标评分和所述权重信息利用所述最终评估模型进行层层汇聚以确定所述目标评估对象的评估结果。
根据本公开的一些实施例，基于前述方案，所述初始评估模型包括意图指标、采集指标、分析指标、决策指标和执行指标中的一种或多种所述二级评估指标。
根据本公开的一些实施例，基于前述方案，所述二级评估指标包括至少一个三级评估指标；其中，所述意图指标包括意图获取指标、意图识别指标、意图解析指标、意图分类指标、意图反译指标中的一种或多种；所述采集指标包括数据采集指标和/或数据处理指标；所述分析指标包括资源分析指标、业务分析指标、客户分析指标中的一种或多种；所述决策指标包括策略生成指标和/或执行判断指标；所述执行指标包括系统执行指标和/或网元执行指标。
根据本公开的一些实施例，基于前述方案，所述目标层级的评估指标为四级评估指标；其中，所述三级评估指标包括至少一个所述四级评估指标。
根据本公开的一些实施例，基于前述方案，所述特征信息包括：产品业务信息、运营场景信息、系统范围信息；其中，所述产品业务信息包括面向政企类、面向家庭类、面向公众个人类和面向内部客户类中的一种或多种；所述运营场景信息包括规划建设、运营准备和支持、实施交付、保障和计费中的一种或多种；所述系统范围信息包括功能模块、功能集合和整个系统中的一种或多种。
根据本公开的一些实施例，基于前述方案，在所述特征信息为面向政企类中的IP虚拟专网、实施交付中的业务开通和整个系统时，所述三级评估指标包括至少一个所述四级评估指标，包括：所述意图获取指标包括订单获取指标和/或产品编排包获取指标；所述意图识别指标包括订单识别指标；所述意图解析指标包括订单分解指标；所述意图分类指标包括能力映射指标；所述意图反译指标包括竣工反馈指标；所述数据采集指标包括网元数据采集指标、带宽数据采集指标、业务性能数据采集指标、管线资源数据采集指标、流程类数据采集指标、客户满意度采集指标中的一种或多种；所述数据处理指标包括数据清洗指标和/或数据核查指标；所述资源分析指标包括空闲端口分析指标和/或管线资源分析指标；所述业务分析指标包括路径计算指标、路径分析指标、竣工测试分指标析中的一种或多种；所述客户分析指标包括开通体验分析指标；所述策略生成指标包括最优路径推荐指标、竣工测试方案生成指标、策略仿真验证指标中的一种或多种；所述执行判断指标包括路径配置决策指标和/或竣工测试决策指标；所述系统执行指标包括配置命令下发指标和/或测试命令下发指标；所述网元执行指标 包括设备网管执行指标和/或设备执行指标。
根据本公开的一些实施例，基于前述方案，所述根据所述指标评分和所述权重信息利用所述最终评估模型进行层层汇聚以确定所述目标评估对象的评估结果，包括：根据所述指标评分确定各所述四级评估指标对应的四级评估结果；基于所述四级评估结果和各所述四级评估指标的权重信息进行加权求和得到各三级评估指标对应的三级评估结果；基于所述三级评估结果和各所述三级评估指标的权重信息进行加权求和得到各二级评估指标对应的二级评估结果；基于所述二级评估结果和各所述二级评估指标的权重信息进行加权求和得到所述目标评估对象的评估结果。
根据本公开实施例的第二方面，提供了一种对象评估装置，包括：响应模块，用于响应于目标评估对象的评估指令，获取初始评估模型以及所述目标评估对象的特征信息；确定模块，用于基于所述特征信息确定所述初始评估模型中目标层级的评估指标以得到最终评估模型，以及确定所述最终评估模型中各层级评估指标的权重信息；评分模块，用于获取所述目标评估对象在所述目标层级的评估指标对应的指标评分；评估模块，用于跟据所述指标评分和所述权重信息利用所述评估模型进行层层汇聚以确定所述目标评估对象的评估结果。
根据本公开实施例的第三方面，提供了一种计算机可读存储介质，其上存储有计算机程序，所述程序被处理器执行时实现如上述实施例中的对象评估方法。
根据本公开实施例的第四方面，提供了一种电子设备，其特征在于，包括：一个或多个处理器；存储装置，用于存储一个或多个程序，当所述一个或多个程序被所述一个或多个处理器执行时，使得所述一个或多个处理器实现如上述实施例中的对象评估方法。
根据本公开实施例的第五方面，提供了一种计算机程序，包括：指令，所述指令当由处理器执行时使所述处理器执行上述实施例中的对象评估方法。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本公开。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理。显而易见地，下面描述中的附图仅仅是本公开的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下， 还可以根据这些附图获得其他的附图。在附图中：
图1示意性示出本公开示例性实施例中一种对象评估方法的流程示意图；
图2示意性示出本公开示例性实施例中一种初始评估模型的组成示意图；
图3示意性示出本公开示例性实施例中一种特征信息的三维坐标示意图；
图4示意性示出本公开示例性实施例中一种确定评估结果方法的流程示意图；
图5示意性示出本公开示例性实施例中一种对象评估装置的组成示意图；
图6示意性示出本公开示例性实施例中一种计算机可读存储介质的示意图；
图7示意性示出本公开示例性实施例中一种电子设备的计算机系统的结构示意图。
具体实施方式
现在将参考附图更全面地描述示例实施方式。然而，示例实施方式能够以多种形式实施，且不应被理解为限于在此阐述的范例；相反，提供这些实施方式使得本公开将更加全面和完整，并将示例实施方式的构思全面地传达给本领域的技术人员。
此外，所描述的特征、结构或特性可以以任何合适的方式结合在一个或更多实施例中。在下面的描述中，提供许多具体细节从而给出对本公开的实施例的充分理解。然而，本领域技术人员将意识到，可以实践本公开的技术方案而没有特定细节中的一个或更多，或者可以采用其它的方法、组元、装置、步骤等。在其它情况下，不详细示出或描述公知方法、装置、实现或者操作以避免模糊本公开的各方面。
附图中所示的方框图仅仅是功能实体，不一定必须与物理上独立的实体相对应。即，可以采用软件形式来实现这些功能实体，或在一个或多个硬件模块或集成电路中实现这些功能实体，或在不同网络和/或处理器装置和/或微控制器装置中实现这些功能实体。
附图中所示的流程图仅是示例性说明，不是必须包括所有的内容和操作/步骤，也不是必须按所描述的顺序执行。例如，有的操作/步骤还可以分解，而有的操作/步骤可以合并或部分合并，因此实际执行的顺序有可能根据实际情况改变。
随着网络的不断发展和5G时代的到来，运营商的网络越来越复杂，业务需求逐渐多样化，导致电信5G网络运营管理复杂化不断加重。引入人工智能(AI)技术来增强电信5G网络运营管理系统的智能化和自动化，将对现有的运营和管理系统框架产生重大影响。需要考虑如何满足网络智能化的要求，实现更高效、更经济、更灵活的网络运维和服务管理。评估当前电信网络管理系统的水平有助于运营商评估自身智 能水平，为网络运维演进规划提供指导。
目前在ITU-T及其它标准组织中对评估自治或智能网络水平的理论进行了研究，但是目前没有一套完整的电信运营管理系统智能等级的评估研究。且目前针对网络智能等级的研究只给出了Level 0-Level 5级别及其基本特征的指导，没有给出评估测量维度或参数，难以用于电信网络运营管理系统智能化等级评估中。
ITU-T于2020年发布了一项关于评估包括IMT(IEMeeting，艾米特(会议设备)-2020在内的未来网络智能水平的框架的标准建议。适用于全网及其运维系统。但是，它仅提供了该方法的理论，并没有提出如何评估电信运营管理系统级别的内容。在2021年2月的TMF(TeleManagement Forum，电信管理论坛)论坛提出了关于自治网络级别的标准文稿。该标准的范围是定义基于TMF论坛IG1218(自治网络业务需求和架构)的自治网络级别。描述了自治网络级的概念，包括自治网络级的方法论和方法、操作流程、底层子流程和任务、任务评估标准。目前标准文件内容主要集中在智能水平理论方面，未涉及运营管理方面的内容。
也就是说，目前没有一套完整的电信运营管理系统智能等级的评估研究。且目前针对网络智能等级的研究只给出了0-5级(Level 0-Level 5)级别及其基本特征的指导，没有给出评估测量维度或参数，难以用于电信网络运营管理系统智能化等级评估中。
因此，针对现有技术中存在的问题，本公开提供了一种对象评估方法，能够针对面向IP虚拟专网开通场景下电信运营管理系统的评估对象，基于层次分析模型构建层次分析评估模型，并制定指标权重确定方法，用于得到评估对象在每一级上的智能级别分类(Level 0-Level 5)作为评估结果。
以下对本公开实施例的技术方案的实现细节进行详细阐述。
图1示意性示出本公开示例性实施例中一种对象评估方法的流程示意图。如图1所示，该对象评估方法包括步骤S101至步骤S104：
步骤S101，响应于目标评估对象的评估指令，获取初始评估模型以及所述目标评估对象的特征信息；
步骤S102，基于所述特征信息确定所述初始评估模型中目标层级的评估指标以得到最终评估模型，以及确定所述最终评估模型中各层级评估指标的权重信息；
步骤S103，获取所述目标评估对象在所述目标层级的评估指标对应的指标评分；
步骤S104，据所述指标评分和所述权重信息利用所述评估模型进行层层汇聚以确 定所述目标评估对象的评估结果。
在一些实施例中，根据评估模型的当前层级中的指标评分，计算上一层级中的指标评分；将上一层级确定为当前层级，重复上述步骤，直到计算出评估模型的最高层级中的指标评分；根据最高层级中的指标评分，确定目标评估对象的评估结果。例如，可以将目标层级确定为初始的当前层级，进行上述层层汇聚处理。
在本公开的一些实施例所提供的技术方案中，首先创建初始评估模型，并在响应评估指令之后，自动获取目标评估对象的特征信息；之后根据目标评估对象的特征信息确定评估模型中目标层级对应的评估指标以得到完整的最终评估模型，并且确定最终评估模型中各评估指标的权重信息；最后再利用确定的最终评估模型层层汇聚得到评估结果，能够对其进行适合、准确的评估，优化评估结果，为系统优化提供更加精准的指导。
下面，将结合附图及实施例对本示例实施方式中的对象评估方法的各个步骤进行更详细的说明。
在步骤S101中，响应于目标评估对象的评估指令，获取初始评估模型以及所述目标评估对象的特征信息。
在一些实施例中，在接收到评估指令后，首先需要获取预先创建的初始评估模型。
例如，可以预先创建的一个符合大多数评估对象评估需求的初始评估模型，初始评估模型可以包括多个层级的评估指标，依次记为二级、三级等评估指标，每一个二级评估指标包括至少一个三级评估指标，其他依次类推。
图2示意性示出本公开示例性实施例中一种初始评估模型的组成示意图。如图2所示，该层次分析评估模型可分为四层。
1)第一层用于存储该目标评估对象的特征信息及评估结果
例如，评估结果可以由二级评估指标对应的评估结果汇聚计算求得，在未评估之前评估结果待定。
2)第二层包括二级评估指标
例如，二级评估指标为意图驱动工作流程闭环的评估指标，包括从意图到执行的五维闭环工作流程，可以由三级评估指标对应的评估结果汇聚计算求得。例如，所述层次分析评估模型包括意图指标、采集指标、分析指标、决策指标和执行指标中的一种或多种所述二级评估指标。
3)第三层包括三级评估指标
例如，三级评估指标是根据二级评估指标分解的具有通用特征描述的14个评估子维度，可以由四级评估指标对应的评估结果汇聚计算求得。
例如，所述意图指标包括意图获取指标、意图识别指标、意图解析指标、意图分类指标、意图反译指标中的一种或多种；所述采集指标包括数据采集指标和/或数据处理指标；所述分析指标包括资源分析指标、业务分析指标、客户分析指标中的一种或多种；所述决策指标包括策略生成指标和/或执行判断指标；所述执行指标包括系统执行指标和/或网元执行指标。
4)第四层包括待定的四级评估指标
例如，四级评估指标是需要根据评估对象不同的特征信息所对应的评估需求来定义四级评估指标，并附评级规则。不同的目标评估对象，特征信息不同，因此此处的四级评估指标待定，需要获取到目标评估对象的特征信息之后才能确认。
因此，在一些实施例中，所述目标层级的评估指标为四级评估指标；例如，所述三级评估指标包括至少一个所述四级评估指标。也就是说，在响应于某个目标评估对象的评估指令之后，需要根据该目标评估对象的特征信息来确定三级评估指标包括哪些四级评估指标。
在一些实施例中，还需要获取目标评估对象的特征信息。例如，所述特征信息包括：产品业务信息、运营场景信息、系统范围信息。
图3示意性示出本公开示例性实施例中一种特征信息的三维坐标示意图。如图3所示，目标评估对象的特征信息主要由三部分构成，即产品业务、运营场景、系统范围，分别对应于图3中的x轴、y轴和z轴。因此，x，y，z三个轴上的三点聚焦可唯一确定一个评估对象，例如用点坐标标记，可记为(1，1，1)。
例如，所述产品业务信息包括面向政企类、面向家庭类、面向公众个人类和面向内部客户类中的一种或多种。下面，对其进行详细介绍：
面向政企类的业务：此类服务提供为企业构建的解决方案。通过行业领先的商业服务推动业务向前发展。与公共部门合作，重新构想学习、实现政府现代化和改造网络。典型产品有VPN(Virtual Private Network，虚拟专用网络)产品、5G切片、IDC(Internet Data Center，互联网数据中心)等。
面向家庭类的业务：该类服务负责打造居家娱乐、教育等服务。典型的业务有CDN(Content Delivery Network，内容分发网络)、宽带接入等。
面向公众个人类的业务：此类业务需要提供快速可靠的服务解决方案，专为小型 业务设计，如移动定位、5G消息等。
面向内部客户类的业务：此类业务为内部客户提供便捷的日常运营、工作解决方案，如企业办公等。
例如，所述运营场景信息包括规划建设、运营准备和支持、实施交付、保障和计费中的一种或多种，下面，对其进行详细介绍：
规划建设：规划建设是关注系统、计算资源以及网络能力的开发和交付，包括：定义、规划和实施所有必要的基础设施(如应用程序、计算和网络)，以及所有其他支持基础设施和业务能力(如合作伙伴关系等)。规划建设确定需求、能力并设计和开发新的或增强的基础设施以支持运营的产品
运营准备和支持：负责为运营提供人力资源配置、管理流程等管理支持，为实施交付、保障和计费收益做好运营准备。比如，产品运维时，直接面向客户的呼叫中心，后端运维人员的配置以及一线实时响应、后端的操作流程等。
实施交付：负责以及时和正确的方式向客户提供其所需的产品。它从客户购买到运营商交付符合客户需求的产品，告知客户订单的状态，按时完成，并确保客户满意。
保障：负责执行主动和被动维护优化活动，以确保向客户提供持续可用的产品，保障SLA(Service-Level Agreement，服务等级协议)或QoS(Quality of Service，为指定的网络通信提供服务能力)性能。保障客户感知、业务质量、网络性能，主动检测故障并排除，预测潜在问题并进行优化。向客户报告服务质量，接收来自客户申告，通知客户故障状态，并确保恢复和维修，确保客户满意。
计费：负责收集客户产品的使用情况，确定收费和计费信息，生成及时准确的账单，向客户提供账单前使用信息和计费，处理他们的付款。能提供客户对账单，提供账单查询状态，及时解决账单问题，使客户满意。同时也支持预付费。
需要说明的是，上述五个运营场景可以根据实际运营情况进一步细化为运营子场景，如在实施交付场景中包括开通子场景。
例如，所述系统范围信息包括功能模块、功能集合和整个系统中的一种或多种。
这个是指在评估时考虑的维度是如何，系统中包括多个功能集合，而每个功能集合又是由不同的功能模块构成。在进行评估时，可以只针对某产品业务在某运营场景内的单一功能模块的维度来评价，也可以基于整个系统架构来进行评级，评估更为灵活。
在步骤S102中，基于所述特征信息确定所述初始评估模型中目标层级的评估指 标以得到最终评估模型，以及确定所述最终评估模型中各层级评估指标的权重信息。
在一些实施例中，确定初始评估模型中目标层级的评估指标，也就是确定各三级评估指标分别包括的四级评估指标。
举例来说，当目标评估对象的特征信息中，产品业务信息为面向政企类中的IP虚拟专网，运营场景信息为实施交付中的业务开通子场景，系统范围信息为整个系统时，其对应的最终评估模型如表1所示。
表1最终评估模型
Figure PCTCN2022133448-appb-000001
例如，三级评估指标包括的四级评估指标分别为：
所述意图获取指标包括订单获取指标和/或产品编排包获取指标；所述意图识别指标包括订单识别指标；所述意图解析指标包括订单分解指标；所述意图分类指标包括能力映射指标；所述意图反译指标包括竣工反馈指标；
所述数据采集指标包括网元数据采集指标、带宽数据采集指标、业务性能数据采集指标、管线资源数据采集指标、流程类数据采集指标、客户满意度采集指标中的一种或多种；所述数据处理指标包括数据清洗指标和/或数据核查指标；
所述资源分析指标包括空闲端口分析指标和/或管线资源分析指标；所述业务分析指标包括路径计算指标、路径分析指标、竣工测试分指标析中的一种或多种；所述客户分析指标包括开通体验分析指标；
所述策略生成指标包括最优路径推荐指标、竣工测试方案生成指标、策略仿真验证指标中的一种或多种；所述执行判断指标包括路径配置决策指标和/或竣工测试决策指标；
所述系统执行指标包括配置命令下发指标和/或测试命令下发指标；所述网元执行指标包括设备网管执行指标和/或设备执行指标。
除了确定评估指标之外，还要制定该指标的评估规则，以便后续得到该指标对应的指标评分。
需要说明的是，根据评估对象不同的特征信息需要制定不同的四级评估指标以及评估规则，所以可以预先进行制定，完成后将特征信息和相应的四级评估指标以及评估规则进行映射存储，之后再获取了特征信息之后即可映射出该特征信息对应的四级评估指标以及评估规则。
在一些实施例中，还需要确定最终评估模型中各层级评估指标的权重信息。
例如，由于评估模型是构建的具有多个层级的评估指标，因此采用层次分析法确定各评估指标的权重。确定权重包括以下步骤。
步骤1：构造判断矩阵。设某层指标为F＝[f 1,f 2,…,f n]，两两比较f i与f j对上层某个指标的影响重要程度，以f ij＝f i/f j表示，f ij的值用数字1～9及其倒数进行量化，最后构成判断矩阵F(f ij) n×n。
步骤2：层次排序单层判断矩阵构造后，计算判断矩阵F最大特征值λ max及其对应的特征向量，然后将特征向量归一化处理，获得同层各指标相对于上层某一指标重要性的排序权值，为获得最底层各指标对于总目标的排序权重，计算层次总排序，总 排序权重从上至下将层次单排序权重进行合成。
步骤3：一致性检验。判断矩阵C R的一致性比例由一致性指标C 1与自由度指标R 1的比值确定，当C R＜0.1时，则判断矩阵的一致性在合理范围，否则应对其进行修正，C 1＝(λ max()(n-1))，C R＝C 1/R 1。
步骤4：判断矩阵权重求解。对判断矩阵采用特征向量法求解权重向量，其中FW＝λ′ max，求解W′得到W′ l＝[w′ l,1,w′ l,2,…,w′ l,n] T，对W′进行归一化，根据下式可得某层各指标对于它上层指标的相对权重，如公式(1)所示：
Figure PCTCN2022133448-appb-000002
在步骤S103中，获取所述目标评估对象在所述目标层级的评估指标对应的指标评分。
在一些实施例中，目标层级的评估指标即为四级评估指标，根据制定的评估规则，基于专家经验确定各四级评估指标对应的指标评分。
当然，除了专家打分的方式，也可以采用其他的评价方式，例如基于测试题目打分或评级，本公开的实施例仅做示例性说明，并不能限制本公开。
在步骤S104中，根据所述指标评分和所述权重信息利用所述评估模型进行层层汇聚以确定所述目标评估对象的评估结果。
图4示意性示出本公开示例性实施例中一种确定评估结果方法的流程示意图。如图4所示，该确定评估结果包括以下步骤：
步骤S401，根据所述指标评分确定各所述四级评估指标对应的四级评估结果；
步骤S402，基于所述四级评估结果和各所述四级评估指标的权重信息进行加权求和得到各三级评估指标对应的三级评估结果；
步骤S403，基于所述三级评估结果和各所述三级评估指标的权重信息进行加权求和得到各二级评估指标对应的二级评估结果；
步骤S404，基于所述二级评估结果和各所述二级评估指标的权重信息进行加权求和得到所述目标评估对象的评估结果。
例如，评估结果可以采用Level 0-Level 5的智能级别用来描述目标评估对象的评估结果。在实际操作时，首先可以将四级评估指标的评分先映射为级别评级结果，然后再根据评估指标的权重信息分别进行加权求和，依次计算出各三级、二级评估指标的智能级别等级，最终得到该目标评估对象的智能级别等级，计算的结果如表2所示：
表2智能级别等级评估结果
Figure PCTCN2022133448-appb-000003
如表2所示，首先将四级评估指标的专家评分映射为级别评级结果，然后根据最终评估模型中的权重信息对层级模型评级进行层层汇聚，计算得到IP虚拟专网业务，在开通运营场景下，整个系统的智能化等级评估分类结果，即智能级别等级为1.06。
在其他实施例中，目标评估对象的特征信息不同，最终评估模型以及智能级别等 级的评估结果不同。例如表3所示：
表3智能级别等级评估结果
Figure PCTCN2022133448-appb-000004
如表3所示，在特征信息为：产品业务信息为IP虚拟专网，运营场景信息为VPN 业务保障，系统范围信息为整个系统时，智能化等级为1.85级。
因此，基于上述方法，一方面，基于层次分析模型构建智能等级评估框架，对评估指标层层定义分解，并进一步确定各级指标权重，给出一套完整的系统智能化等级评估方法和装置，可以推动整个运营管理系统的智能化演进，并指导系统智能化演进落地实施，具有较强的指导意义；另一方面，基于特征信息对应的三维坐标评估对象确定方法，明确对电信运营管理系统进行智能等级评估离不开产品业务和运营场景的先决条件。
图5示意性示出本公开示例性实施例中一种对象评估装置的组成示意图。如图5所示，该对象评估装置500可以包括响应模块501、确定模块502、评分模块503以及评估模块504。其中：
响应模块501，用于响应于目标评估对象的评估指令，获取初始评估模型以及所述目标评估对象的特征信息；
确定模块502，用于基于所述特征信息确定所述初始评估模型中目标层级的评估指标以得到最终评估模型，以及确定所述最终评估模型中各层级评估指标的权重信息；
评分模块503，用于获取所述目标评估对象在所述目标层级的评估指标对应的指标评分；
评估模块504，用于跟据所述指标评分和所述权重信息利用所述评估模型进行层层汇聚以确定所述目标评估对象的评估结果。
根据本公开的示例性实施例，所述初始评估模型包括意图指标、采集指标、分析指标、决策指标和执行指标中的一种或多种所述二级评估指标。
根据本公开的示例性实施例，所述二级评估指标包括至少一个三级评估指标；例如，所述意图指标包括意图获取指标、意图识别指标、意图解析指标、意图分类指标、意图反译指标中的一种或多种；所述采集指标包括数据采集指标和/或数据处理指标；所述分析指标包括资源分析指标、业务分析指标、客户分析指标中的一种或多种；所述决策指标包括策略生成指标和/或执行判断指标；所述执行指标包括系统执行指标和/或网元执行指标。
根据本公开的示例性实施例，所述目标层级的评估指标为四级评估指标；例如，所述三级评估指标包括至少一个所述四级评估指标。
根据本公开的示例性实施例，所述特征信息包括：产品业务信息、运营场景信息、系统范围信息；例如，所述产品业务信息包括面向政企类、面向家庭类、面向公众个 人类和面向内部客户类中的一种或多种；所述运营场景信息包括规划建设、运营准备和支持、实施交付、保障和计费中的一种或多种；所述系统范围信息包括功能模块、功能集合和整个系统中的一种或多种。
根据本公开的示例性实施例，在所述特征信息为面向政企类中的IP虚拟专网、实施交付中的业务开通和整个系统时，所述三级评估指标包括至少一个所述四级评估指标，包括：所述意图获取指标包括订单获取指标和/或产品编排包获取指标；所述意图识别指标包括订单识别指标；所述意图解析指标包括订单分解指标；所述意图分类指标包括能力映射指标；所述意图反译指标包括竣工反馈指标；所述数据采集指标包括网元数据采集指标、带宽数据采集指标、业务性能数据采集指标、管线资源数据采集指标、流程类数据采集指标、客户满意度采集指标中的一种或多种；所述数据处理指标包括数据清洗指标和/或数据核查指标；所述资源分析指标包括空闲端口分析指标和/或管线资源分析指标；所述业务分析指标包括路径计算指标、路径分析指标、竣工测试分指标析中的一种或多种；所述客户分析指标包括开通体验分析指标；所述策略生成指标包括最优路径推荐指标、竣工测试方案生成指标、策略仿真验证指标中的一种或多种；所述执行判断指标包括路径配置决策指标和/或竣工测试决策指标；所述系统执行指标包括配置命令下发指标和/或测试命令下发指标；所述网元执行指标包括设备网管执行指标和/或设备执行指标。
根据本公开的示例性实施例，所述评估模块504用于根据所述指标评分确定各所述四级评估指标对应的四级评估结果；基于所述四级评估结果和各所述四级评估指标的权重信息进行加权求和得到各三级评估指标对应的三级评估结果；基于所述三级评估结果和各所述三级评估指标的权重信息进行加权求和得到各二级评估指标对应的二级评估结果；基于所述二级评估结果和各所述二级评估指标的权重信息进行加权求和得到所述目标评估对象的评估结果。
上述的对象评估装置500中各模块的具体细节已经在对应的xxx方法中进行了详细的描述，因此此处不再赘述。
应当注意，尽管在上文详细描述中提及了用于动作执行的设备的若干模块或者单元，但是这种划分并非强制性的。实际上，根据本公开的实施方式，上文描述的两个或更多模块或者单元的特征和功能可以在一个模块或者单元中具体化。反之，上文描述的一个模块或者单元的特征和功能可以进一步划分为由多个模块或者单元来具体化。
在本公开的示例性实施例中，还提供了一种能够实现上述方法的存储介质。图6示意性示出本公开示例性实施例中一种计算机可读存储介质的示意图，如图6所示，描述了根据本公开的实施方式的用于实现上述方法的程序产品600，其可以采用便携式紧凑盘只读存储器(CD-ROM)并包括程序代码，并可以在终端设备，例如手机上运行。然而，本公开的程序产品不限于此，在本文件中，可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。
在本公开的示例性实施例中，还提供了一种能够实现上述方法的电子设备。图7示意性示出本公开示例性实施例中一种电子设备的计算机系统的结构示意图。
需要说明的是，图7示出的电子设备的计算机系统700仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图7所示，计算机系统700包括中央处理单元(Central Processing Unit，CPU)701，其可以根据存储在只读存储器(Read-Only Memory，ROM)702中的程序或者从存储部分708加载到随机访问存储器(Random Access Memory，RAM)703中的程序而执行各种适当的动作和处理。在RAM 703中，还存储有系统操作所需的各种程序和数据。CPU 701、ROM 702以及RAM 703通过总线704彼此相连。输入/输出(Input/Output，I/O)接口705也连接至总线704。
以下部件连接至I/O接口705：包括键盘、鼠标等的输入部分706；包括诸如阴极射线管(Cathode Ray Tube，CRT)、液晶显示器(Liquid Crystal Display，LCD)等以及扬声器等的输出部分707；包括硬盘等的存储部分708；以及包括诸如LAN(Local Area Network，局域网)卡、调制解调器等的网络接口卡的通信部分709。通信部分709经由诸如因特网的网络执行通信处理。驱动器710也根据需要连接至I/O接口705。可拆卸介质711，诸如磁盘、光盘、磁光盘、半导体存储器等等，根据需要安装在驱动器710上，以便于从其上读出的计算机程序根据需要被安装入存储部分708。
特别地，根据本公开的实施例，下文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信部分709从网络上被下载和安装，和/或从可拆卸介质711被安装。在该计算机程序被中央处理单元(CPU)701执行时，执行本公开的系统中限定的各种功能。
需要说明的是，本公开实施例所示的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(Erasable Programmable Read Only Memory，EPROM)、闪存、光纤、便携式紧凑磁盘只读存储器(Compact Disc Read-Only Memory，CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：无线、有线等等，或者上述的任意合适的组合。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，上述模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图或流程图中的每个方框、以及框图或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现，所描述的单元也可以设置在处理器中。其中，这些单元的名称在某种情况下并不构成对该单元本身的限定。
作为另一方面，本公开还提供了一种计算机可读介质，该计算机可读介质可以是 上述实施例中描述的电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被一个该电子设备执行时，使得该电子设备实现上述实施例中所述的方法。
应当注意，尽管在上文详细描述中提及了用于动作执行的设备的若干模块或者单元，但是这种划分并非强制性的。实际上，根据本公开的实施方式，上文描述的两个或更多模块或者单元的特征和功能可以在一个模块或者单元中具体化。反之，上文描述的一个模块或者单元的特征和功能可以进一步划分为由多个模块或者单元来具体化。
通过以上的实施方式的描述，本领域的技术人员易于理解，这里描述的示例实施方式可以通过软件实现，也可以通过软件结合必要的硬件的方式来实现。因此，根据本公开实施方式的技术方案可以以软件产品的形式体现出来，该软件产品可以存储在一个非易失性存储介质(可以是CD-ROM，U盘，移动硬盘等)中或网络上，包括若干指令以使得一台计算设备(可以是个人计算机、服务器、触控终端、或者网络设备等)执行根据本公开实施方式的方法。
本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本公开的其它实施方案。本公开旨在涵盖本公开的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本公开的一般性原理并包括本公开未公开的本技术领域中的公知常识或惯用技术手段。
应当理解的是，本公开并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本公开的范围仅由所附的权利要求来限制。

Claims (15)

  1. 一种对象评估方法，包括：
    响应于目标评估对象的评估指令，获取初始评估模型以及所述目标评估对象的特征信息；
    基于所述特征信息，确定所述初始评估模型中目标层级的评估指标，以得到最终评估模型，以及确定所述最终评估模型中各层级评估指标的权重信息；
    获取所述目标评估对象在所述目标层级的评估指标对应的指标评分；
    根据所述指标评分和所述权重信息，利用所述最终评估模型进行层层汇聚，以确定所述目标评估对象的评估结果。
  2. 根据权利要求1所述的对象评估方法，其中，所述初始评估模型包括二级评估指标，所述二级评估指标包括意图指标、采集指标、分析指标、决策指标和执行指标中的一种或多种。
  3. 根据权利要求2所述的对象评估方法，其中：
    所述二级评估指标包括至少一个三级评估指标；
    所述意图指标包括的三级评估指标包括意图获取指标、意图识别指标。
  4. 根据权利要求3所述的对象评估方法，其中，所述意图指标包括的三级评估指标包括意图解析指标、意图分类指标或意图反译指标中的一种或多种。
  5. 根据权利要求3所述的对象评估方法，其中：
    所述采集指标包括的三级评估指标包括数据采集指标或数据处理指标中的一种或多种；
    所述分析指标包括的三级评估指标包括资源分析指标、业务分析指标或客户分析指标中的一种或多种；
    所述决策指标包括的三级评估指标包括策略生成指标或执行判断指标中的一种或多种；
    所述执行指标包括的三级评估指标包括系统执行指标或网元执行指标中的一种或多种。
  6. 根据权利要求3～5任一项所述的对象评估方法，其中，所述目标层级的评估指标为四级评估指标，所述三级评估指标包括至少一个所述四级评估指标。
  7. 根据权利要求1所述的对象评估方法，其中：
    所述特征信息包括产品业务信息、运营场景信息、系统范围信息；
    所述产品业务信息包括面向政企类、面向家庭类、面向公众个人类或面向内部客户类中的一种或多种；
    所述运营场景信息包括规划建设、运营准备和支持、实施交付或保障和计费中的一种或多种；
    所述系统范围信息包括功能模块、功能集合或整个系统中的一种或多种。
  8. 根据权利要求6所述的对象评估方法，其中，在所述特征信息包括面向政企类中的IP虚拟专网、实施交付中的业务开通和整个系统的情况下，所述意图获取指标包括的四级评估指标包括订单获取指标或产品编排包获取指标中的至少一个，所述意图识别指标包括的四级评估指标包括订单识别指标。
  9. 根据权利要求6所述的对象评估方法，其中，在所述特征信息包括面向政企类中的IP虚拟专网、实施交付中的业务开通和整个系统的情况下，所述意图解析指标包括的四级评估指标包括订单分解指标，所述意图分类指标包括的四级评估指标包括能力映射指标，所述意图反译指标包括的四级评估指标包括竣工反馈指标。
  10. 根据权利要求6所述的对象评估方法，其中：
    所述数据处理指标包括的三级评估指标包括数据清洗指标或数据核查指标中的一种或多种；
    所述资源分析指标包括的三级评估指标包括空闲端口分析指标或管线资源分析指标中的一种或多种；
    所述业务分析指标包括的三级评估指标包括路径计算指标、路径分析指标或竣工测试分指标析中的一种或多种；
    所述客户分析指标包括的三级评估指标包括开通体验分析指标；
    所述策略生成指标包括的三级评估指标包括最优路径推荐指标、竣工测试方案生成指标或策略仿真验证指标中的一种或多种；
    所述执行判断指标包括的三级评估指标包括路径配置决策指标或竣工测试决策指标中的一种或多种；
    所述系统执行指标包括的三级评估指标包括配置命令下发指标或测试命令下发指标中的一种或多种；
    所述网元执行指标包括的三级评估指标包括设备网管执行指标或设备执行指标中的一种或多种。
  11. 根据权利要求6所述的对象评估方法，其中，所述根据所述指标评分和所述权重信息，利用所述最终评估模型进行层层汇聚，以确定所述目标评估对象的评估结果，包括：
    根据所述指标评分确定各所述四级评估指标对应的四级评估结果；
    基于所述四级评估结果和各所述四级评估指标的权重信息进行加权求和得到各三级评估指标对应的三级评估结果；
    基于所述三级评估结果和各所述三级评估指标的权重信息进行加权求和得到各二级评估指标对应的二级评估结果；
    基于所述二级评估结果和各所述二级评估指标的权重信息进行加权求和得到所述目标评估对象的评估结果。
  12. 一种对象评估装置，包括：
    响应模块，用于响应于目标评估对象的评估指令，获取初始评估模型以及所述目标评估对象的特征信息；
    确定模块，用于基于所述特征信息确定所述初始评估模型中目标层级的评估指标以得到最终评估模型，以及确定所述最终评估模型中各层级评估指标的权重信息；
    评分模块，用于获取所述目标评估对象在所述目标层级的评估指标对应的指标评分；
    评估模块，用于跟据所述指标评分和所述权重信息，利用所述最终评估模型进行层层汇聚以确定所述目标评估对象的评估结果。
  13. 一种计算机可读存储介质，其上存储有计算机程序，所述程序被处理器执行时实现如权利要求1至11任一项所述的对象评估方法。
  14. 一种电子设备，包括：
    一个或多个处理器；
    存储装置，用于存储一个或多个程序，当所述一个或多个程序被所述一个或多个处理器执行时，使得所述一个或多个处理器实现如权利要求1至11任一项所述的对象评估方法。
  15. 一种计算机程序，包括：
    指令，所述指令当由处理器执行时使所述处理器执行根据权利要求1至11任一项所述的对象评估方法。
PCT/CN2022/133448 2022-04-19 2022-11-22 对象评估方法、装置、存储介质及电子设备 Ceased WO2023202067A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210411597.3 2022-04-19
CN202210411597.3A CN116976713A (zh) 2022-04-19 2022-04-19 对象评估方法、装置、存储介质及电子设备

Publications (1)

Publication Number Publication Date
WO2023202067A1 true WO2023202067A1 (zh) 2023-10-26

Family

ID=88418986

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/133448 Ceased WO2023202067A1 (zh) 2022-04-19 2022-11-22 对象评估方法、装置、存储介质及电子设备

Country Status (2)

Country Link
CN (1) CN116976713A (zh)
WO (1) WO2023202067A1 (zh)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117235610A (zh) * 2023-11-16 2023-12-15 一网互通(北京)科技有限公司 社交媒体内容数据分段方法、装置及电子设备
CN117634942A (zh) * 2023-11-10 2024-03-01 南京审计大学 不确定审计评估数据的融合方法及装置、设备、介质
CN117745129A (zh) * 2023-12-05 2024-03-22 武汉国信映盛互动技术有限公司 一种新型用户等级评价管理方法、装置、电子设备及介质
CN118195329A (zh) * 2024-05-20 2024-06-14 苔花科迈(西安)信息技术有限公司 煤矿安全生产智能风险识别的多维融合处理方法及系统
CN118211228A (zh) * 2024-03-07 2024-06-18 中国电子科技集团公司第十五研究所 一种信息技术产品组合的安全性增长评估方法和评估系统
CN118964340A (zh) * 2024-07-18 2024-11-15 中国移动通信集团设计院有限公司 一种网络资源数据的质量评估方法、装置、介质及产品
CN119312923A (zh) * 2024-09-30 2025-01-14 苏州元脑智能科技有限公司 一种推理服务评估方法、装置、计算机设备和存储介质
CN119536959A (zh) * 2025-01-23 2025-02-28 中国船舶集团有限公司第七〇七研究所 基于评估层次任务网络的任务处理方法、装置及存储介质
CN121217226A (zh) * 2025-11-25 2025-12-26 浙江信测通信股份有限公司 一种光纤业务承载能力评估方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6625511B1 (en) * 1999-09-27 2003-09-23 Hitachi, Ltd. Evaluation method and its apparatus of work shop and product quality
US20100082362A1 (en) * 2008-09-17 2010-04-01 Baker Salsbury Method and Apparatus for Assessing Salient Characteristics of a Community
CN103761630A (zh) * 2014-02-20 2014-04-30 上海正信方晟资信评估有限公司 一种基于模糊逻辑处理的评估算法
CN112613664A (zh) * 2020-12-25 2021-04-06 武汉理工大学 基于水上交通事故风险预测与评估的预警方法和系统
CN113988476A (zh) * 2021-11-26 2022-01-28 苏交科集团股份有限公司 一种针对道路运输安全风险的动态评估预测方法

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111985751B (zh) * 2019-05-23 2023-09-26 百度在线网络技术（北京）有限公司 人机聊天体验评估体系
CN113850476A (zh) * 2021-08-27 2021-12-28 中国电力科学研究院有限公司 一种区域综合能源系统规划方案的仿真评估方法和系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6625511B1 (en) * 1999-09-27 2003-09-23 Hitachi, Ltd. Evaluation method and its apparatus of work shop and product quality
US20100082362A1 (en) * 2008-09-17 2010-04-01 Baker Salsbury Method and Apparatus for Assessing Salient Characteristics of a Community
CN103761630A (zh) * 2014-02-20 2014-04-30 上海正信方晟资信评估有限公司 一种基于模糊逻辑处理的评估算法
CN112613664A (zh) * 2020-12-25 2021-04-06 武汉理工大学 基于水上交通事故风险预测与评估的预警方法和系统
CN113988476A (zh) * 2021-11-26 2022-01-28 苏交科集团股份有限公司 一种针对道路运输安全风险的动态评估预测方法

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117634942A (zh) * 2023-11-10 2024-03-01 南京审计大学 不确定审计评估数据的融合方法及装置、设备、介质
CN117235610A (zh) * 2023-11-16 2023-12-15 一网互通(北京)科技有限公司 社交媒体内容数据分段方法、装置及电子设备
CN117235610B (zh) * 2023-11-16 2024-02-23 一网互通(北京)科技有限公司 社交媒体内容数据分段方法、装置及电子设备
CN117745129A (zh) * 2023-12-05 2024-03-22 武汉国信映盛互动技术有限公司 一种新型用户等级评价管理方法、装置、电子设备及介质
CN118211228A (zh) * 2024-03-07 2024-06-18 中国电子科技集团公司第十五研究所 一种信息技术产品组合的安全性增长评估方法和评估系统
CN118195329A (zh) * 2024-05-20 2024-06-14 苔花科迈(西安)信息技术有限公司 煤矿安全生产智能风险识别的多维融合处理方法及系统
CN118964340A (zh) * 2024-07-18 2024-11-15 中国移动通信集团设计院有限公司 一种网络资源数据的质量评估方法、装置、介质及产品
CN119312923A (zh) * 2024-09-30 2025-01-14 苏州元脑智能科技有限公司 一种推理服务评估方法、装置、计算机设备和存储介质
CN119536959A (zh) * 2025-01-23 2025-02-28 中国船舶集团有限公司第七〇七研究所 基于评估层次任务网络的任务处理方法、装置及存储介质
CN121217226A (zh) * 2025-11-25 2025-12-26 浙江信测通信股份有限公司 一种光纤业务承载能力评估方法

Also Published As

Publication number Publication date
CN116976713A (zh) 2023-10-31

Similar Documents

Publication Publication Date Title
WO2023202067A1 (zh) 对象评估方法、装置、存储介质及电子设备
Vakili et al. Comprehensive and systematic review of the service composition mechanisms in the cloud environments
CN107194743B (zh) 一种网络调查问卷生成方法和装置
US11687352B2 (en) Machine-learning models applied to interaction data for determining interaction goals and facilitating experience-based modifications to interface elements in online environments
US11769099B2 (en) Apparatuses, computer-implemented methods, and computer program products for improved monitored building environment monitoring and scoring
CN110070391A (zh) 数据处理方法、装置、计算机可读介质及电子设备
WO2018157808A1 (zh) 互联网征信评估方法和系统
CN113298121B (zh) 基于多数据源建模的消息发送方法、装置和电子设备
CN113037877A (zh) 云边端架构下时空数据及资源调度的优化方法
Le et al. Multicriteria decision making with fuzziness and criteria interdependence in cloud service selection
JP6049799B2 (ja) 経営支援プログラム
CN112017042A (zh) 基于tweedie分布的资源配额确定方法、装置和电子设备
CN119558865B (zh) 数控机床后市场服务项目管理方法及管理系统
CN106970953A (zh) 基于大数据分析的网络建设管控方法及其平台
CN112488865A (zh) 基于金融时间节点的金融风险预测方法、装置和电子设备
Li et al. TSLAM: a trust-enabled self-learning agent model for service matching in the cloud market
CN118469529A (zh) 就业信息匹配管理方法、系统、设备及存储介质
Cheng et al. Modeling the maximum perceived utility consensus based on prospect theory
CN117875768A (zh) 一种面向资质审核的远程人资绩效管理系统及方法
US20200226688A1 (en) Computer-readable recording medium recording portfolio presentation program, portfolio presentation method, and information processing apparatus
CN116226531A (zh) 一种小微企业金融产品智能推荐方法及相关产品
Chen Individual Mobile Communication Services and Tariffs
CN112966873B (zh) 工业园区综合能源系统优化运行方法、装置、存储介质
Kokshenev et al. A web-based decision support center for electrical energy companies
CN112015978A (zh) 一种定制信息发送方法、装置和电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22938286

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22938286

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 17/07/2025)

122 Ep: pct application non-entry in european phase

Ref document number: 22938286

Country of ref document: EP

Kind code of ref document: A1