CN118427108A

CN118427108A - Operating skill testing method and system for SaaS software

Info

Publication number: CN118427108A
Application number: CN202410889029.3A
Authority: CN
Inventors: 王克飞; 徐超; 应春红; 应立锋
Original assignee: Puhuizhizao Technology Co ltd
Current assignee: Puhuizhizao Technology Co ltd
Priority date: 2024-07-04
Filing date: 2024-07-04
Publication date: 2024-08-02
Anticipated expiration: 2044-07-04
Also published as: CN118427108B

Abstract

The application discloses a method and a system for testing operating skills of software as a service (SaaS), wherein the method comprises the following steps: the server receives user behavior data and real operation service data sent by the client, wherein the user behavior data is human-computer interaction data monitored when various operation events are preset in SaaS software in the process of responding to a target test paper by a user, and the real operation service data is data related to service logic generated when the user triggers various operation events; generating operation scores of the users according to the user behavior data and the standard behavior data of the target test paper; generating a data score of the user according to the actual operation service data of the user and the standard service data of the target test paper; and carrying out weighted summation on the operation score and the data score to obtain the operation proficiency of the user on the SaaS software. Therefore, by adopting the embodiment of the application, the complex work requirements in the SaaS software can be covered comprehensively, and the performance of the user in a complex operation flow can be accurately captured and estimated, so that the accuracy of the operation skill test is improved.

Description

Operating skill testing method and system for SaaS software

Technical Field

The application relates to the technical field of computers, in particular to an operating skill testing method and system for SaaS software.

Background

With the rapid development of information technology, the SaaS (Software AS A SERVICE ) mode has become one of the main modes of Software deployment and use. The SaaS mode allows users to access and use software via the internet without the need for local installation and maintenance. However, in the implementation of SaaS software, the implementation personnel are faced with problems: how to evaluate the customer staff's mastery and proficiency in the new software system.

In the related art, the grasping degree and proficiency of the client staff on the new software system are evaluated by an investigation method, which takes a lot of time to observe and evaluate the operation of each staff, which is time-consuming and inefficient; the existing automatic assessment tools often cannot fully cover the complex work demands in the SaaS software, and the performance of a user in a complex operation flow is difficult to accurately capture and assess, so that the accuracy of operation skill testing is reduced.

Disclosure of Invention

The embodiment of the application provides an operation skill testing method and system for SaaS software. The following presents a simplified summary in order to provide a basic understanding of some aspects of the disclosed embodiments. This summary is not an extensive overview and is intended to neither identify key/critical elements nor delineate the scope of such embodiments. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is presented later.

In a first aspect, an embodiment of the present application provides a method for testing an operation skill of SaaS software, which is applied to a server, and the method includes:

Receiving user behavior data and real operation service data sent by a client, wherein the user behavior data is human-computer interaction data monitored when various operation events preset in SaaS software are triggered in the answering process of a target test paper by a user, the real operation service data is data related to service logic generated when various operation events are triggered by the user, and the target test paper is constructed based on the various operation events;

Generating operation scores of the users according to the user behavior data and the standard behavior data of the target test paper;

Generating a data score of the user according to the actual operation service data of the user and the standard service data of the target test paper;

and carrying out weighted summation on the operation score and the data score to obtain the operation proficiency of the user on the SaaS software.

Optionally, the target test paper is generated according to the following steps:

Collecting business logic of SaaS software to obtain a demand analysis result;

classifying and describing the operation events in the SaaS software according to the requirement analysis result to obtain multiple types of operation events;

selecting operation events applicable to different user roles from multiple types of operation events;

Creating standard behavior data and standard business data aiming at selected operation events applicable to different user roles;

and generating target test papers applicable to different user roles according to the operation events applicable to the different user roles, the standard behavior data and the standard service data thereof.

Optionally, generating the target test paper applicable to different user roles according to the operation event applicable to different user roles, the standard behavior data and the standard service data thereof, includes:

receiving topics set for operation events applicable to different user roles;

storing the mapping relation between the topics set by the operation events applicable to different user roles and the standard behavior data and the standard business data of the operation events applicable to different user roles;

And integrating the mapping relation with the SaaS software so that the user roles can enter the SaaS software for operation through the triggering questions, and obtaining target test papers suitable for different user roles.

Optionally, before receiving the user behavior data and the actual operation service data sent by the client, the method further includes:

Adding data embedded points aiming at various preset operation events in the SaaS software, wherein the data embedded points are used for monitoring and generating man-machine interaction data generated when a user triggers various preset operation events in the SaaS software;

Acquiring a plurality of questions existing in a target test paper answered by a user;

and obtaining standard behavior data and standard service data corresponding to a plurality of topics from the mapping relation to obtain standard behavior data of the target test paper and standard service data of the target test paper.

Optionally, generating the operation score of the user according to the user behavior data and the standard behavior data of the target test paper includes:

Fitting a plurality of actual behaviors of the user according to the user behavior data;

establishing a standard behavior model according to standard behavior data of the target test paper;

Matching the user behavior data with the standard behavior model to generate the matching degree between each actual behavior and each standard behavior in the standard behavior model;

According to the matching degree between each actual behavior and each standard behavior in the standard behavior model, distributing a score to each actual behavior to obtain the score of each actual behavior;

and summing the scores of each actual behavior to obtain the operation score of the user.

Optionally, building a standard behavior model according to standard behavior data of the target test paper includes:

fitting a plurality of standard behaviors according to standard behavior data of the target test paper;

Defining a quantization index, a standard of successful operation and each standard behavior based on service requirements, and creating standard operation steps and operation sequences;

performing visual processing on the operation steps and the operation sequence by adopting a flow chart or a flow mapping technology to obtain a standard operation flow node diagram;

converting the standard operation flow node diagram into operation rules and conditional sentences;

the operation rules and the conditional statements are used as standard behavior models.

Optionally, the standard behavior model includes operation rules and conditional statements;

matching the user behavior data with the standard behavior model to generate the matching degree between each actual behavior and each standard behavior in the standard behavior model, comprising:

Acquiring an operation event identifier of user behavior data;

mapping the user behavior data to operation rules and conditional sentences according to the operation event identification of the user behavior data to obtain operation rules and conditional sentences of the associated behavior data;

executing operation rules and conditional sentences of the associated behavior data to check whether each actual behavior accords with the operation rules and the conditional sentences or not, and obtaining a check result of each actual behavior;

And taking the ratio between the abnormal node quantity contained in the checking result of each actual behavior and the node total quantity contained in the checking result of each actual behavior as the matching degree between each actual behavior and each standard behavior in the standard behavior model.

Optionally, according to the matching degree between each actual behavior and each standard behavior in the standard behavior model, assigning a score to each actual behavior to obtain a score of each actual behavior, including:

Processing the matching degree between each actual behavior and each standard behavior in the standard behavior model by adopting a logic conversion function to obtain the score of each actual behavior; the logic conversion function is used for mapping the matching degree between each actual behavior and each standard behavior in the standard behavior model into a (0, 1) interval;

wherein, As a fraction of the actual behavior of the vehicle,Is the base of the natural logarithm,AndIs an adjustment parameter for controlling the sensitivity and offset of the score curve,For the degree of matching between each actual behavior and each standard behavior in the standard behavior model,Is a value between 0 and 100, where 100 indicates that the user's operation is fully compliant with the standard behavior and 0 indicates a complete mismatch.

10. Optionally, the user behavior data includes a preset embedded point identifier and a first operation behavior, and the standard behavior data includes a preset embedded point identifier and a second operation behavior; in the first operation behavior, each investigation item with the decoiler mark is executed in sequence or only needs to be executed in the operation process; the operation process executed in sequence indicates that the obtained data is wrong when the operation sequence is wrong; only the operation process which is executed is needed to indicate that the operation behavior existing at any time in the whole operation behavior does not influence the scoring result;

generating an operation score of the user according to the user behavior data and the standard behavior data of the target test paper, wherein the operation score comprises the following steps:

The processing procedure performed in sequence is that,

Grouping the first operation behaviors and the second operation behaviors according to the preset embedded point identification to obtain a plurality of groups of first operation behaviors and second operation behaviors;

Judging whether an inspection item of the operation configuration of the target test paper is a preset first mark or not in the process of comparing buried points in the same group, if so, traversing each first operation behavior to judge whether an operation of the preset first mark exists or not, and if so, marking the operation; judging whether an inspection item of the operation configuration of the target test paper is a preset second mark and a first block exists in the inspection item, if so, traversing each first operation behavior in sequence according to the sequence of the second operation behaviors configured in the standard behavior data, marking the traversed first operation behaviors consistent with the operation matching configured in the standard behavior data, and if the second block exists in the inspection item, traversing each first operation behavior from the first block to the back according to the last operation in the sequence check, and marking the traversed first operation behaviors consistent with the operation matching configured in the standard behavior data; or traversing untagged first operation behaviors from the first operation behaviors in each group when traversing the first operation of different groups in the buried point comparison process in different groups, and marking the traversed first operation behaviors consistent with the operation matching configured in the standard behavior data; generating an operation score of the user according to the marked operation; or alternatively

Only the processing procedure to be performed is that,

Judging whether each first operation behavior exists in the second operation behaviors, if so, marking the existing first operation behaviors, and generating operation scores of users according to marked operations.

Optionally, generating the data score of the user according to the actual operation service data of the user and the standard service data of the target test paper includes:

Comparing the real operation service data of the user with the standard service data of the target test paper to determine target data existing in the standard service data in the real operation service data of the user;

Counting a first data amount of the target data;

counting a second data amount of standard business data of the target test paper;

calculating the ratio of the first data quantity to the second data quantity;

and processing the ratio into the mathematical representation dimension which is the same as the operation score to obtain the data score of the user.

In a second aspect, an embodiment of the present application provides an operating skill testing system for SaaS software, the system including:

The data receiving module is used for receiving user behavior data and practical operation service data sent by the client, wherein the user behavior data is human-computer interaction data monitored when various operation events are preset in SaaS software in a response process of a user aiming at a target test paper, the practical operation service data is data related to service logic generated when the user triggers various operation events, and the target test paper is constructed based on the various operation events;

The operation score generating module is used for generating an operation score of the user according to the user behavior data and the standard behavior data of the target test paper;

the data score generating module is used for generating a data score of the user according to the actual operation service data of the user and the standard service data of the target test paper;

And the operation proficiency generating module is used for carrying out weighted summation on the operation score and the data score to obtain the operation proficiency of the user on the SaaS software.

The technical scheme provided by the embodiment of the application can have the following beneficial effects:

In the embodiment of the application, on one hand, the operation skill test can be automatically carried out on the client staff by receiving the user behavior data and the actual operation business data sent by the client, thereby avoiding the grasping degree and the proficiency of the client staff on the new software system by a manual investigation mode, saving the skill test time and improving the skill test efficiency; on the other hand, the user behavior data is the man-machine interaction data monitored when various operation events are preset in the SaaS software in the response process of the user aiming at the target test paper, the real operation business data is data related to business logic generated when the user triggers various operation events, the generation of the data corresponds to various complex work requirements in the SaaS software, the complex work requirements in the SaaS software can be covered comprehensively in the skill test process, meanwhile, the real business environment and the real-time data used by the user can be accurately reflected based on the combination of the user behavior data and the real operation business data, and the final test result can accurately reflect the performance of the user in the actual work, so that the accuracy of the operation skill test is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application as claimed.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.

FIG. 1 is a schematic flow chart of a method for testing the operating skills of SaaS software according to an embodiment of the present application;

FIG. 2 is a schematic diagram of the test paper content of a target test paper provided by the application;

FIG. 3 is a schematic diagram of a configuration interface of a target test paper provided by the application;

FIG. 4 is a schematic diagram of another configuration interface of a target test paper provided by the present application;

FIG. 5 is a schematic diagram of a scoring result for a user according to the present application;

FIG. 6 is a schematic diagram of an operating skill testing system for SaaS software according to the present application;

Fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The following description and the drawings sufficiently illustrate specific embodiments of the application to enable those skilled in the art to practice them.

It should be understood that the described embodiments are merely some, but not all, embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the application. Rather, they are merely examples of systems and methods that are consistent with aspects of the application as detailed in the accompanying claims.

In the description of the present application, it should be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. The specific meaning of the above terms in the present application will be understood in specific cases by those of ordinary skill in the art. Furthermore, in the description of the present application, unless otherwise indicated, "a plurality" means two or more. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship.

The application provides a method and a system for testing operation skills of SaaS software, which are used for solving the problems in the related technical problems. In the embodiment of the application, on one hand, the operation skill test can be automatically carried out on the client staff by receiving the user behavior data and the actual operation business data sent by the client, thereby avoiding the grasping degree and the proficiency of the client staff on the new software system by a manual investigation mode, saving the skill test time and improving the skill test efficiency; on the other hand, the user behavior data is the man-machine interaction data monitored when various operation events are preset in the SaaS software in the response process of the user aiming at the target test paper, the real operation business data is data related to business logic generated when the user triggers various operation events, the generation of the data corresponds to various complex work requirements in the SaaS software, the complex work requirements in the SaaS software can be covered comprehensively in the skill test process, meanwhile, the real business environment and the real-time data used by the user can be accurately reflected based on the combination of the user behavior data and the real operation business data, the final test result can accurately reflect the performance of the user in the actual work, and therefore the accuracy of operation skill test is improved.

The following describes in detail the method for testing the operation skills of SaaS software according to the embodiment of the present application with reference to fig. 1 to 5. The method may be implemented in dependence on a computer program, and may be run on an operating skill testing system for SaaS software based on von neumann system. The computer program may be integrated in the application or may run as a stand-alone tool class application.

Referring to fig. 1, a flow chart of a method for testing operation skills of SaaS software is provided for an embodiment of the present application, and is applied to a server. As shown in fig. 1, the method according to the embodiment of the present application may include the following steps:

s101, receiving user behavior data and real operation service data sent by a client, wherein the user behavior data is human-computer interaction data monitored when various operation events are preset in SaaS software in a response process of a user aiming at a target test paper, the real operation service data is data related to service logic generated when the user triggers various operation events, and the target test paper is constructed based on the various operation events;

The man-machine interaction data are data generated by interaction between a user and the SaaS software interface in the test process, and the data can comprise mouse clicking, key input, page scrolling and the like, for example, each click of the user on the interface, a button clicked by the user comprises operations such as submitting, saving and the like, or data input by the user in a form or a field. The human-machine interaction data may capture each interaction of the user by deploying event listeners (e.g., javaScript listeners) at the SaaS software front-end, which trigger and record the relevant data as the user performs an operation, and then send the data to the server.

The data related to the business logic may be transaction records (such as sales orders, purchase orders, etc.), customer information (such as contact information of customers, transaction histories, etc.), inventory data (such as inventory levels of products, in-out records, etc.), and financial data, such as amounts input by users, selected payment methods, etc. when a financial operation is completed, and these data directly participate in the business process. In the backend system of SaaS software, the system will record these data when user behavior triggers business logic. For example, when a user submits a form, the backend API may receive, process, and save the data to the database.

The preset operation events refer to operation events which are predefined in SaaS software and are possibly triggered by a user when the user completes a service task. Such as button clicks, data submissions, query requests, etc. During the development of SaaS software, a developer defines a series of operational events and binds specific processing logic for each event. These events may be programmed in the front-end and back-end code of the software.

The target test paper refers to a set of evaluation tools or tests designed based on business logic and operation flow of SaaS software and used for evaluating the proficiency and accuracy of a user on software operation. The target test paper is cooperated by a business analyst and an education specialist, and the test paper is designed according to the functions of SaaS software and business requirements. Each question or task in the test paper corresponds to one or more operational events in the software.

In some embodiments, a business expert and an education designer cooperate to design a set of target test paper according to the functions and business requirements of the SaaS software, wherein the test paper comprises a plurality of operation tasks, and each task corresponds to one or more preset operation events in the software; defining and realizing preset operation events such as button clicking, form submission and the like in SaaS software, and binding a monitor for each event for capturing user interaction; and deploying an event monitor in a front-end interface of the SaaS software.

When the user carries out operation skill test after deployment, the user can log in an operation skill test system for SaaS software through a client, after logging in the system, users with different roles can obtain test papers for answering by themselves, when the user executes tasks in the test papers, a monitor captures and records man-machine interaction data, when the user triggers a preset operation event in the answering process, a front-end monitor captures user behavior data in real time and sends the data to a server, and when the user triggers the operation event, a back-end service receives and processes business logic to generate real operation business data and sends the data to a server; the server receives user behavior data and real operation business data from the client.

In the embodiment of the application, the target test paper can be generated according to the following steps: collecting business logic of SaaS software to obtain a demand analysis result; classifying and describing the operation events in the SaaS software according to the requirement analysis result to obtain multiple types of operation events; selecting operation events applicable to different user roles from multiple types of operation events; creating standard behavior data and standard business data aiming at selected operation events applicable to different user roles; and generating target test papers applicable to different user roles according to the operation events applicable to the different user roles, the standard behavior data and the standard service data thereof. The target test paper is shown in fig. 2, and the configuration interface of the target test paper is shown in fig. 3 and 4.

Business logic is business operation mode and flow supported by SaaS software, including business rules, operation steps, data flows and the like, and can be collected and understood through interviews with business specialists, analysis of user stories and use cases. The result of the demand analysis is based on information collected by the business logic to analyze and determine the functional and performance requirements that the evaluation system needs to meet. The operation event refers to a specific operation that the user can perform in the SaaS software, such as clicking a button, inputting data, selecting a menu, and the like. The user roles refer to different user categories, such as administrators, ordinary users, auditors, and the like, which are divided according to responsibilities and permissions in the SaaS software. Standard behavioral data refers to the standard execution form defined for each operational event, including the correct steps, order, and outcome. Standard business data refers to expected business data results related to operational events, such as correct calculation results, data status, etc.

Specifically, the specific process of generating the target test paper applicable to different user roles according to the operation event applicable to different user roles, the standard behavior data and the standard service data thereof comprises the following steps: receiving topics set for operation events applicable to different user roles; storing the mapping relation between the topics set by the operation events applicable to different user roles and the standard behavior data and the standard business data of the operation events applicable to different user roles; and integrating the mapping relation with the SaaS software so that the user roles can enter the SaaS software for operation through the triggering questions, and obtaining target test papers suitable for different user roles.

In the embodiment of the application, by considering the characteristics and the requirements of different user roles, personalized evaluation test papers can be provided for each user or user group, so that the evaluation content is ensured to be closely related to the actual work of the user or user group.

Specifically, standard behavior data of the target test paper and standard business data of the target test paper can be obtained according to the following steps: adding data embedded points aiming at various preset operation events in the SaaS software, wherein the data embedded points are used for monitoring and generating man-machine interaction data generated when a user triggers various preset operation events in the SaaS software; acquiring a plurality of questions existing in a target test paper answered by a user; and obtaining standard behavior data and standard service data corresponding to a plurality of topics from the mapping relation to obtain standard behavior data of the target test paper and standard service data of the target test paper.

In the embodiment of the application, by collecting the business logic of the SaaS software and creating the standard behavior data and the standard business data, whether the operation skills of the user accord with the expected business standard can be more accurately estimated.

S102, generating operation scores of users according to user behavior data and standard behavior data of target test papers;

the user behavior data comprise preset buried point identifiers and first operation behaviors, and the standard behavior data comprise preset buried point identifiers and second operation behaviors; in the first operation behavior, each investigation item with the decoiler mark is executed in sequence or only needs to be executed in the operation process; the operation process executed in sequence indicates that the obtained data is wrong when the operation sequence is wrong; only the operational procedure that is performed indicates that the operational behavior that the operation exists at any time in the overall operational behavior does not affect the scoring result.

In some embodiments of the present application, the specific process of generating the operation score of the user according to the user behavior data and the standard behavior data of the target test paper includes: the processing procedure of the orderly execution is that the first operation behaviors and the second operation behaviors are grouped according to the preset embedded point identification to obtain a plurality of groups of first operation behaviors and second operation behaviors; judging whether an inspection item of the operation configuration of the target test paper is a preset first mark or not in the process of comparing buried points in the same group, if so, traversing each first operation behavior to judge whether an operation of the preset first mark exists or not, and if so, marking the operation; judging whether an inspection item of the operation configuration of the target test paper is a preset second mark and a first block exists in the inspection item, if so, traversing each first operation behavior in sequence according to the sequence of the second operation behaviors configured in the standard behavior data, marking the traversed first operation behaviors consistent with the operation matching configured in the standard behavior data, and if the second block exists in the inspection item, traversing each first operation behavior from the first block to the back according to the last operation in the sequence check, and marking the traversed first operation behaviors consistent with the operation matching configured in the standard behavior data; or traversing untagged first operation behaviors from the first operation behaviors in each group when traversing the first operation of different groups in the buried point comparison process in different groups, and marking the traversed first operation behaviors consistent with the operation matching configured in the standard behavior data; and generating an operation score of the user according to the marked operation. The operation of presetting a first group of specific functions in the SaaS software is called a first block; after the first block, the operation of the preset second set of specific functions that needs to be continued is called a second block.

Or only the processing procedure is needed to be executed, whether each first operation behavior exists in the second operation behaviors is judged, if so, the existing first operation behaviors are marked, and the operation score of the user is generated according to marked operations.

In other embodiments of the present application, the specific process of generating the operation score of the user according to the user behavior data and the standard behavior data of the target test paper includes: fitting a plurality of actual behaviors of the user according to the user behavior data; establishing a standard behavior model according to standard behavior data of the target test paper; matching the user behavior data with the standard behavior model to generate the matching degree between each actual behavior and each standard behavior in the standard behavior model; according to the matching degree between each actual behavior and each standard behavior in the standard behavior model, distributing a score to each actual behavior to obtain the score of each actual behavior; and summing the scores of each actual behavior to obtain the operation score of the user.

Specifically, the process of establishing the standard behavior model according to the standard behavior data of the target test paper comprises the following steps: fitting a plurality of standard behaviors according to standard behavior data of the target test paper; defining a quantization index, a standard of successful operation and each standard behavior based on service requirements, and creating standard operation steps and operation sequences; performing visual processing on the operation steps and the operation sequence by adopting a flow chart or a flow mapping technology to obtain a standard operation flow node diagram; converting the standard operation flow node diagram into operation rules and conditional sentences; the operation rules and the conditional statements are used as standard behavior models.

Wherein the standard behavior model includes operation rules and conditional statements.

Specifically, in the process of matching the user behavior data with the standard behavior model, the specific process of generating the matching degree between each actual behavior and each standard behavior in the standard behavior model includes: acquiring an operation event identifier of user behavior data; mapping the user behavior data to operation rules and conditional sentences according to the operation event identification of the user behavior data to obtain operation rules and conditional sentences of the associated behavior data; executing operation rules and conditional sentences of the associated behavior data to check whether each actual behavior accords with the operation rules and the conditional sentences or not, and obtaining a check result of each actual behavior; and taking the ratio between the abnormal node quantity contained in the checking result of each actual behavior and the node total quantity contained in the checking result of each actual behavior as the matching degree between each actual behavior and each standard behavior in the standard behavior model.

Specifically, in the specific process of distributing the score to each actual behavior according to the matching degree between each actual behavior and each standard behavior in the standard behavior model, the score obtaining the score of each actual behavior comprises the following steps: processing the matching degree between each actual behavior and each standard behavior in the standard behavior model by adopting a logic conversion function to obtain the score of each actual behavior; the logic conversion function is used for mapping the matching degree between each actual behavior and each standard behavior in the standard behavior model into a (0, 1) interval;

S103, generating a data score of the user according to the actual operation service data of the user and the standard service data of the target test paper;

the data score refers to a score calculated according to the matching degree between the actual operation service data of the user and the standard service data of the target test paper, and the score reflects the accuracy of the operation of the user and the correct application of the service logic.

In the embodiment of the application, the specific process of generating the data score of the user according to the actual operation service data of the user and the standard service data of the target test paper comprises the following steps: comparing the real operation service data of the user with the standard service data of the target test paper to determine target data existing in the standard service data in the real operation service data of the user; counting a first data amount of the target data; counting a second data amount of standard business data of the target test paper; calculating the ratio of the first data quantity to the second data quantity; and processing the ratio into the mathematical representation dimension which is the same as the operation score to obtain the data score of the user.

The data comparison is a process of comparing the actual operation service data of the user with the standard service data one by one so as to determine the accuracy of the user operation. The first data amount refers to the number or measure of target data found in the real operation service data that matches the standard service data. The second data amount refers to the total amount or measurement of all expected data preset in the standard service data of the target test paper. The ratio is the ratio of the first data volume to the second data volume, reflecting the accuracy or completion of the user's actual operation of the business data. The mathematical representation dimension refers to converting the ratio into the same numerical range or metric as the operational score, e.g., a normalized score of 0 to 100 points or 0 to 1.

And S104, carrying out weighted summation on the operation score and the data score to obtain the operation proficiency of the user on the SaaS software.

Wherein the weighted sum calculation formula of the operation score and the data score can be:

The operation proficiency score= (operation score multiplied by w1+data score multiplied by w 2)/(w1+w2), w1 and w2 are weight values of the operation score and the data score, respectively, which are set according to the evaluation target and the business requirement, the weight value of the operation score may be set according to the step accuracy and the operation speed, and the weight value of the data score may be set based on the result accuracy and the data integrity.

In the embodiment of the application, the operation proficiency of the user to the SaaS software can be comprehensively evaluated by the method, the accuracy and the efficiency of the operation are considered, the actual business result generated by the operation is considered, and a comprehensive score is provided to help the user and the organization to know and improve the operation skill. The final scoring result is shown in fig. 5, for example.

The following are system embodiments of the present application that may be used to perform method embodiments of the present application. For details not disclosed in the system embodiments of the present application, please refer to the method embodiments of the present application.

Referring to fig. 6, a schematic structural diagram of an operation skill testing system for SaaS software according to an exemplary embodiment of the present application is shown, where the system is configured to execute the operation skill testing method for SaaS software applied to a server provided in any one of the above embodiments. The operating skill test system for SaaS software may be implemented as all or part of an electronic device by software, hardware, or a combination of both. The system 1 includes a data receiving module 10, an operation score generating module 20, and a data score generating module 30.

The data receiving module 10 is configured to receive user behavior data and real operation service data sent by a client, where the user behavior data is human-computer interaction data monitored when a user triggers various preset operation events in SaaS software in a response process of a target test paper, the real operation service data is data related to service logic generated when the user triggers various preset operation events, and the target test paper is constructed based on the various preset operation events;

The operation score generating module 20 is configured to generate an operation score of the user according to the user behavior data and the standard behavior data of the target test paper;

The data score generating module 30 is configured to generate a data score of the user according to the actual operation service data of the user and the standard service data of the target test paper;

It should be noted that, when the operation skill testing system for SaaS software provided in the foregoing embodiment executes the operation skill testing method for SaaS software, only the division of the foregoing functional modules is used for illustration, and in practical application, the foregoing functional allocation may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the operation skill testing system for SaaS software and the operation skill testing method for SaaS software provided in the foregoing embodiments belong to the same concept, which embody detailed implementation procedures and are not described herein.

The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.

The present application also provides a computer readable medium having stored thereon program instructions which, when executed by a processor, implement the operating skill testing method for SaaS software provided by the above-described method embodiments.

The application also provides a computer program product containing instructions which, when run on a computer, cause the computer to perform the method of operating skill testing for SaaS software of the various method embodiments described above.

Referring to fig. 7, a schematic structural diagram of an electronic device is provided in an embodiment of the present application. As shown in fig. 7, the electronic device 1000 may include: at least one processor 1001, at least one network interface 1004, a user interface 1003, a memory 1005, at least one communication bus 1002.

Wherein the communication bus 1002 is used to enable connected communication between these components.

The user interface 1003 may include a Display screen (Display) and a Camera (Camera), and the optional user interface 1003 may further include a standard wired interface and a wireless interface.

The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), among others.

Wherein the processor 1001 may include one or more processing cores. The processor 1001 connects various parts within the overall electronic device 1000 using various interfaces and lines, performs various functions of the electronic device 1000 and processes data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 1005, and invoking data stored in the memory 1005. Alternatively, the processor 1001 may be implemented in at least one hardware form of digital signal Processing (DIGITAL SIGNAL Processing, DSP), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA), programmable logic array (Programmable Logic Array, PLA). The processor 1001 may integrate one or a combination of several of a central processing unit (Central Processing Unit, CPU), an image processor (Graphics Processing Unit, GPU), and a modem, etc. The CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing the content required to be displayed by the display screen; the modem is used to handle wireless communications. It will be appreciated that the modem may not be integrated into the processor 1001 and may be implemented by a single chip.

The Memory 1005 may include a random access Memory (Random Access Memory, RAM) or a Read-Only Memory (Read-Only Memory). Optionally, the memory 1005 includes a non-transitory computer readable medium (non-transitory computer-readable storage medium). The memory 1005 may be used to store instructions, programs, code, sets of codes, or sets of instructions. The memory 1005 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-described respective method embodiments, etc.; the storage data area may store data or the like referred to in the above respective method embodiments. The memory 1005 may also optionally be at least one storage system located remotely from the processor 1001. As shown in fig. 7, an operating system, a network communication module, a user interface module, and an operating skill test application for SaaS software may be included in the memory 1005 as one type of computer storage medium.

In the electronic device 1000 shown in fig. 7, the user interface 1003 is mainly used for providing an input interface for a user, and acquiring data input by the user; and the processor 1001 may be configured to call an operating skill test application for SaaS software stored in the memory 1005, and specifically perform the following operations:

In one embodiment, the processor 1001, when executing the generation of the target test paper, specifically performs the following operations:

Collecting business logic of SaaS software to obtain a demand analysis result;

In one embodiment, the processor 1001, when executing the generation of the target test paper applicable to different user roles according to the operation event applicable to different user roles and the standard behavior data and the standard business data thereof, specifically executes the following operations:

receiving topics set for operation events applicable to different user roles;

In one embodiment, the processor 1001, before executing the receiving user behavior data and real operation service data sent by the client, further executes the following operations:

In one embodiment, the processor 1001, when executing the generation of the operation score of the user according to the user behavior data and the standard behavior data of the target test paper, specifically performs the following operations:

In one embodiment, the processor 1001, when executing the standard behavior data according to the target test paper, specifically performs the following operations:

In one embodiment, the processor 1001, when performing matching of the user behavior data with the standard behavior model, generates a degree of matching between each actual behavior and each standard behavior in the standard behavior model, specifically performs the following operations:

Acquiring an operation event identifier of user behavior data;

The processing procedure performed in sequence is that,

Only the processing procedure to be performed is that,

In one embodiment, the processor 1001, when executing the generation of the data score of the user according to the real operation service data of the user and the standard service data of the target test paper, specifically performs the following operations:

Counting a first data amount of the target data;

calculating the ratio of the first data quantity to the second data quantity;

Those skilled in the art will appreciate that implementing all or part of the above-described methods in the embodiments may be accomplished by a computer program for instructing related hardware, and that the program for skill testing of the operations of SaaS software may be stored in a computer readable storage medium, which when executed may include the steps of the embodiments of the methods as described above. The storage medium of the program for the skill test of the SaaS software may be a magnetic disk, an optical disk, a read-only memory, a random access memory, or the like.

The foregoing disclosure is illustrative of the present application and is not to be construed as limiting the scope of the application, which is defined by the appended claims.

Claims

1. An operation skill testing method for SaaS software, which is characterized by being applied to a server side, the method comprising:

Receiving user behavior data and real operation service data sent by a client, wherein the user behavior data is human-computer interaction data monitored when various operation events are preset in SaaS software in a response process of a user aiming at a target test paper, the real operation service data is data related to service logic generated when the user triggers various operation events, and the target test paper is constructed based on the various operation events;

Generating an operation score of the user according to the user behavior data and the standard behavior data of the target test paper;

2. The method of claim 1, wherein generating the target test paper comprises:

Collecting business logic of the SaaS software to obtain a demand analysis result;

Selecting operation events applicable to different user roles from the multiple types of operation events;

and generating target test papers applicable to different user roles according to the operation events applicable to different user roles, the standard behavior data and the standard service data thereof.

3. The method according to claim 2, wherein the generating the target test paper applicable to the different user roles according to the operation event applicable to the different user roles and the standard behavior data and the standard business data thereof comprises:

receiving topics set for the operation events applicable to different user roles;

And integrating the mapping relation with the SaaS software so that the user role can enter the SaaS software to operate by triggering the questions to obtain target test papers suitable for different user roles.

4. The method of claim 3, further comprising, prior to receiving the user behavior data and the real operation business data sent by the client:

Adding data embedded points aiming at various preset operation events in the SaaS software, wherein the data embedded points are used for monitoring and generating man-machine interaction data generated when the user triggers various preset operation events in the SaaS software;

acquiring a plurality of questions existing in the target test paper answered by the user;

And obtaining standard behavior data and standard service data corresponding to the topics from the mapping relation to obtain the standard behavior data of the target test paper and the standard service data of the target test paper.

5. The method of claim 1, wherein generating the user's operational score based on the user's behavioral data and the standard behavioral data of the target test paper comprises:

establishing a standard behavior model according to the standard behavior data of the target test paper;

distributing a score to each actual behavior according to the matching degree between each actual behavior and each standard behavior in the standard behavior model to obtain the score of each actual behavior;

and summing the scores of the actual behaviors to obtain the operation score of the user.

6. The method of claim 5, wherein the establishing a standard behavior model based on standard behavior data of the target test paper comprises:

fitting a plurality of standard behaviors according to the standard behavior data of the target test paper;

performing visualization processing on the operation steps and the operation sequence by adopting a flow chart or a flow mapping technology to obtain a standard operation flow node diagram;

and taking the operation rules and the conditional sentences as standard behavior models.

7. The method of claim 5, wherein the standard behavioral model comprises operational rules and conditional statements;

the step of matching the user behavior data with the standard behavior model to generate the matching degree between each actual behavior and each standard behavior in the standard behavior model comprises the following steps:

acquiring an operation event identifier of the user behavior data;

According to the operation event identification of the user behavior data, mapping the user behavior data to the operation rules and the conditional sentences to obtain operation rules and conditional sentences of the associated behavior data;

executing the operation rule and the conditional statement of the associated behavior data to check whether each actual behavior accords with the operation rule and the conditional statement or not, and obtaining a check result of each actual behavior;

and taking the ratio of the abnormal node quantity contained in the checking result of each actual behavior to the node total quantity contained in the checking result of each actual behavior as the matching degree between each actual behavior and each standard behavior in the standard behavior model.

8. The method of claim 5, wherein assigning a score to each of the actual behaviors based on the degree of matching between each of the actual behaviors and each of the standard behaviors in the standard behavior model, the score for each of the actual behaviors comprising:

9. The method of claim 1, wherein the user behavior data comprises a preset buried point identification and a first operational behavior, and the standard behavior data comprises a preset buried point identification and a second operational behavior; in the first operation behavior, each investigation item with the rolling person mark is executed in sequence or only needs to be executed in the operation process; the operation process executed in sequence indicates that the obtained data is wrong when the operation sequence is wrong; the operation process which only needs to be executed indicates that the operation behavior existing at any time in the whole operation behavior does not influence the scoring result;

And generating an operation score of the user according to the user behavior data and the standard behavior data of the target test paper, wherein the operation score comprises the following steps:

the processing procedure performed in sequence is that,

Judging whether an inspection item of the operation configuration of the target test paper is a preset first mark or not in the process of comparing buried points in the same group, if so, traversing each first operation behavior to judge whether an operation of the preset first mark exists or not, and if so, marking the operation; judging whether an inspection item of the operation configuration of the target test paper is a preset second mark and a first block exists in the inspection item, if so, traversing each first operation behavior in sequence according to the sequence of the second operation behaviors configured in the standard behavior data, marking the traversed first operation behaviors consistent with the operation matching configured in the standard behavior data, and if the second block exists in the inspection item, traversing each first operation behavior backwards from the first block according to the last operation in the sequence examination, and marking the traversed first operation behaviors consistent with the operation matching configured in the standard behavior data; or traversing untagged first operation behaviors from the first operation behaviors in each group when traversing the first operation of different groups in the buried point comparison process in different groups, and marking the traversed first operation behaviors consistent with the operation matching configured in the standard behavior data; generating an operation score of the user according to the marked operation; or alternatively

The only processing that needs to be performed is,

Judging whether each first operation behavior exists in the second operation behaviors, if so, marking the existing first operation behaviors, and generating the operation score of the user according to marked operation.

10. The method of claim 1, wherein the generating the data score of the user based on the real operation service data of the user and the standard service data of the target test paper comprises:

Counting a first data amount of the target data;

Counting a second data volume of standard business data of the target test paper;

Calculating a ratio of the first data amount to the second data amount;

11. An operating skill testing system for SaaS software, the system comprising:

The data receiving module is used for receiving user behavior data and real operation business data sent by a client, wherein the user behavior data is human-computer interaction data monitored when various operation events preset in SaaS software are triggered in the answering process of a target test paper, the real operation business data is data related to business logic generated when the user triggers various operation events, and the target test paper is constructed based on the various operation events;

the operation score generation module is used for generating the operation score of the user according to the user behavior data and the standard behavior data of the target test paper;

the data score generating module is used for generating the data score of the user according to the actual operation service data of the user and the standard service data of the target test paper;