CN110287103B

CN110287103B - Software product evaluation processing method and device, computer equipment and storage medium

Info

Publication number: CN110287103B
Application number: CN201910430095.3A
Authority: CN
Inventors: 赵昊; 赵晔菲; 张佩茜
Original assignee: OneConnect Financial Technology Co Ltd Shanghai
Current assignee: OneConnect Financial Technology Co Ltd Shanghai
Priority date: 2019-05-22
Filing date: 2019-05-22
Publication date: 2022-05-17
Anticipated expiration: 2039-05-22
Also published as: CN110287103A

Abstract

The invention discloses a software product evaluation processing method and device, computer equipment and a storage medium. The method comprises the following steps: obtaining an evaluation analysis sample based on the target product type of the evaluation analysis request, and extracting at least one corresponding evaluation problem and an original evaluation dimension; performing importance analysis on the original evaluation dimensionality to obtain at least two target evaluation dimensionalities; performing importance analysis on all the evaluation problems corresponding to each target evaluation dimension to obtain an evaluation index; carrying out model construction on the target evaluation dimension and the evaluation index to obtain a target evaluation analysis model; acquiring a target evaluation problem corresponding to each evaluation index, and performing consistency check to acquire a consistency check result; and if the consistency check result is that the check is passed, acquiring product evaluation questionnaire information based on the target evaluation question, and embedding the product evaluation questionnaire information into an evaluation interface of the software product to be tested to form the target software product. The method can reduce the evaluation cost of the software product and improve the efficiency and the real-time performance.

Description

Software product evaluation processing method and device, computer equipment and storage medium

Technical Field

The invention relates to the technical field of software testing, in particular to a software product evaluation processing method and device, computer equipment and a storage medium.

Background

Currently, financial institutions such as banks, securities and insurance, or other product development institutions develop different software products (such as financial products APP developed by the banking institutions) so as to implement business promotion based on the software products. In the software product popularization process, the user experience of a user on the software product is an important factor influencing the utilization rate or the popularization rate of the software product. Generally, the better the user experience, the more actively the user uses and popularizes the software product, so that the higher the usage rate and the popularization rate, the more favorable the business popularization of the software product. Therefore, before the software product is on line, corresponding internal testing is generally performed to collect evaluation and research data fed back by internal testing personnel based on the preset evaluation problem, or after the software product is on line, evaluation and research data fed back by a user to the preset evaluation problem are collected through a specific research project, so that an evaluation result corresponding to an evaluation index is determined according to the collected evaluation and research data, and optimization and improvement of the software product are performed according to the evaluation result.

The traditional software product evaluation is to comment on evaluation problems set by a product development institution in an evaluation period by adopting traditional investigation modes such as user interviews or questionnaires, to obtain evaluation investigation data and analyze the evaluation data so as to optimize products according to analysis results. The traditional software product evaluation process has the following defects: firstly, the cost is higher and the efficiency is lower, namely, a product development mechanism is required to be equipped with corresponding evaluation personnel for evaluation problem design and subsequent evaluation analysis, the labor cost and the time cost are higher, and the efficiency is lower; secondly, the evaluation problem design is one-sidedness, namely the design of the evaluation problem is limited by the professional ability and the knowledge of an evaluation person, and the designed evaluation problem possibly cannot well reflect the evaluation requirement of the software product; thirdly, the acquisition of the evaluation research data does not have real-time performance, namely, the corresponding evaluation research data cannot be acquired in real time only by commenting in an evaluation period set by a product development organization generally.

Disclosure of Invention

The embodiment of the invention provides a software product evaluation processing method, a software product evaluation processing device, computer equipment and a storage medium, and aims to solve the problems of high cost, low efficiency, and poor one-sidedness and instantaneity in the current software product evaluation process.

A software product evaluation processing method comprising:

acquiring an evaluation analysis request, wherein the evaluation analysis request comprises a software product to be tested and a target product type corresponding to the software product to be tested;

inquiring an evaluation questionnaire information base based on the target product type, if the evaluation questionnaire information base does not store original evaluation questionnaire information corresponding to the target product type, acquiring corresponding evaluation analysis samples based on the target product type, and extracting at least one evaluation problem corresponding to each evaluation analysis sample and an original evaluation dimension corresponding to each evaluation problem;

performing importance analysis on original evaluation dimensions corresponding to all the evaluation questions to obtain at least two target evaluation dimensions;

performing importance analysis on all the evaluation questions corresponding to each target evaluation dimension to obtain an evaluation index corresponding to the target evaluation dimension;

adopting structural equation model analysis software to carry out model construction on at least two target evaluation dimensions and the evaluation index corresponding to each target evaluation dimension to obtain a target evaluation analysis model;

acquiring a target evaluation problem corresponding to each evaluation index based on the evaluation indexes in the target evaluation analysis model, and performing consistency check on the target evaluation problem corresponding to each evaluation index to acquire a consistency check result;

if the consistency check result is that the check is passed, acquiring product evaluation questionnaire information based on target evaluation problems corresponding to all evaluation indexes, and storing the product evaluation questionnaire information and the target product type in an evaluation questionnaire information base in an associated manner;

and embedding the product evaluation questionnaire information into the evaluation interface of the software product to be tested based on the information acquisition interface on the software product to be tested to form a target software product.

A software product evaluation processing apparatus comprising:

the evaluation analysis request acquisition module is used for acquiring an evaluation analysis request, wherein the evaluation analysis request comprises a software product to be tested and a target product type corresponding to the software product to be tested;

the evaluation analysis sample acquisition module is used for inquiring an evaluation questionnaire information base based on the target product type, acquiring corresponding evaluation analysis samples based on the target product type if the evaluation questionnaire information base does not store original evaluation questionnaire information corresponding to the target product type, and extracting at least one evaluation problem corresponding to each evaluation analysis sample and an original evaluation dimension corresponding to each evaluation problem;

the target evaluation dimension acquisition module is used for performing importance analysis on original evaluation dimensions corresponding to all the evaluation questions to acquire at least two target evaluation dimensions;

the evaluation index acquisition module is used for analyzing the importance of all the evaluation problems corresponding to each target evaluation dimension to acquire an evaluation index corresponding to the target evaluation dimension;

the evaluation analysis model acquisition module is used for adopting structural equation model analysis software to carry out model construction on at least two target evaluation dimensions and the evaluation index corresponding to each target evaluation dimension to acquire a target evaluation analysis model;

the consistency check result acquisition module is used for acquiring a target evaluation problem corresponding to each evaluation index based on the evaluation indexes in the target evaluation analysis model, and performing consistency check on the target evaluation problem corresponding to each evaluation index to acquire a consistency check result;

the evaluation questionnaire information acquisition module is used for acquiring product evaluation questionnaire information based on target evaluation problems corresponding to all evaluation indexes if the consistency check result is that the check is passed, and storing the product evaluation questionnaire information and the target product type in an evaluation questionnaire information base in an associated manner;

and the target software product acquisition module is used for embedding the product evaluation questionnaire information into the evaluation interface of the software product to be tested based on the information acquisition interface on the software product to be tested to form the target software product.

A computer device comprising a memory, a processor and a computer program stored in said memory and executable on said processor, said processor implementing the steps of the software product evaluation processing method described above when executing said computer program.

A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out the steps of the above-mentioned software product evaluation processing method.

According to the software product evaluation processing method, the software product evaluation processing device, the computer equipment and the storage medium, when the original evaluation questionnaire information corresponding to the target product type is not stored in the evaluation questionnaire information base, the evaluation problem and the corresponding original evaluation dimension are extracted from the evaluation analysis sample corresponding to the target product type to determine the evaluation problem and the original evaluation dimension acquired by the existing software product corresponding to the target product type in the evaluation process, so that the data adopted in the evaluation analysis process has objectivity. And then, performing importance analysis on the original evaluation dimensions corresponding to all the evaluation problems so as to ensure the universal applicability of the target evaluation dimensions in the software product corresponding to the target product type according to the importance of the original evaluation dimensions in all the evaluation analysis samples corresponding to the target product type. And then, performing importance analysis on all the evaluation problems corresponding to each target evaluation dimension to obtain an evaluation index corresponding to the target evaluation dimension, and ensuring the general applicability of the evaluation index in the software product corresponding to the target product type. And then, model construction is carried out on the target evaluation dimension and the evaluation index by adopting structural equation model analysis software, so that the constructed target evaluation analysis model can reflect a relatively stable potential relationship between the target evaluation dimension and the evaluation index, and the target evaluation dimension and the evaluation index in the finally determined target evaluation analysis model have objectivity and universal applicability in the software product evaluation process corresponding to the target product type. And forming corresponding target evaluation questions according to evaluation indexes in the target evaluation analysis model, carrying out consistency check, and obtaining product evaluation questionnaire information when the consistency check result is that the check passes, so that the target evaluation questions in the product evaluation questionnaire information are objectively and comprehensively set. The product evaluation questionnaire information is embedded into an evaluation interface of the software product to be evaluated to form a target software product, so that a user can input feedback of target evaluation questions set for different evaluation indexes in real time in the process of using the target software product, and the real-time performance of user comments is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.

FIG. 1 is a schematic diagram of an application environment of a software product evaluation processing method according to an embodiment of the present invention;

FIG. 2 is a flowchart of a software product evaluation processing method according to an embodiment of the present invention;

FIG. 3 is another flow chart of a software product evaluation processing method according to an embodiment of the present invention;

FIG. 4 is another flow chart of a software product evaluation processing method according to an embodiment of the present invention;

FIG. 5 is another flow chart of a software product evaluation processing method according to an embodiment of the present invention;

FIG. 6 is another flow chart of a software product evaluation processing method according to an embodiment of the present invention;

FIG. 7 is another flow chart of a software product evaluation processing method according to an embodiment of the present invention;

FIG. 8 is another flow chart of a software product evaluation processing method according to an embodiment of the present invention;

FIG. 9 is a schematic diagram of the software product evaluation processing device according to an embodiment of the present invention;

FIG. 10 is a schematic diagram of a computer device according to an embodiment of the invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The software product evaluation processing method provided by the embodiment of the invention can be applied to the application environment shown in fig. 1. Specifically, the software product evaluation processing method is applied to a product evaluation platform, the product evaluation platform comprises a client and a server shown in fig. 1, the client and the server are communicated through a network and are used for realizing the construction of an evaluation analysis model matched with the type of a target product, so that product evaluation questionnaire information is determined according to evaluation indexes in the evaluation analysis model and is embedded into a software product to be tested, the target software product comprising an evaluation interface is formed, the objectivity of an evaluation problem is ensured, a user can conveniently conduct real-time evaluation, and the efficiency and quality of software product optimization based on the evaluation indexes in the evaluation analysis model are improved. The client is also called a user side, and refers to a program corresponding to the server and providing local services for the client. The client may be installed on, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server may be implemented as a stand-alone server or as a server cluster consisting of a plurality of servers.

In an embodiment, as shown in fig. 2, a software product evaluation processing method is provided, which is described by taking the server in fig. 1 as an example, and includes the following steps:

s201: and acquiring an evaluation analysis request, wherein the evaluation analysis request comprises a software product to be tested and a target product type corresponding to the software product to be tested.

The evaluation analysis request is a request for triggering the server to perform evaluation analysis on a specific software product. The software product to be tested is a software product which needs to be evaluated and analyzed, and an information acquisition interface is reserved on the software product to be tested and used for receiving product evaluation questionnaire information for evaluating and analyzing the software product to be tested so as to acquire corresponding evaluation and research data based on the product evaluation questionnaire information and perform product optimization processing. The target product type refers to the product type of the software product targeted by the evaluation analysis, and the product type includes but is not limited to a financial tool type (such as palm banking), a communication tool type (such as WeChat) and an office tool type.

S202: and inquiring the evaluation questionnaire information base based on the type of the target product, if the original evaluation questionnaire information corresponding to the type of the target product is not stored in the evaluation questionnaire information base, acquiring corresponding evaluation analysis samples based on the type of the target product, and extracting at least one evaluation problem corresponding to each evaluation analysis sample and an original evaluation dimension corresponding to each evaluation problem.

Wherein the evaluation questionnaire information base is a database for storing original evaluation questionnaire information corresponding to each product type. The original evaluation questionnaire information refers to previously generated and stored evaluation questionnaire information. The evaluation analysis sample is a sample that can be used for evaluation analysis corresponding to the type of target product. The evaluation analysis sample can be a questionnaire adopted in the evaluation investigation process of the existing software product corresponding to the target product type or a dialect text adopted in the voice return visit process in the evaluation investigation process. Specifically, the server may extract keywords (or feature words) from each evaluation analysis sample by using a keyword extraction algorithm or other text feature word extraction algorithms to determine the evaluation questions contained therein and the corresponding original evaluation dimensions, thereby ensuring smooth proceeding of the subsequent evaluation analysis processing process.

The evaluation problem is specifically a specific problem mentioned in an evaluation analysis sample, and can be various problems existing in the evaluation process of a certain software product, such as a problem related to user experience, so that performance improvement is performed on the software product based on a statistical result of the evaluation problem, the software product is enabled to better meet the user requirement, and the adhesiveness of the user to the software product is improved. The original evaluation dimension is a dimension defining an attribute to which each evaluation question belongs in the evaluation analysis sample.

Any evaluation analysis sample comprises at least one evaluation question, and each evaluation question corresponds to an original evaluation dimension. That is, any one evaluation question corresponds to one original evaluation dimension (i.e., 1-to-1 relationship), and any one original evaluation dimension corresponds to at least one evaluation question (i.e., 1-to-M relationship). For example, the evaluation problem of 'the APP is not well operated' belongs to the original evaluation dimension of 'easy to use', but not belongs to the original evaluation dimensions of 'useful' or 'friendly', and the evaluation problem has uniqueness; for the original evaluation dimension of "easy to use", it may correspond to the evaluation problem of "this APP is not well operated", and may also correspond to the evaluation problem of "whether the operation flow of this APP is easy to understand". Generally, in an evaluation analysis sample, the original evaluation dimension can be a header portion of at least one evaluation question.

In this embodiment, the server queries the evaluation questionnaire information base based on the target product type, and determines whether the original evaluation questionnaire information corresponding to the target product type is stored; if the original evaluation questionnaire information corresponding to the target product type is stored in the evaluation questionnaire information base, performing adaptive adjustment on the original evaluation questionnaire information (for example, a specific function module in certain original evaluation questionnaire information is replaced by a function module in the software product to be detected) so as to improve the acquisition efficiency of the target software product; if the original evaluation questionnaire information corresponding to the target product type is not stored in the evaluation questionnaire information base, the steps S203-S208 are executed to obtain the target software product, so that the target software product is embedded into the obtained product evaluation questionnaire information, the target evaluation problem on the evaluation interface of the target software product has objectivity and comprehensiveness, the real-time evaluation of the user can be guaranteed, the evaluation cost is saved, and the evaluation efficiency is improved.

S203: and analyzing the importance of the original evaluation dimensions corresponding to all the evaluation problems to obtain at least two target evaluation dimensions.

Specifically, the process of performing importance analysis on all original evaluation dimensions corresponding to all evaluation questions is to perform importance analysis on all original evaluation dimensions contained in an evaluation analysis sample so as to determine a target evaluation dimension of an evaluation analysis model corresponding to the target product type, which can be finally determined, according to the analyzed importance. Generally, the more times (i.e., the greater the number) a certain original evaluation dimension is referred to in all evaluation analysis samples, the more important it is, and thus, the target evaluation dimension can be determined. And the target evaluation dimension is the original evaluation dimension which is determined from all the original evaluation dimensions and has larger influence on the evaluation of the software product corresponding to the target product type.

In this embodiment, importance analysis may be performed on all original evaluation dimensions corresponding to all evaluation questions in all evaluation analysis samples to determine a target evaluation dimension having a large influence on evaluation analysis of a software product corresponding to a target product type in all original evaluation dimensions. For example, the three useful, easy-to-use and friendly target evaluation dimensions with a large influence on the analysis of the software product corresponding to the target product type can be determined by analyzing the importance of all the original evaluation dimensions, and the three target evaluation dimensions can be clearly defined, such as whether the function configuration of the useful finger meets the requirement, whether the operation experience of the easy-to-use finger is smooth and the emotional design of the friendly finger product.

S204: and analyzing the importance of all the evaluation questions corresponding to each target evaluation dimension to obtain an evaluation index corresponding to the target evaluation dimension.

Each original evaluation dimension corresponds to at least one evaluation question, namely, a relationship of 1 to M, and the target evaluation dimension is determined according to the importance degree of the original evaluation dimension and also corresponds to at least one evaluation question. Specifically, the server may perform importance analysis on all the evaluation questions corresponding to each target evaluation dimension to determine which functions of the software product the evaluation question mainly aims at for evaluation, so as to determine corresponding evaluation indexes based on the main functions. The evaluation index corresponding to the target evaluation dimension can adopt an index corresponding to the target product type and used for evaluating the performance of the software product.

For example, if the target product type is a financial instrument type, and all the evaluation problems corresponding to the target evaluation dimension of "easy to use" mainly aim at the evaluation problems of whether the transfer function of a financial software product is smooth, whether the operation of the opening function is smooth, whether the entry of the key function is easy to search, whether the distribution of the important function modules is reasonable, and the like. When the server performs importance analysis on all the evaluation problems corresponding to the target evaluation dimension, namely 'easy-to-use', the proportion of the number of the problems of a certain evaluation problem in the number of the problems corresponding to all the evaluation problems needs to be analyzed, and whether the evaluation problem can be used as the evaluation index corresponding to the target evaluation dimension is determined according to the proportion. Generally, among all the evaluation questions corresponding to a certain target evaluation dimension, the larger the proportion of the number of the questions of a certain evaluation question to the number of the questions corresponding to all the evaluation questions is, the more concerned the product development organization is about the influence of the evaluation question on the software product, and the more important the product development organization is to evaluate the software product, so that the evaluation index corresponding to the target evaluation dimension can be determined.

S205: and adopting structural equation model analysis software to carry out model construction on at least two target evaluation dimensions and the evaluation index corresponding to each target evaluation dimension to obtain a target evaluation analysis model.

Among them, Structural Equation Model (SEM) is a method for establishing, estimating and checking a causal relationship model. The model contains both explicit variables that are observable and potentially latent variables that cannot be directly observed. The structural equation model can replace methods such as multiple regression, path analysis, factor analysis, covariance analysis and the like, and clearly analyzes the action of the single indexes on the whole and the correlation among the single indexes.

In this embodiment, the structural equation model analysis software includes, but is not limited to, AMOS software, that is, the server uses AMOS software, takes at least two target evaluation dimensions as potential variables, takes all evaluation indexes as observation variables, and determines the target evaluation analysis model after performing factor analysis (i.e., verification factor analysis to determine a potential relationship between the two) and stability analysis (i.e., analysis of stability of the potential relationship between the observation variables and the potential variables) on the observation variables and the potential variables. It is to be understood that the target evaluation analysis model is a model determined after performing factor analysis and modeling on the target evaluation dimensions as the potential variables and the evaluation indexes as the observation variables using structural equation model analysis software. The method can be understood that structural equation model analysis software is adopted to carry out model construction on the target evaluation dimension and the evaluation index, so that the constructed target evaluation analysis model can objectively reflect the potential relationship between the target evaluation dimension and the evaluation index, and the finally determined target evaluation dimension and the evaluation index have objectivity.

Further, step S205 specifically includes the following steps:

s2051: and acquiring at least two original evaluation data, wherein the original evaluation data comprise a dimension score value corresponding to each target evaluation dimension and an index score value corresponding to each evaluation index.

Specifically, after determining at least two target evaluation dimensions and corresponding evaluation indexes, the product development organization may determine a dimension score range corresponding to each target evaluation dimension according to the number of summary questions in the target synonym set corresponding to each target evaluation dimension, where the dimension score ranges of the three target evaluation dimensions, such as "easy to use", "useful", and "friendly", are all set to 0-20 points. And determining an index scoring mode and an index scoring weight corresponding to each evaluation index according to the problem proportion of the index problem number and the dimension problem number corresponding to the evaluation index. The server sends the dimension value range, the index grading mode and the index grading weight to a corresponding evaluation terminal of a professional evaluation person, and obtains at least two original evaluation data uploaded by the professional evaluation person through the evaluation terminal, wherein the original evaluation data comprise dimension evaluation values (such as dimension evaluation values corresponding to 'easy use', 'useful' and 'friendly') corresponding to each target evaluation dimension and index values corresponding to each evaluation index.

For example, if the target evaluation dimensions include "easy to use", "useful", and "friendly", then "easy to use", "useful", and "friendly" are determined as potential variables; determining the evaluation index corresponding to each target evaluation dimension as an observation variable (namely, an evaluation item), wherein the observation variable corresponding to the target evaluation dimension is transfer function score, account opening function score, entrance search score and function module distribution score if the observation variable corresponding to the target evaluation dimension is easy to use; the observation variables corresponding to the useful target evaluation dimension are credit card module score, investment and financing module score, transfer module score, account management module score and security center module score; the observation variables corresponding to the 'friendly' target evaluation dimension are 'fluency score', 'installation package size score', 'compatibility score', 'prompt accuracy score' and 'service experience score'. The corresponding scores of each evaluation index can be scored according to the same grade result, such as very good, relatively good, general, relatively poor and very good, and can also be scored according to 5, 4, 3, 2 and 1.

S2052: and carrying out verification factor analysis on the dimension score value corresponding to the target evaluation dimension and the index score value corresponding to the evaluation index by adopting AMOS software to obtain the analysis fitting degree of the constructed path simulation model, and determining the path simulation model as the target evaluation analysis model if the analysis fitting degree is greater than a preset fitting degree threshold value.

Wherein, the preset fitting degree threshold value is a preset threshold value which can be used for evaluating the fitting degree of the model to reach the stable standard. The AMOS software is one of structural equation model analysis software, and can implement a tool for performing a verification factor analysis between a latent variable and an observed variable to determine a latent relationship between the latent variable and the observed variable. Specifically, the server may perform verification factor analysis on a dimension score value corresponding to a target evaluation dimension and an index score value corresponding to an evaluation index in the original evaluation data by using the AMOS software, and analyze and determine an analysis fitting degree of a path simulation model constructed based on the target evaluation dimension (i.e., a latent variable) and the evaluation index (an observation variable). And comparing the analysis fitting degree with a preset fitting degree threshold value, and if the analysis fitting degree is greater than the preset fitting degree threshold value, determining that the path relation in the constructed path simulation model is obvious, namely the relation between the target evaluation dimension (namely the latent variable) and the evaluation index (observation variable) is stable, so that the path simulation model can be determined as the target evaluation analysis model. In this embodiment, the process of performing the verification factor analysis by using the AMOS software is a common function of the AMOS software, and is not described herein again for the prior art.

In the software product evaluation processing method provided by this embodiment, at least two pieces of original evaluation data are acquired, a dimension score value corresponding to a target evaluation dimension and an index score value corresponding to an evaluation index are determined, and the AMOS software is used to perform verification factor analysis on the dimension score value corresponding to the target evaluation dimension and the index score value corresponding to the evaluation index, so that a finally acquired target evaluation analysis model can reflect a relatively stable potential relationship between the two, and the finally determined target evaluation dimension and the evaluation index have objectivity.

S206: and acquiring a target evaluation problem corresponding to each evaluation index based on the evaluation indexes in the target evaluation analysis model, and performing consistency check on the target evaluation problem corresponding to each evaluation index to acquire a consistency check result.

The target evaluation problem refers to an evaluation problem corresponding to the software product to be tested, which is generated according to the evaluation index in the target evaluation analysis model. Specifically, the server configures different types of evaluation problem templates in advance, wherein the evaluation problem templates comprise an evaluation problem frame, an index filling area for filling evaluation indexes and a function filling area for filling specific function modules in the software product to be tested. For example, the evaluation question framework of an evaluation question template of a judgment type is "whether XX functions of this software are YY", where XX is a function fill area and YY is an index fill area, and when the login function for a software product is good, the function fill area is filled in "login", and the index fill area is filled in "good use". In this embodiment, the function filling area in the target evaluation question may be left empty, so that the user can fill in the function filling area autonomously according to actual requirements; or may be configured with specific options so that the user can make the selection decision according to actual needs. Specifically, for each evaluation index in the target evaluation analysis model, the client can be controlled to enter a problem configuration interface corresponding to the evaluation index, so that an evaluation problem template and a function module corresponding to each evaluation index are determined on the problem configuration interface, and thus, a corresponding target evaluation problem is automatically generated, and the determination efficiency of the target evaluation problem is guaranteed.

The consistency check method comprises the steps of carrying out consistency check on target evaluation problems corresponding to each evaluation index to obtain a consistency check result, and specifically, internally evaluating the target evaluation problems determined by each evaluation index in a target evaluation analysis model to determine whether the target evaluation problems are consistent with the consistency check result of a judgment standard, so that the pertinence and the comprehensiveness of the finally formed target evaluation problems can be determined. The consistency judgment standard can be understood as that in the internal evaluation process, when different target evaluators perform evaluation analysis on the target evaluation questions, whether the consistency judgment standard meets the preset consistency judgment standard (i.e. whether the consistency of the evaluation scores determined by the evaluation analysis meets the standard) is determined, so as to ensure the objectivity and comprehensiveness of the setting of the target evaluation questions.

S207: and if the consistency check result is that the check is passed, acquiring product evaluation questionnaire information based on target evaluation problems corresponding to all evaluation indexes, and storing the product evaluation questionnaire information and the target product type in an evaluation questionnaire information base in an associated manner.

Specifically, if the consistency check result is that the consistency check is passed, it indicates that the target evaluation questions determined based on each evaluation index in the target evaluation analysis model reach the preset consistency judgment standard when being subjected to internal evaluation, and the setting proportion of the target evaluation questions is determined to be objective and comprehensive, so that the product evaluation questionnaire information corresponding to the software product to be tested can be generated based on the target evaluation questions corresponding to all the evaluation indexes, so as to ensure the objectivity and comprehensiveness of the finally formed product evaluation questionnaire information. Furthermore, the product evaluation questionnaire information is used as original evaluation questionnaire information and is stored in an evaluation questionnaire information base in association with the target product type, so that the corresponding original evaluation questionnaire information can be rapidly determined according to the target product type, and the acquisition efficiency of the product evaluation questionnaire information corresponding to the software product to be detected is improved.

S208: and embedding the product evaluation questionnaire information into the evaluation interface of the software product to be tested based on the preset information acquisition interface of the software product to be tested to form a target software product.

The preset information acquisition interface of the software product to be tested is a preset interface for receiving product evaluation questionnaire information. For example, a link interface on the function module of "help and feedback" may be set on the software product to be tested, and the user may click the link interface corresponding to the "help and feedback" to upload the product evaluation questionnaire information, form an evaluation interface corresponding to the software product to be tested and an evaluation button corresponding to the evaluation interface based on the product evaluation questionnaire information, form the target software product, so that the user may click the evaluation button on the target software product to enter the corresponding evaluation interface, thereby uploading the evaluation investigation data corresponding to the target evaluation question in real time.

In the software product evaluation processing method provided by this embodiment, when the original evaluation questionnaire information corresponding to the target product type is not stored in the evaluation questionnaire information base, the evaluation question and the corresponding original evaluation dimension are extracted from the evaluation analysis sample corresponding to the target product type, so as to determine the evaluation question and the original evaluation dimension acquired during the evaluation process of the existing software product corresponding to the target product type, so that the data used in the evaluation analysis process has objectivity. And then, performing importance analysis on the original evaluation dimensions corresponding to all the evaluation problems so as to ensure the universal applicability of the evaluation of the target evaluation dimensions in the software product corresponding to the target product type according to the importance of the original evaluation dimensions in all the evaluation analysis samples corresponding to the target product type. And then, performing importance analysis on all the evaluation problems corresponding to each target evaluation dimension to obtain an evaluation index corresponding to the target evaluation dimension, and ensuring the general applicability of the evaluation index in the software product corresponding to the target product type. And then, model construction is carried out on the target evaluation dimension and the evaluation index by adopting structural equation model analysis software, so that the constructed target evaluation analysis model can reflect a relatively stable potential relationship between the target evaluation dimension and the evaluation index, and the target evaluation dimension and the evaluation index in the finally determined target evaluation analysis model have objectivity and universal applicability in the software product evaluation process corresponding to the target product type. And forming corresponding target evaluation questions according to evaluation indexes in the target evaluation analysis model, carrying out consistency check, and obtaining product evaluation questionnaire information when the consistency check result is that the check passes, so that the target evaluation questions in the product evaluation questionnaire information are objectively and comprehensively set. The product evaluation questionnaire information is embedded into an evaluation interface of the software product to be evaluated to form a target software product, so that a user can input feedback of target evaluation questions set for different evaluation indexes in real time in the process of using the target software product, and the real-time performance of user comments is improved.

In an embodiment, as shown in fig. 3, the obtaining of the corresponding evaluation analysis sample based on the target product type in step S202 specifically includes the following steps:

s301: original analysis samples are obtained from a sample database, and each original analysis sample corresponds to an original product type.

The sample database is a database in which the original analysis samples are stored in advance. The original analysis sample is a sample which is obtained by crawling from the internet by using a crawler tool in advance and is stored in a sample database and can be used for evaluation analysis, or is a sample which is uploaded in a local uploading mode and is stored in the sample database and can be used for evaluation analysis. Each original analysis sample corresponds to an original product type, which is a product type corresponding to the original analysis sample, including but not limited to a financial tool type, a communication tool type, and an office tool type.

S302: and carrying out deduplication processing on the original analysis sample to obtain a deduplication analysis sample.

Specifically, the server performs deduplication processing on the original analysis sample by using a text deduplication tool or a text similarity algorithm to remove duplicate original analysis samples, so that the obtained deduplication analysis samples have uniqueness, and interference of the duplicate original analysis samples on subsequent evaluation analysis processing is eliminated. For example, if 100 repeated original analysis samples are stored in the sample database, when the evaluation analysis processing is performed on the basis of the 100 repeated original analysis samples and other original analysis samples, the number of evaluation problems and original evaluation dimensions in the 100 repeated original analysis samples may increase, which affects the importance analysis results of subsequent original evaluation dimensions and evaluation problems, thereby affecting the quality of the evaluation analysis processing.

S303: and carrying out standardized detection on the deduplication analysis sample to obtain an effective evaluation sample, wherein the effective evaluation sample comprises at least one evaluation problem and an evaluation dimension corresponding to each evaluation problem.

The standardized detection is a process for detecting whether the text content in the deduplication analysis sample meets the subsequent evaluation analysis requirement. For example, the evaluation analysis requirement may be a formal requirement including the evaluation question and the corresponding original evaluation dimension or other formal requirements.

In this embodiment, the server may perform normalized detection on the text content in the deduplication analysis sample by using a regular expression matching algorithm to detect whether the text content of the deduplication analysis sample meets the form requirement in the evaluation analysis requirement, and if so, determine the deduplication analysis sample as the effective evaluation sample. The evaluation problem and the corresponding original evaluation dimension can be extracted from the evaluation analysis sample in the subsequent evaluation analysis process, and the implementability of the scheme can be guaranteed.

S304: and if the sample number of the effective evaluation samples with the same type of the original product and the target product is greater than or equal to the preset number, taking all the effective evaluation samples corresponding to the type of the target product as evaluation analysis samples.

The preset number is a preset number, and is specifically a sample number required for evaluating whether an evaluation analysis model corresponding to any product type is analyzed. The sample number of the effective evaluation samples with the same type of the original product and the target product refers to the sample number corresponding to the effective evaluation sample with the same type of the original product and the target product, which is screened from all the effective evaluation samples obtained from the sample database.

Specifically, the server screens out effective evaluation samples with the same original product type and target product type from all effective evaluation samples acquired from a sample database, and compares the sample number with a preset number after determining the corresponding sample number; and if the number of the samples is greater than or equal to the preset number, taking all effective evaluation samples corresponding to the target product type as evaluation analysis samples. The determination of the evaluation analysis sample not only ensures that the number of the evaluation analysis sample reaches the preset number required by the analysis evaluation analysis model to ensure the objectivity of the subsequent evaluation analysis model determined based on the evaluation analysis sample and the corresponding evaluation index, but also avoids the condition that the finally determined evaluation index has one-sidedness due to the over-small number of the sample; and all the evaluation analysis samples correspond to the types of the target products, so that evaluation indexes in subsequently acquired evaluation analysis models have higher pertinence.

S305: if the sample number of the effective evaluation samples with the same type of the original product and the target product is smaller than the preset number, obtaining the residual number based on the preset number and the sample number, taking all the effective evaluation samples corresponding to the type of the target product as evaluation analysis samples, and randomly selecting the residual number of the effective evaluation samples from the effective evaluation samples with different types from the type of the target product as the evaluation analysis samples.

Specifically, the server screens out effective evaluation samples with the same original product type and target product type from all effective evaluation samples acquired from a sample database, and compares the sample number with a preset number after determining the corresponding sample number; if the number of samples is smaller than the preset number, determining the residual number based on the difference between the preset number and the number of samples. Then, all the effective evaluation samples with the same type as the target product are used as evaluation analysis samples, and the rest of the effective evaluation samples are randomly selected from the effective evaluation samples with different types from the target product to be used as the evaluation analysis samples. All effective evaluation samples which correspond to the number of the samples and are the same as the types of the target products and all effective evaluation samples which correspond to the number of the samples and are different from the types of the target products and are the same as the number of the residual samples are determined as evaluation analysis samples, so that the number of the final evaluation analysis samples reaches a preset number, the objectivity of a subsequent evaluation analysis model determined based on the evaluation analysis samples and the corresponding evaluation indexes of the evaluation analysis model is guaranteed, and the condition that the finally determined evaluation indexes have one-sidedness due to the fact that the number of the samples is too small is avoided; and the pertinence of the evaluation indexes in the subsequently obtained evaluation analysis model can be guaranteed to the maximum extent.

In the software product evaluation processing method provided by this embodiment, duplicate removal processing and normalized detection are performed on an original analysis sample to obtain an effective evaluation sample, thereby achieving the purposes of eliminating interference and ensuring the implementability of a scheme. According to the comparison result of the number of the samples of the effective evaluation samples with the same type as the target product and the preset number, selecting the effective evaluation samples with the number larger than or equal to the preset number to determine the effective evaluation samples as evaluation analysis samples so as to ensure the objectivity of a subsequent evaluation analysis model determined based on the evaluation analysis samples and the corresponding evaluation indexes of the evaluation analysis model; the pertinence of the evaluation analysis model determined based on the evaluation analysis sample and the corresponding evaluation index can be guaranteed to the maximum extent.

In an embodiment, as shown in fig. 4, in step S203, performing importance analysis on the original evaluation dimensions corresponding to all the evaluation questions to obtain at least two target evaluation dimensions, which specifically includes the following steps:

s401: and carrying out synonym analysis on the original evaluation dimensions corresponding to all the evaluation problems to obtain at least one original synonym set.

And if the original evaluation dimensions are synonyms, the original evaluation dimensions corresponding to the synonyms are stored in an original synonym set. The original synonym set is a set for storing at least two original evaluation dimensions that are synonyms of each other. In this embodiment, the synonym analysis refers to a process of querying a predefined synonym association relationship in a synonym library based on any two original evaluation dimensions to determine whether the two are synonyms.

In this embodiment, step S401 specifically includes the following steps: (1) an array to be analyzed is created based on all original evaluation dimensions (such as L original evaluation dimensions), an original synonym set is created based on the 1 st original evaluation dimension, that is, the 1 st original evaluation dimension is stored in the original synonym set. (2) Traversing the 2 nd to L th original evaluation dimensions in the array to be analyzed, carrying out synonym analysis on the original evaluation dimensions and any original evaluation dimension in a pre-created original synonym set, namely querying a synonym library based on the two original evaluation dimensions, and determining whether the two original evaluation dimensions are synonyms or not according to a synonym association relation defined in advance by the synonym library. (2) And if the evaluation dimensions are synonyms, storing the 2 nd to L th original evaluation dimensions in the array to be analyzed in the original synonym set. (4) If the two dimensions are not synonyms, a new original synonym set … … is created based on the 2 nd to L original evaluation dimensions in the array to be analyzed, and so on until the L th original evaluation dimension in the array to be analyzed is traversed. For example, in the synonym library, it may be predefined that nouns such as "easy to use", "easy to operate", and "easy to operate" have synonym association relationships.

S402: and counting the number of single problems corresponding to each original evaluation dimension and the number of summary problems corresponding to all the original evaluation dimensions in each original synonym set.

As above, any original evaluation dimension corresponds to at least one evaluation question (i.e., a 1-to-M relationship). Counting the number of single questions corresponding to each original evaluation dimension in each original synonym set, specifically, counting the number of evaluation questions corresponding to each original evaluation dimension, that is, determining the specific numerical value of M in the relationship of 1 pair M. Counting the number of the summary problems corresponding to all the original evaluation dimensions in each original synonym set, specifically, counting the Sum of the number of the single problems corresponding to all the original evaluation dimensions in the original synonym set, that is, determining the Sum M. For example, if three original evaluation dimensions, a1, a2, and A3, are included in a set of original synonyms, the number of single questions is 20, 30, and 20, respectively, and the number of summary questions is 70.

S403: and sequencing the quantity of the summarized problems corresponding to all the original synonym sets, and determining the top X original synonym sets with the maximum quantity of the summarized problems as target synonym sets.

Specifically, the server ranks the number of summary problems corresponding to all original evaluation dimensions in each original synonym set, selects a larger number of first X original synonym sets to determine the target synonym set, and can determine whether the finally determined target synonym set can reflect the original evaluation dimensions most concerned when the existing software product corresponding to the target product type is evaluated, so that an evaluation analysis model corresponding to the target product type is constructed based on the original evaluation dimensions, and the evaluation index required by software product evaluation is determined to be reflected most based on the evaluation analysis model. The target synonym set refers to the top X-bit original synonym set with the maximum number of summary problems determined from all the original synonym sets, namely the target synonym set corresponding to the target product type is determined based on a majority principle so as to guarantee the objectivity determined by the target synonym set.

S404: and determining the original evaluation dimension with the largest number of single questions as the target evaluation dimension according to the number of single questions corresponding to each original evaluation dimension in the target synonym set.

Specifically, the server compares the number of the single questions corresponding to all the original evaluation dimensions according to the number of the single questions corresponding to each original evaluation dimension in the target synonym set, so that the original evaluation dimension with the largest number of the single questions is determined as the target evaluation dimension. For example, in a target synonym set, the original evaluation dimensions such as "easy to use", "easy to operate", and "easy to operate" are included, and the number of single questions corresponding to the original evaluation dimension of "easy to use" is 60, the number of single questions corresponding to the original evaluation dimension of "easy to operate" is 40, and the number of single questions corresponding to the original evaluation dimension of "easy to operate" is 30, and the original evaluation dimension with the largest number of single questions is "easy to use", and therefore "easy to use" can be determined as the target evaluation dimension corresponding to the target synonym set.

In the software product evaluation processing method provided by this embodiment, synonym analysis is performed on the original evaluation dimensions to determine a corresponding original synonym set, so as to implement classification processing on the original evaluation dimensions, which is helpful for ensuring the objectivity and accuracy of the determined target evaluation dimensions, and avoiding the occurrence of a situation that a single original evaluation dimension cannot meet the standard determined as the target evaluation dimension and a plurality of synonym combinations can meet the standard determined as the target evaluation dimension; or to avoid determining two or more target evaluation dimensions having the same meaning. And determining the original synonym set with the largest quantity of the summarized problems as a target synonym set so as to eliminate the interference of the original synonym set with the smaller quantity of the summarized problems on the determination of the target evaluation dimension and improve the determination efficiency of the target evaluation dimension. And determining the original evaluation dimension with the largest number of single problems in each target synonym set as a target evaluation dimension so as to ensure the general applicability of the target evaluation dimension in the evaluation process of the software product corresponding to the target product type, namely adopting the expression to define the target evaluation dimension in more evaluation analysis samples.

In an embodiment, as shown in fig. 5, in step S403, the total number of questions corresponding to all the original synonym sets is ranked, and the top X original synonym sets with the largest total number of questions are determined as the target synonym set, which specifically includes the following steps:

s501: and performing descending sorting on the quantity of the summary problems corresponding to all the original synonym sets to obtain descending sorting results.

The descending sorting result refers to a sorting result determined after descending sorting is performed on the basis of the number of the summary problems corresponding to all the original synonym sets. For example, the total number of questions corresponding to the original synonym sets of Q1, Q2, Q3, Q4 and Q5 is 400, 60, 200, 300 and 40 respectively, the descending sorting result after the descending sorting of the total number of questions is 400, 300, 200, 60 and 40, and the sorting of the corresponding original synonym sets is Q1, Q4, Q3, Q2 and Q5.

S502: and counting the accumulated sum of the first X summary problem quantities corresponding to the descending sorting result, and calculating the summary sum of all summary problem quantities.

In particular, the server employs

The calculation formula counts the cumulative sum of the first X total problem numbers corresponding to the descending sorting result, that is, P1 is the cumulative sum of the first X total problem numbers corresponding to the descending sorting result, X_iAnd sorting the ith quantity of the summary problems in the result in a descending order. Accordingly, the server adopts

A summary sum of all summary issue quantities is calculated. Where P2 is the sum of the total number of questions for all the original synonym sets, and H is the number of all the original synonym sets.

S503: and determining a target proportion value based on the accumulated sum value and the summarized sum value, and if the target proportion value is greater than a first proportion threshold, determining the original synonym set corresponding to the first X summarized problems as the target synonym set.

Wherein the first proportion threshold is a preset threshold for limiting reaching the standard proportion determined as the target synonym set. Specifically, the server calculates and determines a target proportion value P corresponding to the first X total problem quantities by using P1/P2 based on the cumulative sum value P1 of the first X total problem quantities and the total sum value P2 of all total problem quantities. Then, comparing the target proportional value P with a first proportional threshold value; if the target proportion value P is larger than the first proportion threshold, determining the original synonym set corresponding to the first X summary problem quantities as a target synonym set; and if the target proportion value P is not larger than the first proportion threshold value, re-assigning the value of X by adopting X +1, and repeatedly executing the step S501 and the step S502 until the value of X is determined, so as to select the first X synonym sets from the original synonym sets corresponding to the descending ordering result and determine the first X synonym sets as target synonym sets. For example, if the first scale threshold is 70%, the descending sort results after the total question number is sorted in descending order are 400, 300, 200, 60 and 40, and the corresponding original synonym set is sorted into Q1, Q4, Q3, Q2 and Q5, if X is 1, the calculated P is 400/1000 which is 40%, and is not greater than 75%; let X be 2, calculated P700/1000 be 70%, not greater than 75%; if X is 3, the calculated P is 800/1000 is 80%, and is greater than 75%, the finally determined value of X is 3, and the first 3 original synonym sets are selected from Q1, Q4, Q3, Q2, and Q5 as the target synonym sets, that is, Q1, Q4, and Q3, from the descending sorting results.

In the software product evaluation processing method provided by this embodiment, a target proportion value is determined based on an accumulated sum value and a summarized sum value, and a value of X is determined according to a comparison result between the target proportion value and a first proportion threshold, so that the first X original synonym sets are selected from the original synonym sets corresponding to the descending order result and determined as a target synonym set, so as to ensure that the target synonym set is an original synonym set corresponding to a larger proportion of summarized problem quantity, so that a target evaluation dimension determined based on the target synonym set has objectivity and accuracy, and thus, the evaluation has general applicability in the software product corresponding to the target product type.

In an embodiment, as shown in fig. 6, in step 204, performing importance analysis on all the evaluation questions corresponding to each target evaluation dimension to obtain an evaluation index corresponding to the target evaluation dimension, which specifically includes the following steps:

s601: and extracting keywords of the evaluation questions corresponding to all original evaluation dimensions in the target synonym set corresponding to each target evaluation dimension to obtain the functional keywords corresponding to each evaluation question.

Specifically, the server extracts keywords from each evaluation problem corresponding to each original evaluation dimension in a target synonym set corresponding to each target evaluation dimension by using a keyword extraction algorithm, obtains a functional keyword corresponding to each evaluation problem, that is, determines all functional keywords determined by the target synonym set as the functional keywords corresponding to the target evaluation dimension. In this embodiment, the process of extracting the keywords from each evaluation problem by using the keyword extraction algorithm may include performing word segmentation on the evaluation problem to obtain a plurality of words; and performing part-of-speech tagging on each word segmentation to determine corresponding functional keywords. For example, after word segmentation processing is performed on the transfer function, the transfer, the function, the whether or not and the smooth are obtained, and the transfer key word is determined according to the part of speech of the words.

S602: and merging synonyms of the functional keywords corresponding to each evaluation problem to obtain an original index.

Specifically, the server performs synonym merging processing on the functional keywords corresponding to each evaluation question, so as to merge a plurality of functional keywords which are synonyms of each other into an original index. Specifically, the server can query the synonym library based on any two functional keywords, and determine whether the two are synonyms according to a predefined synonym association relation in the synonym library; if the two are synonyms, the original index can be determined directly according to the standard word corresponding to the synonym association relationship. For example, a synonym library is predefined that "transfer" and "remittance" have synonym association relationship, and the corresponding standard word is "transfer"; when the server identifies that the functional keywords corresponding to the target evaluation dimension comprise 'transfer' and 'remittance', the server queries the synonym library to determine that the synonym library and the target evaluation dimension have synonym association relation, and then directly determines the standard word 'transfer' as an original index. Understandably, the synonym merging processing is carried out on the functional keywords to determine the original indexes, so that the number of the original indexes can be effectively reduced, and the accuracy of the finally determined evaluation indexes is ensured.

S603: and determining the number of the evaluation problems containing the original indexes as the number of the index problems corresponding to the original indexes, and counting the number of the dimension problems corresponding to all the evaluation problems.

Specifically, the server determines the number of evaluation questions including the original index as the index question number S1 corresponding to the original index, where the number of evaluation questions including the original index is specifically the sum of the number of evaluation questions including the noun "original index" and the number of evaluation questions including the synonym corresponding to the "original index". For example, if the original index is "transfer", the server screens out the sum of the number of evaluation questions including the noun of "transfer" and the number of evaluation questions including a noun (e.g., "remittance") synonymous with "transfer" from among the evaluation questions corresponding to the target evaluation dimension, and determines the sum as the index question number S1 corresponding to the original index.

Specifically, the server counts the number of dimension problems S2 corresponding to all the evaluation problems, which means that the server counts the number of all the evaluation problems corresponding to the target evaluation dimension, that is, if the number of evaluation problems corresponding to a target evaluation dimension is 1000, the number of dimension problems is 1000.

S604: and if the problem ratio of the index problem quantity corresponding to the original index to the dimension problem quantity is larger than a second ratio threshold value, determining the original index as the evaluation index corresponding to the target evaluation dimension.

Wherein the second ratio threshold value is a threshold value set in advance for limiting reaching of a standard ratio determined as an evaluation index. Specifically, the server calculates a problem ratio S of the index problem number S1 corresponding to the original index to the dimension problem number S2 by using S1/S2. Then, comparing the problem ratio S with a second ratio threshold; and if the problem ratio S is larger than a second ratio threshold, determining the original index as the evaluation index corresponding to the target evaluation dimension. For example, the second percentage threshold is 10%, and if the problem percentage S corresponding to an original index is 15%, the original index is an evaluation index.

In the software product evaluation processing method provided in this embodiment, a problem ratio between the index problem quantity corresponding to the original index and the dimension problem quantity is calculated, and according to a comparison result between the problem ratio and a second ratio threshold, which original indexes are determined as evaluation indexes, so that the determined evaluation indexes have universality, that is, the method is generally applicable to software products corresponding to target product types.

In an embodiment, as shown in fig. 7, in step S206, performing consistency check on the target evaluation question corresponding to each evaluation index to obtain a consistency check result, which specifically includes the following steps:

701: and generating a product internal test request based on each target evaluation question, sending the product internal test request to at least two evaluation terminals corresponding to target evaluation personnel, and acquiring to-be-analyzed evaluation data which is returned by the evaluation terminals and is determined based on the target evaluation questions, wherein the to-be-analyzed evaluation data comprises evaluation scores corresponding to each target evaluation question.

And the target evaluation question is determined to correspond to the software product to be tested based on the evaluation index in the target evaluation analysis model corresponding to the target product type. The evaluation terminal is used for a target evaluation person to make, so that the target evaluation person can receive the product internal test request through the evaluation terminal and upload the evaluation data to be analyzed. The target evaluating personnel are professional practitioners for carrying out internal evaluation on the software product to be tested, generally are user researchers, product managers and interactive designers, and can effectively guarantee the consistency of the internal evaluation. The evaluation data to be analyzed is the evaluation data which is uploaded by the target evaluation personnel and determined when the target evaluation problem is evaluated. The evaluation data to be analyzed comprises evaluation scores corresponding to the target evaluation questions.

Specifically, the server sends the in-product test request to the test terminals corresponding to the at least two target test personnel, so that the at least two target test personnel test the software product to be tested based on the target test question, and obtains test data to be analyzed, determined based on the target test question, returned by each target test personnel through the test terminal, and each test data to be analyzed is a test score determined when the target test personnel tests the target test question.

702: and carrying out consistency check on the evaluation scores corresponding to the target evaluation questions to obtain Kendel harmony coefficients.

The Kendall (kandall) harmony coefficient is a correlation quantity for calculating the correlation degree of a plurality of grade variables, and the Kendall harmony coefficient is suitable for data which are multi-column related grade data, namely k scorers score (N) objects, and the same person successively scores the N objects k times. In this embodiment, when configuring the target evaluation problem corresponding to each evaluation index and the evaluation description of the target evaluation problem, the user may be prompted to perform rating evaluation so that the finally obtained evaluation score is a specific evaluation rating; or, a grade comparison table is configured in advance, and the grade comparison table is used for explaining the evaluation grade corresponding to each evaluation score, so that the grade comparison table is queried according to the obtained evaluation scores, the evaluation grade corresponding to the evaluation score is determined, and the Kendel harmony coefficient can be conveniently calculated.

In this embodiment, when the same target evaluator does not have the same evaluation level evaluation, formula (1) is adopted

Calculating Kendel harmony coefficients; when the same target appraiser has the same appraisal level appraisal, the formula (2) is adopted

A Kendell harmonic coefficient is calculated, wherein,

n is the number of questions of the target evaluation question, K is the number of persons of the target evaluation person, R_iIs the sum of the evaluation grades corresponding to all the target evaluation questions, and S is the sum of the evaluation grades corresponding to all the target evaluation questions R_iAnd the sum of all evaluation grades R_iSum of squared deviations of the mean of (1), m_iThe number of the evaluation results of the ith target evaluator without repeated evaluation grades, n_ijThe same grade number of the jth repeated evaluation grade in the evaluation result of the ith target evaluation person.

703: and if the Kendell harmony coefficient is larger than the preset coefficient threshold value, the obtained consistency verification result is that the verification is passed.

If the preset coefficient threshold value can be set to 0.9, that is, when it is stated that the consistency of the evaluation scores corresponding to the evaluation indexes (that is, the kender harmony coefficient) needs to reach 0.9 when all target evaluation personnel evaluate the software product to be evaluated, the consistency check result corresponding to the software product to be evaluated is determined to be passed, which indicates that the acceptance of the target evaluation personnel on the target evaluation problem determined by the software product to be evaluated is relatively close, so as to avoid the one-sidedness of the target evaluation problem.

Specifically, the Kendell harmony coefficient determined according to the evaluation score corresponding to the evaluation index can reflect the consistency of the evaluation of the target evaluation questions set by the evaluation index in the target evaluation analysis model by a plurality of target evaluation personnel; if the consistency is higher, the result shows that the degrees of identity of the target evaluation questions set by the evaluation indexes by the target evaluation personnel are closer, the user experience requirements of the target evaluation personnel can be determined to be met, the consistency verification result is determined to be passed through verification, and therefore product evaluation questionnaire information can be generated based on the target evaluation questions. Otherwise, if the consistency is low, it indicates that the identity of the target evaluation questions set by the evaluation indexes is greatly deviated by the target evaluation personnel, and indicates that the software product to be tested needs to be improved on the target evaluation questions set by the evaluation indexes with low consistency, so that the setting of the target evaluation questions is more objective and reasonable.

In the software product evaluation processing method provided by this embodiment, when consistency check is performed on target evaluation problems of a software product to be tested, consistency check is performed on evaluation scores of the target evaluation problems corresponding to evaluation indexes in the target evaluation data to be analyzed by collecting evaluation data to be analyzed fed back by at least two target evaluation personnel to obtain a kender harmony coefficient, and when the kender harmony coefficient is greater than a preset coefficient threshold value, a consistency check result is determined as a check pass to ensure objectivity and comprehensiveness of target evaluation problem setting, which is beneficial to improving optimization efficiency of the software product to be tested.

In an embodiment, as shown in fig. 8, after step S208, that is, after embedding the product evaluation questionnaire information into the evaluation interface of the software product to be tested to form the target software product, the software product evaluation processing method further includes:

s801: and acquiring evaluation investigation data input by an evaluation interface based on the target software product, wherein the evaluation investigation data comprises an evaluation functional module corresponding to the target evaluation problem and an evaluation tendency.

Wherein, the evaluation research data is data which is input on an evaluation interface by a user of the target software product and is used for reflecting the idea in the using process. In this embodiment, when a user uses a target software product and considers that there is a problem or an improved idea in implementing some function modules of the target software product, the user may click an evaluation button of the target software product to enter an evaluation interface, so that the evaluation interface displays at least one target evaluation question and a to-be-selected item corresponding to the target evaluation question (i.e., a plurality of options preset for the target evaluation question) or an input box corresponding to the target evaluation question (i.e., an input box for the user to autonomously input information), and the user inputs corresponding evaluation investigation data through the to-be-selected item or the input box.

The evaluation function module refers to a function module for a certain target evaluation question. The evaluation and review tendency is used for reflecting the user experience tendency of the evaluation function module aiming at the target evaluation problem of the user corresponding to the evaluation and research data. For example, if the to-be-evaluated item of the target evaluation question of whether the login function of the APP is good to use or not comprises two types of "good to use" and "bad to use", the corresponding evaluation function module is the login function module, and if the user clicks "good to use", the evaluation comment tendency is positive comment; if the user clicks 'not good use', the evaluation comment of the user tends to be a negative comment.

S802: and counting the total number of the comments corresponding to any one evaluation functional module and the number of the negative comments with the evaluation comment tendency as the negative comments in real time, and determining the proportion of the negative comments corresponding to the evaluation functional module.

Specifically, the server counts the number of all evaluation investigation data corresponding to any evaluation functional module in real time, and determines the total comment data corresponding to the evaluation functional module; determining the number of the evaluation investigation data with the evaluation tendency of negative comments in all the evaluation investigation data corresponding to the evaluation functional module as the number of the negative comments corresponding to the evaluation functional module; then, the quotient of the number of the negative comments and the total number of the comments is determined as the negative comment proportion corresponding to the evaluation function module.

S803: if the negative comment proportion is larger than a preset proportion threshold value, a comment early warning mechanism is triggered, and modification reminding information corresponding to the evaluation function module is generated.

The preset proportion threshold is a preset proportion value used for limiting whether early warning is needed or not. The comment early warning mechanism is a preset mechanism for limiting the corresponding response time and the response flow. The response time is a time for responding to the evaluation function module in which the negative comment proportion is greater than the preset proportion threshold. The response process is a process for responding to the evaluation function module with the negative comment proportion larger than the preset proportion threshold value and corresponding processing personnel, so that the response processing is more reasonable and efficient, the response efficiency is ensured, and the optimization efficiency of the software product is improved. The modification reminding information is information for reminding the corresponding processing personnel to modify.

In the software product evaluation processing method provided by this embodiment, the negative comment proportion of any evaluation function module is calculated through statistics, and when the negative comment proportion is greater than a preset proportion threshold, a comment early warning mechanism is triggered, so that modification of evaluation function modules with more negative comments is realized, and the optimization efficiency of a software product is improved.

It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

In an embodiment, a software product evaluation processing device is provided, and the software product evaluation processing device corresponds to the software product evaluation processing method in the above embodiment one to one. As shown in fig. 9, the functional modules are explained in detail as follows:

the evaluation analysis request obtaining module 901 is configured to obtain an evaluation analysis request, where the evaluation analysis request includes a software product to be detected and a target product type corresponding to the software product to be detected.

An evaluation analysis sample acquisition module 902, configured to query an evaluation questionnaire information base based on a target product type, and if the evaluation questionnaire information base does not store original evaluation questionnaire information corresponding to the target product type, acquire a corresponding evaluation analysis sample based on the target product type, and extract at least one evaluation problem corresponding to each evaluation analysis sample and an original evaluation dimension corresponding to each evaluation problem.

And the target evaluation dimension acquisition module 903 is configured to perform importance analysis on the original evaluation dimensions corresponding to all the evaluation questions to acquire at least two target evaluation dimensions.

The evaluation index obtaining module 904 is configured to perform importance analysis on all evaluation questions corresponding to each target evaluation dimension, and obtain an evaluation index corresponding to the target evaluation dimension.

The evaluation analysis model obtaining module 905 is configured to use structural equation model analysis software to perform model construction on at least two target evaluation dimensions and the evaluation index corresponding to each target evaluation dimension, so as to obtain a target evaluation analysis model.

The consistency check result obtaining module 906 is configured to obtain a target evaluation problem corresponding to each evaluation index based on the evaluation indexes in the target evaluation analysis model, perform consistency check on the target evaluation problem corresponding to each evaluation index, and obtain a consistency check result.

And the evaluation questionnaire information acquisition module 907 is used for acquiring product evaluation questionnaire information based on target evaluation problems corresponding to all evaluation indexes if the consistency check result is that the check is passed, and storing the product evaluation questionnaire information and the target product type in an evaluation questionnaire information base in an associated manner.

And the target software product acquisition module 908 is configured to embed the product evaluation questionnaire information into an evaluation interface of the software product to be tested based on the information acquisition interface on the software product to be tested, so as to form a target software product.

Preferably, the evaluation analysis sample acquisition module 902 includes: the device comprises an original analysis sample obtaining unit, a de-duplication analysis sample obtaining unit, an effective evaluation sample obtaining unit, a first evaluation sample obtaining unit and a second evaluation sample obtaining unit.

The original analysis sample acquisition unit is used for acquiring original analysis samples from a sample database, and each original analysis sample corresponds to an original product type.

And the de-duplication analysis sample acquisition unit is used for carrying out de-duplication processing on the original analysis sample to acquire a de-duplication analysis sample.

And the effective evaluation sample obtaining unit is used for carrying out standardized detection on the deduplication analysis sample to obtain an effective evaluation sample, and the effective evaluation sample comprises at least one evaluation problem and an evaluation dimension corresponding to each evaluation problem.

The first evaluation sample obtaining unit is used for taking all effective evaluation samples corresponding to the target product type as evaluation analysis samples if the sample number of the effective evaluation samples of which the original product type is the same as the target product type is greater than or equal to a preset number.

And the second evaluation sample obtaining unit is used for obtaining the residual quantity based on the preset quantity and the sample quantity if the sample quantity of the effective evaluation samples of which the original product types are the same as the target product types is less than the preset quantity, taking all the effective evaluation samples corresponding to the target product types as evaluation analysis samples, and randomly selecting the residual quantity of the effective evaluation samples from the effective evaluation samples different from the target product types as the evaluation analysis samples.

Preferably, the target evaluation dimension obtaining module 903 includes: the system comprises an original synonym set acquisition unit, a set problem quantity acquisition unit, a target synonym set acquisition unit and a target evaluation dimension determination unit.

And the original synonym set acquisition unit is used for carrying out synonym analysis on the original evaluation dimensions corresponding to all the evaluation problems to acquire at least one original synonym set.

And the set problem quantity acquisition unit is used for counting the single problem quantity corresponding to each original evaluation dimension and the summary problem quantity corresponding to all the original evaluation dimensions in each original synonym set.

And the target synonym set acquisition unit is used for sequencing the summarized problem quantity corresponding to all the original synonym sets and determining the top X original synonym sets with the most summarized problem quantity as the target synonym sets.

And the target evaluation dimension determining unit is used for determining the original evaluation dimension with the largest single question number as the target evaluation dimension according to the single question number corresponding to each original evaluation dimension in the target synonym set.

Preferably, the target synonym set acquiring unit includes: the device comprises a descending sorting result acquisition subunit, a sum value statistical processing subunit and a proportion value calculation processing subunit.

And the descending sorting result obtaining subunit is used for carrying out descending sorting on the quantity of the summary problems corresponding to all the original synonym sets and obtaining a descending sorting result.

And the sum value statistical processing subunit is used for counting the cumulative sum value of the first X summary problem quantities corresponding to the descending sorting result and calculating the summary sum value of all the summary problem quantities.

And the proportion value calculation processing subunit is used for determining a target proportion value based on the accumulated sum value and the summarized sum value, and if the target proportion value is greater than a first proportion threshold, determining the original synonym set corresponding to the first X summarized problems as the target synonym set.

Preferably, the evaluation index obtaining module 904 includes: the system comprises a function keyword acquisition unit, an original index acquisition unit, an evaluation question quantity acquisition unit and an evaluation index determination unit.

And the function keyword acquisition unit is used for extracting keywords of the evaluation questions corresponding to all original evaluation dimensions in the target synonym set corresponding to each target evaluation dimension to acquire the function keywords corresponding to each evaluation question.

And the original index acquisition unit is used for merging synonyms of the functional keywords corresponding to each evaluation problem to acquire an original index.

And the evaluation problem quantity acquisition unit is used for determining the quantity of the evaluation problems containing the original indexes as the index problem quantity corresponding to the original indexes and counting the dimension problem quantity corresponding to all the evaluation problems.

And the evaluation index determining unit is used for determining the original index as the evaluation index corresponding to the target evaluation dimension if the problem ratio of the index problem quantity corresponding to the original index to the dimension problem quantity is greater than a second ratio threshold.

Preferably, the consistency check result obtaining module 906 includes: the device comprises an evaluation data acquisition unit to be analyzed, a Kendell harmony coefficient acquisition unit and an evaluation result acquisition unit.

And the to-be-analyzed evaluation data acquisition unit is used for generating a product internal evaluation request based on each target evaluation question, sending the product internal evaluation request to the evaluation terminals corresponding to at least two target evaluation personnel, and acquiring to-be-analyzed evaluation data which is returned by the evaluation terminals and is determined based on the target evaluation questions, wherein the to-be-analyzed evaluation data comprises evaluation scores corresponding to each target evaluation question.

And the Kendel harmony coefficient acquisition unit is used for carrying out consistency check on the evaluation scores corresponding to the target evaluation problems to acquire the Kendel harmony coefficient.

And the evaluation result acquisition unit is used for acquiring the consistency check result as a check pass if the Kendell harmony coefficient is larger than the preset coefficient threshold.

Preferably, after the target software product acquisition module 908, the product evaluation processing device further includes: the system comprises a test and investigation data acquisition module, a negative comment proportion acquisition unit and a comment early warning processing unit.

And the evaluation investigation data acquisition module is used for acquiring evaluation investigation data input on the basis of an evaluation interface of the target software product, and the evaluation investigation data comprises an evaluation function module corresponding to the target evaluation problem and an evaluation tendency.

And the negative comment proportion acquisition unit is used for counting the total number of comments corresponding to any one evaluation functional module and the number of negative comments with the evaluation comment tendency as negative comments in real time and determining the negative comment proportion corresponding to the evaluation functional module.

And the comment early warning processing unit is used for triggering a comment early warning mechanism and generating modification reminding information corresponding to the evaluation function module if the negative comment proportion is greater than a preset proportion threshold value.

For the specific limitations of the software product evaluation processing device, reference may be made to the limitations of the software product evaluation processing method above, and details are not repeated here. The modules in the software product evaluation processing device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 10. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing data adopted or generated in the process of executing the software product evaluation processing method, such as a target evaluation analysis model. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a software product evaluation processing method.

In an embodiment, a computer device is provided, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and when the processor executes the computer program, the steps of the software product evaluation processing method in the foregoing embodiments are implemented, for example, steps S201 to S204 shown in fig. 2 or steps shown in fig. 3 to fig. 8, which are not described herein again to avoid repetition. Alternatively, the processor implements the functions of each module/unit in the embodiment of the software product evaluation processing apparatus when executing the computer program, for example, the functions of each module/unit shown in fig. 9, and are not described herein again to avoid repetition.

In an embodiment, a computer-readable storage medium is provided, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the software product evaluation processing method in the foregoing embodiments are implemented, for example, steps S201 to S204 shown in fig. 2 or steps shown in fig. 3 to fig. 8, and are not repeated here to avoid repetition. Alternatively, the computer program, when executed by the processor, implements the functions of each module/unit in the embodiment of the software product evaluation processing apparatus, for example, the functions of each module/unit shown in fig. 9, and is not described herein again to avoid repetition.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above may be implemented by hardware instructions of a computer program, which may be stored in a non-volatile computer-readable storage medium, and when executed, may include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions.

The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.

Claims

1. A software product evaluation processing method is characterized by comprising the following steps:

2. The software product evaluation processing method of claim 1, wherein the obtaining of the corresponding evaluation analysis sample based on the target product type comprises:

acquiring original analysis samples from a sample database, wherein each original analysis sample corresponds to an original product type;

carrying out duplicate removal processing on the original analysis sample to obtain a duplicate removal analysis sample;

carrying out standardized detection on the deduplication analysis sample to obtain an effective evaluation sample, wherein the effective evaluation sample comprises at least one evaluation problem and an evaluation dimension corresponding to each evaluation problem;

if the sample number of the effective evaluation samples with the same original product type and the target product type is larger than or equal to a preset number, taking all the effective evaluation samples corresponding to the target product type as evaluation analysis samples;

if the sample number of the effective evaluation samples with the same original product type and the target product type is smaller than a preset number, obtaining a residual number based on the preset number and the sample number, taking all the effective evaluation samples corresponding to the target product type as the evaluation analysis samples, and randomly selecting the residual number of the effective evaluation samples as the evaluation analysis samples from the effective evaluation samples with different target product types.

3. The software product evaluation processing method of claim 1, wherein the analyzing the importance of the original evaluation dimensions corresponding to all the evaluation questions to obtain at least two target evaluation dimensions comprises:

performing synonym analysis on original evaluation dimensions corresponding to all the evaluation problems to obtain at least one original synonym set;

counting the number of single problems corresponding to each original evaluation dimension and the number of summary problems corresponding to all the original evaluation dimensions in each original synonym set; the single problem quantity corresponding to each original evaluation dimension is the quantity of the evaluation problems corresponding to each original evaluation dimension, and the summary problem quantity corresponding to all the original evaluation dimensions is the sum of the single problem quantities corresponding to all the original evaluation dimensions in the original synonym set;

ordering the quantity of the summary problems corresponding to all the original synonym sets, and determining the top X original synonym sets with the maximum quantity of the summary problems as target synonym sets;

and determining the original evaluation dimension with the maximum single question number as the target evaluation dimension according to the single question number corresponding to each original evaluation dimension in the target synonym set.

4. The software product evaluation processing method according to claim 3, wherein the step of sorting the number of the summary questions corresponding to all the original synonym sets and determining the top X original synonym sets with the largest number of the summary questions as the target synonym set comprises:

sorting the quantity of the summary problems corresponding to all the original synonym sets in a descending order mode to obtain a descending order result;

counting the accumulated sum of the first X summarized problem quantities corresponding to the descending sorting result, and calculating the accumulated sum of all the summarized problem quantities;

and determining a target proportion value based on the accumulated sum value and the summary sum value, and if the target proportion value is greater than a first proportion threshold, determining the original synonym set corresponding to the first X summary problem quantities as a target synonym set.

5. The software product evaluation processing method of claim 3, wherein the performing importance analysis on all the evaluation questions corresponding to each of the target evaluation dimensions to obtain the evaluation index corresponding to the target evaluation dimension comprises:

extracting keywords of the evaluation questions corresponding to the original evaluation dimensions in the target synonym set corresponding to each target evaluation dimension to obtain functional keywords corresponding to each evaluation question;

merging synonyms of the functional keywords corresponding to each evaluation problem to obtain an original index;

determining the number of the evaluation problems containing the original indexes as the number of index problems corresponding to the original indexes, and counting the number of dimension problems corresponding to all the evaluation problems;

and if the problem ratio of the index problem quantity corresponding to the original index to the dimension problem quantity is larger than a second ratio threshold value, determining the original index as the evaluation index corresponding to the target evaluation dimension.

6. The software product evaluation processing method of claim 1, wherein the performing consistency check on the target evaluation problem corresponding to each evaluation index to obtain a consistency check result comprises:

generating a product internal test request based on each target evaluation question, sending the product internal test request to at least two evaluation terminals corresponding to target evaluation personnel, and obtaining to-be-analyzed evaluation data which is returned by the evaluation terminals and determined based on the target evaluation questions, wherein the to-be-analyzed evaluation data comprises evaluation scores corresponding to each target evaluation question;

carrying out consistency check on the evaluation score corresponding to each target evaluation question to obtain a Kendell harmony coefficient;

and if the Kendell harmony coefficient is larger than a preset coefficient threshold value, the obtained consistency verification result is that the verification is passed.

7. The software product evaluation processing method of claim 1, wherein after the embedding of the product evaluation questionnaire information into the evaluation interface of the software product to be tested to form a target software product, the software product evaluation processing method further comprises:

acquiring evaluation investigation data input on the basis of an evaluation interface of the target software product, wherein the evaluation investigation data comprises an evaluation functional module corresponding to a target evaluation problem and an evaluation comment tendency;

counting the total number of comments corresponding to any one evaluation functional module and the number of negative comments with evaluation comment tendency as negative comments in real time, and determining the proportion of the negative comments corresponding to the evaluation functional module;

and if the negative comment proportion is larger than a preset proportion threshold value, triggering a comment early warning mechanism and generating modification reminding information corresponding to the evaluation function module.

8. A software product evaluation processing apparatus, comprising:

the target evaluation dimension acquisition module is used for performing importance analysis on the original evaluation dimensions corresponding to all the evaluation problems to acquire at least two target evaluation dimensions;

the evaluation index acquisition module is used for performing importance analysis on all the evaluation problems corresponding to each target evaluation dimension to acquire an evaluation index corresponding to the target evaluation dimension;

9. A computer device comprising a memory, a processor and a computer program stored in said memory and executable on said processor, wherein said processor implements the steps of the software product evaluation processing method according to any one of claims 1 to 7 when executing said computer program.

10. A computer-readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the steps of the software product evaluation processing method according to any one of claims 1 to 7.