CN111506510B - Software quality determining method and related device - Google Patents
Software quality determining method and related device Download PDFInfo
- Publication number
- CN111506510B CN111506510B CN202010316245.0A CN202010316245A CN111506510B CN 111506510 B CN111506510 B CN 111506510B CN 202010316245 A CN202010316245 A CN 202010316245A CN 111506510 B CN111506510 B CN 111506510B
- Authority
- CN
- China
- Prior art keywords
- quality
- target
- software
- nodes
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3604—Software analysis for verifying properties of programs
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Stored Programmes (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the application discloses a software quality determining method and a related device, when the target quality of target software is required to be determined, behavior characteristics of feature dimensions corresponding to the target quality can be extracted according to user behavior data of the target software, a node diagram is generated based on the behavior characteristics, nodes in the node diagram can be determined through the feature dimensions, and a connection relationship among the nodes represents a jump relationship among the behavior characteristics identified by the nodes. Therefore, the node diagram can quantitatively show the response mode and time sequence relation of the target software to the user behavior in the process of using the target software by the user, so that the quality parameter of the target quality corresponding to the target software can be conveniently and automatically determined through the node diagram, the quality is not required to be identified by means of manpower, and the quality determining efficiency and accuracy are improved.
Description
Technical Field
The present disclosure relates to the field of data processing, and in particular, to a software quality determining method and related apparatus.
Background
The user can obtain the corresponding functional service by using various types of software installed in the terminal equipment. For example, the user may purchase a new purchase overnight by opening a take-away applet, may play a game by running a game application, etc.
The newly-introduced software is layered endlessly every day, the quality is poor, and users can only know the newly-introduced software through very limited information such as user evaluation and the like before installing and using the newly-introduced software, but the situation that the newly-introduced software does not conform to expected software and even personal information is leaked is still difficult to avoid.
Therefore, various software platforms start to evaluate the quality of the newly developed software, and the evaluation result provides a reference for users. In the related technology of quality assessment adopted at present, the specific quality of the software to be assessed is determined mainly through manual auditing, for example, through the use of the software to be assessed by a special person based on the content checked in the use process.
The manual auditing mode is low in efficiency and high in cost, and is difficult to meet the current software quality assessment requirements.
Disclosure of Invention
In order to solve the technical problems, the application provides a software quality determining method and a related device, which improve the efficiency and the accuracy of determining the software quality.
The embodiment of the application discloses the following technical scheme:
in one aspect, an embodiment of the present application provides a method for determining software quality, where the method includes:
acquiring user behavior data of target software;
according to the target quality to be determined, extracting behavior characteristics of characteristic dimensions corresponding to the target quality from the user behavior data;
Generating a node diagram corresponding to the target software according to the behavior characteristics; the node diagram comprises a plurality of nodes with connection relations, the nodes are determined according to the feature dimensions, and the connection relations among the nodes are used for reflecting jump relations among behavior features identified by the nodes;
and determining a quality parameter of the target software corresponding to the target quality according to the node diagram.
On the other hand, the embodiment of the application provides a software quality determining device, which comprises an acquisition unit, an extraction unit, a generation unit and a determining unit:
the acquisition unit is used for acquiring user behavior data of the target software;
the extraction unit is used for extracting behavior characteristics of characteristic dimensions corresponding to the target quality from the user behavior data according to the target quality to be determined;
the generating unit is used for generating a node diagram corresponding to the target software according to the behavior characteristics; the node diagram comprises a plurality of nodes with connection relations, the nodes are determined according to the feature dimensions, and the connection relations among the nodes are used for reflecting jump relations among behavior features identified by the nodes;
And the determining unit is used for determining a quality parameter of the target software corresponding to the target quality according to the node diagram.
In another aspect, an embodiment of the present application provides a software quality determination device, including a processor and a memory:
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is configured to perform the method of the above aspect according to instructions in the program code.
In another aspect, embodiments of the present application provide a computer-readable storage medium for storing a computer program for performing the method described in the above aspect.
According to the technical scheme, when the target quality of the target software needs to be determined, the behavior characteristics of the feature dimension corresponding to the target quality can be extracted according to the user behavior data of the target software, a node diagram is generated based on the behavior characteristics, the nodes in the node diagram can be determined through the feature dimension, and the connection relationship among the nodes represents the jump relationship among the behavior characteristics identified by the nodes. Therefore, the node diagram can quantitatively show the response mode and time sequence relation of the target software to the user behavior in the process of using the target software by the user, so that the quality parameter of the target quality corresponding to the target software can be conveniently and automatically determined through the node diagram, the quality is not required to be identified by means of manpower, and the quality determining efficiency and accuracy are improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive faculty for a person skilled in the art.
Fig. 1 is an application scenario schematic diagram of a software quality determining method provided in an embodiment of the present application;
fig. 2 is a schematic flow chart of a software quality determining method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a user behavior trace according to an embodiment of the present disclosure;
FIG. 4 is a schematic diagram of a node diagram according to an embodiment of the present disclosure;
FIG. 5 is a schematic flow chart of generating a node map according to an embodiment of the present disclosure;
FIG. 6 is a schematic flow chart of determining target quality of target software by using a graph convolution model according to an embodiment of the present application;
FIG. 7 is a flowchart of another software quality determination method according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of a software quality determining apparatus according to an embodiment of the present application;
Fig. 9 is a schematic structural diagram of a server according to an embodiment of the present application;
fig. 10 is a schematic structural diagram of a terminal device according to an embodiment of the present application.
Detailed Description
Embodiments of the present application are described below with reference to the accompanying drawings.
In order to improve efficiency and accuracy of determining software quality, the embodiment of the application provides a software quality determining method and a related device.
The software quality determination method provided by the embodiment of the application is realized based on artificial intelligence, wherein the artificial intelligence (Artificial Intelligence, AI) is a theory, a method, a technology and an application system which simulate, extend and extend human intelligence by using a digital computer or a machine controlled by the digital computer, sense environment, acquire knowledge and acquire an optimal result by using the knowledge. In other words, artificial intelligence is an integrated technology of computer science that attempts to understand the essence of intelligence and to produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence, i.e. research on design principles and implementation methods of various intelligent machines, enables the machines to have functions of sensing, reasoning and decision.
The artificial intelligence technology is a comprehensive subject, and relates to the technology with wide fields, namely the technology with a hardware level and the technology with a software level. Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.
In the embodiments of the present application, the artificial intelligence software technology mainly includes the machine learning/deep learning directions. For example, deep Learning (ML) may be involved in Machine Learning (ML), including various types of artificial neural networks (Artificial Neural Network, ANN).
In order to facilitate understanding of the technical scheme of the application, the software quality determining method provided by the embodiment of the application is described below in connection with an actual application scene.
The software quality determining method provided by the application can be applied to software quality determining equipment with data processing capability, such as terminal equipment and a server. The terminal equipment can be a smart phone, a computer, a personal digital assistant (Personal Digital Assistant, PDA), a tablet personal computer and the like; the server may be an independent server or a cluster server.
In the embodiment of the application, the data processing device can determine the quality of the software through an artificial neural network model.
The data processing device may be provided with machine learning capabilities. Machine learning is a multi-domain interdisciplinary, involving multiple disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, and the like. It is specially studied how a computer simulates or implements learning behavior of a human to acquire new knowledge or skills, and reorganizes existing knowledge structures to continuously improve own performance. Machine learning is the core of artificial intelligence, a fundamental approach to letting computers have intelligence, which is applied throughout various areas of artificial intelligence. Machine learning and deep learning typically involve techniques such as artificial neural networks.
The software quality determination method provided by the embodiment of the application mainly relates to application to various artificial neural networks.
Referring to fig. 1, fig. 1 is an application scenario schematic diagram of a software quality determining method provided in an embodiment of the present application. For convenience of description, a server is used as an execution subject of the software quality determination method provided in the embodiment of the present application. In the application scenario shown in fig. 1, a server 101 and a terminal device 102 on the user side are included. Wherein the server 101 is used to determine the quality of the target software. The user may use the target software through the terminal device 102.
In practical application, in the process that the user uses the target software by using the terminal equipment, the terminal equipment can generate corresponding user behavior data aiming at the behavior of the user using the target software, and send the user behavior data to the server, so that the server can evaluate the quality of the target software based on the user behavior data.
As shown in fig. 1, the terminal device 102 generates corresponding a user behavior data for the behavior of the user a using the target software, and transmits the corresponding a user behavior data to the server 101, so that the server 101 can obtain the a user behavior data transmitted from the terminal device 102, and evaluate the quality of the target software using the a user behavior data.
Because the user behavior data comprises behavior features of different feature dimensions, in practical application, the behavior features of the feature dimensions corresponding to the target quality can be extracted from the user behavior data according to the target quality to be determined. Wherein the target quality identifies the quality of different aspects of the target software, as shown in fig. 1, the target quality is a quality of service. The behavioral characteristics identify user behavioral data for a characteristic dimension corresponding to the target quality. The behavior characteristics extracted aiming at the target quality form a behavior track with a direction according to the time sequence relation of the terminal equipment responding to the user behavior.
As shown in fig. 1, in a task of evaluating the quality of service of the target software, the feature dimensions corresponding to the quality of service may include a function type, a page position and a duration, and then extracting, from the a user behavior data, behavior features of three feature dimensions corresponding to the quality of service, including: page 1, pay, share, page 2, open fifth generation hypertext markup language (Hyper Text Markup Language, HTML5/h 5) page and exit, so that a behavior trace with a time-sequential direction can be obtained. The behavior trace identifies the behavior of the terminal end device 102 in time-series relation in response to the user's use of the target software.
It will be appreciated that for conventional static software such as text, pictures, etc., the user behavior data generated is of the same type and is of a canonical european style structure, and the quality of the software can be readily determined using natural language processing (Nature Language Processing, NLP) techniques or image recognition techniques. However, dynamic software has many functions, various carriers, various contents, and various structures, and it is difficult to identify the quality of the software by using a single type of technology.
In this regard, in the embodiment of the present application, a node map corresponding to the target software may be generated according to the above-mentioned extracted behavior feature. That is, the task of determining software quality is translated into a node map recognition task using a node map. The node diagram uniformly represents the user behavior data with different feature dimensions in the form of a node network, namely, the purpose of identifying the quality of multiple types of software by using a single technology is realized.
Wherein the node map includes a plurality of nodes having a connection relationship. The nodes in the node graph are determined according to the feature dimensions corresponding to the target quality. In the application scenario shown in fig. 1, a plurality of nodes may be divided according to the function type and the page position, for example, the node n2 indicates that the user uses the sharing function in the page 1 of the target software, and the node n3 indicates that the user opens the h5 page in the page 2 of the target software.
The connection relationship between the nodes is used to represent the jump relationship between the behavior features identified by the nodes, and in fig. 1, the connection relationship between the nodes is represented by a connection line with an arrow between the nodes. For example, the arrowed line between node n1 and node n2 in fig. 1 indicates that user a jumps from the payment function to the sharing function in page 1 during use of the target software. The nodes in the node diagram identify the response mode of the target software in the terminal device 102 to the user behavior, and the connection relations among the nodes in the node diagram identify the time sequence relation of the target software in the terminal device 102 to the user behavior. I.e. the node graph identifies the behavior trace of the user using the target software through the connection relationship between the nodes.
After the server 101 generates the node diagram by using the behavior characteristics, the node diagram can be identified by using an artificial intelligence technology, so as to determine a quality parameter of the target software corresponding to the target quality, wherein the quality parameter identifies the quality of the target software.
Because the node diagram quantitatively reflects the response mode (node) and the time sequence relation (connection relation) of the target software to the user behavior in the process of using the software by the user, the quality parameter of the target quality corresponding to the target software can be conveniently determined by utilizing the node diagram in the task of determining the target quality of the target software, the quality is not required to be identified by means of manual means, and the quality efficiency and the accuracy of determining the software are improved.
Referring to fig. 2, fig. 2 is a flow chart of a method for determining quality of software according to an embodiment of the present application. In fig. 2, the method comprises the steps of:
s201: user behavior data of the target software are obtained.
In practical application, the server may acquire user behavior data for the target software sent from the user side terminal device. The target software refers to application software provided for users to use, and includes but is not limited to: application, applet (Mini Program), etc. Such as social apps, game applets, etc. The user behavior data refers to data generated by the target software in response to the user behavior during the process of using the target software by the user.
The embodiment of the application can be applied to the scene of quality determination for the small program. The applet belongs to an application that can be used without installation and download. Different applets are loaded by the framework of the same application platform and provided for users, for example, the users can run various applets through a certain social platform, and services such as take-out, online shopping and the like are provided for the users conveniently.
The small program has the characteristics of low development cost and short development period, so that the new development speed of the small program is very high, the small program has larger fluctuation of quality compared with the application program and the like due to the small development input cost, the platform ecology can be seriously influenced if a large number of small programs with low values are put into the market, and even the influence and the loss are brought to users, thereby having the necessity of rapidly determining the quality aiming at the small program.
Because different applets run under the framework of the same application platform, even if the situations of back-and-forth jump, call and the like among the applets occur in the process of using the applets by a user, the platform can acquire complete jump and call information in an interface call and the like mode, so that user behavior data aiming at the applets can be acquired more completely, the user behavior data can clearly show a complete response path of the applets when the user uses the applets serving as target software, and a solid foundation is laid for the accuracy of quality determination of the applets.
Further, by the jump and call relation among the applets, the entity association among the applets can be expressed from one angle, such as whether the applets actually belong to the same company, person and the like. The association may be applied to quality determination (described in detail in the embodiments below), or may provide processing basis for other data processing tasks, etc.
For the process of obtaining user behavior data, in one possible implementation, the server may obtain user behavior data generated by the target software over a historical period of time. For example, the server may obtain user behavior data generated by all users using the target software within 30 days of history.
The user behavior data generated by the user during the historical time period using the target software increases the data used to analyze the quality of the target software from the time dimension. Because the user behavior data identifies the software quality, increasing the user behavior data can enrich the embodiment of the software quality and improve the evaluation accuracy of the software quality.
S202: and extracting behavior characteristics of characteristic dimensions corresponding to the target quality from the user behavior data according to the target quality to be determined.
In the actual software development process, various low-quality software which affects the use experience of users, such as low business quality (such as traffic cheating and low service quality), low content quality (such as spam advertisement and false news) and the like, may occur. The quality of the software can be evaluated from various aspects. Such as content quality, quality of service, etc.
After determining the target quality for the target software, the server may extract the behavior feature of the feature dimension corresponding to the target quality from the user behavior data. Different target quality assessment tasks have different corresponding feature dimensions. Target masses include, but are not limited to: quality of service and quality of content. The behavioral characteristics identify user behavioral data for a characteristic dimension corresponding to the target quality. The behavior track with the time sequence direction is formed by a plurality of behavior features extracted aiming at the target quality according to the time sequence relation of the target software responding to the user behavior.
For example, in a task of determining the content quality of the target software, feature dimensions corresponding to the content quality include page text features and page image features. Thus, the server can extract the text of the user browsing the target software and the image of the user browsing the target software from the user behavior data for evaluating the content quality of the software.
If the target quality of the target software is determined to be the service quality, in one possible implementation manner, the feature dimensions corresponding to the service quality include: function type, page location, and duration. The server may extract behavior characteristics from the user behavior data based on the quality of service. The behavior characteristics comprise the function type related to the user behavior, a software interface responding to the user behavior and the duration of the user behavior.
For the application scenario shown in fig. 1, if the server acquires the a user behavior data, the B user behavior data and the C user behavior data sent by the terminal device, the behavior features of feature dimensions (function types and page positions) corresponding to the service quality are extracted from the user behavior data, and the extracted multiple behavior features generate a behavior track with a time sequence relationship as shown in fig. 3. For example, for a B-user, the behavior features extracted by the server from the B-user behavior data include: page 1, customer service, share, page 2, customer service, page 1, and exit. As can be seen from the behavior trace of the B user shown in fig. 3, during the process of using the target software, the B user opens the page 1, uses the customer service function and the sharing function, jumps to the page 2, continues to use the customer service function, and finally exits the target software.
Because the behavior features extracted through the feature dimension can identify the content, the response behavior and the like related to the target quality provided by the target software when responding to the user operation, the target software quality can be evaluated by utilizing the behavior features in the task of determining the target quality of the target software, the processing of user behavior data unrelated to the target quality is reduced, and the efficiency and the accuracy of determining the software quality are improved.
S203: and generating a node diagram corresponding to the target software according to the behavior characteristics.
In practical application, the server may generate a corresponding node map according to a behavior trace with a time sequence direction formed by a plurality of behavior features. As shown in fig. 4, the behavior trace of the a user, the B user, and the C user shown in fig. 3 is converted into a node map having a plurality of nodes and having connection relations between the nodes. The nodes in the node diagram are determined according to the feature dimension corresponding to the target quality. In the node map shown in fig. 4, nodes are determined according to function types and page positions. For example, node x2 represents a customer service function of the user using the target software in page 1, and node x6 represents a customer service function of the user using the target software in page 2.
The connection relation among the nodes is used for reflecting the jump relation among behavior characteristics identified by the nodes. In fig. 4, the connection relationship between nodes is represented by directional line segments with arrows between nodes, which are used to identify the process of a user jumping from one behavior to another during the use of the target software. For example, the directed line segment between node x1 and node x2 represents that the C user jumps to use the customer service function after using the payment function in page 1.
The process of generating the node diagram by using the behaviors can be regarded as a process of fusing the behavior tracks corresponding to all users in the same target software to obtain a directed weighted diagram. In practical application, the connection relation between the nodes can also carry weight information, the size of the weight marks the jump probability between the nodes and is used for representing the influence degree between different nodes, and the weight can be determined by comprehensive behavior characteristics. For example, for the nodes x1, x2 and x3 shown in fig. 4, the behavior feature determines that the behavior of jumping from x1 to x2 occurs 8 times, and the behavior of jumping from x1 to x3 occurs 2 times, so that the weight of jumping from x1 to node x2 can be determined to be 0.8, which indicates that the probability of using the customer service function after the user uses the payment function is 0.8, and the influence degree of node x2 on node x1 is greater; the weight value of the node x3 from the jump of the node x1 is set to be 0.2, which indicates that the probability of using the sharing function after using the payment function by the user is 0.2, and the influence degree of the node x3 on the node x1 is small.
It will be appreciated that the length of time that the user uses the target software identifies the quality of the software from the time dimension. The duration of using the target software by the user is longer, which indicates that the demand level of the user on the software is larger and the target quality of the target software is higher; the duration of the user using the target software is short, which indicates that the user has a small demand on the software and the target quality of the target software is low.
Likewise, in the quality of service determination task of the target software, for a plurality of nodes in the node diagram, in one possible implementation, it may be determined according to the function type and duration.
It will be appreciated that for a target software, the duration of use of the target software by multiple users may be considered to be continuously distributed. In the process of dividing the nodes, discretization processing can be performed on the duration of the user behavior, and the duration in a time period is divided into one node according to a quantization rule.
For example, the duration is divided by 10 minutes as a quantization standard for dividing one node, and then one node is divided every 10 minutes for the duration. If the duration of using the target software by the user a shown in fig. 1 is 5 minutes, the duration of using the target software by the user B is 7 minutes, and the duration of using the target software by the user C is 13 minutes, then the user a and the user B may be classified into a node m1, where the node m1 indicates that the duration of using the target software by the user is less than or equal to 10 minutes, and the node m2 indicates that the duration of using the target software by the user is greater than 10 minutes and less than or equal to 2 minutes.
For the node diagram shown in fig. 4, the multiple nodes are obtained by dividing according to the function type and the page position, and if the node is divided by the feature dimension of increasing the duration, the node in fig. 4 can be further split according to the clustering result of the duration of the user behavior. For example, the node x2 is further split according to the clustering result of the duration of the user behavior to obtain nodes x21 and x22, where x21 represents that the duration of the user using the customer service function in the page 1 is within 10 minutes, and x22 represents that the duration of the user using the customer service function in the page 1 is 10 minutes to 20 minutes.
The nodes can be divided from the time dimension according to the duration of the user behaviors, so that the divided nodes quantify the user behavior data from the time angle, the node diagram structure is enriched, the node diagram is used for evaluating the target quality of the target software, and the accuracy of evaluating the quality of the software can be improved. It will be appreciated that, in practical applications, the nodes in the node map may be divided according to one or more feature dimensions corresponding to the target quality, which is not limited in any way.
S204: and determining a quality parameter of the target software corresponding to the target quality according to the node diagram.
In practical application, the server may determine a quality parameter of the target quality corresponding to the target software according to the node map. Wherein, the size of the quality parameter identifies the degree of the corresponding target quality of the target software. The quality parameter is large, which indicates that the quality of the target software corresponding to the target is high; the quality parameter is small, which indicates that the quality of the target software corresponding to the target is low.
It will be appreciated that in the same target quality determination task, there is a difference in the structure of node diagrams corresponding to different target software. Node diagrams of different structures identify the level of the corresponding target quality of different software. As shown in fig. 5, node diagrams generated using the behavior features corresponding to 4 different software are different for the quality of service determination task. According to the node diagram result corresponding to the software, it can be seen that for the software 1, the switching between pages is biased, and the service quality is higher; for software 2, the preference is to jump to different software, and the service quality is lower; for the software 3, the service quality is higher for the switching between deflection functions; for software 4, the preference is to jump to the same h5, and the service quality is lower.
For different target software with the same or similar node diagram, the quality of the different target software is indicated to be similar. For example, the node map p1 of the target software s1 is similar to the node map p2 corresponding to the target software s2, indicating that the quality of the target software s1 and the quality of the target software s2 are similar. And evaluating the quality of different target software with the same or similar node diagrams, and determining the quality of different target software by utilizing the feature similarity among the different node diagrams, so that the efficiency and the accuracy of the software quality evaluation can be improved.
In practical applications, if the target software s1 and the target software s2 have a jump relationship, and the target software s1 is determined to be low-quality software, that is, the target software s1 is poor software, the target software s2 is highly likely to be low-quality software. Therefore, when determining the quality of the target software s2, the degree of attention to the target software s2 can be increased, for example, the extraction granularity of the user behavior data for the target software s2 is increased, the division granularity of the nodes in the node diagram of the target software s2 is increased, and the like, thereby improving the accuracy of the evaluation of the quality of the target software s 2. According to the software quality determining method provided by the embodiment, when the target quality of the target software needs to be determined, the behavior characteristics of the feature dimension corresponding to the target quality can be extracted according to the user behavior data of the target software, a node diagram is generated based on the behavior characteristics, the nodes in the node diagram can be determined through the feature dimension, and the connection relationship among the nodes represents the jump relationship among the behavior characteristics identified by the nodes. Therefore, the node diagram can quantitatively show the response mode and time sequence relation of the target software to the user behavior in the process of using the target software by the user, so that the quality parameter of the target quality corresponding to the target software can be conveniently and automatically determined through the node diagram, the quality is not required to be identified by means of manpower, and the quality determining efficiency and accuracy are improved.
For the above process of determining the quality parameter of the target quality corresponding to the target software according to the node map, in one possible implementation manner, the feature vector corresponding to the node map may be determined through a graph convolution model, and then the quality parameter of the target quality corresponding to the target software is determined according to the feature vector.
The graph convolution model at least comprises an ith convolution layer and a jth convolution layer and is used for extracting node characteristics and structural characteristics of the node graph. The graph roll-up model may be any of the following models with different network structures: a Graph attention model (Graph Attention Network, GAN), a Graph model combining sampling algorithms and aggregators (Graph SAmple and aggreGatE, graph SAGE), a Semi-supervised Graph convolutional neural network model (Semi-Supervised Classification with Graph Convolutional Networks, semi gcn). In practical application, the structure of the graph convolution model can be set according to the application scene and the application requirement, and the structure is not limited in any way.
In practical application, if the ith convolution layer is close to the input layer, the resolution of the output features of the ith convolution layer is high, including finer node features and structural features; and if the j-th convolution layer is close to the output layer, the j-th convolution layer has low resolution of the output features, including more macroscopic node features and structural features. The feature vector is determined by the output features of the ith layer of convolution and the output features of the jth layer of convolution. The feature vector includes node features with different resolutions and structural features for evaluating software quality.
The graph convolution model as shown in fig. 6 includes 2 layers of convolution layers, with the output features of the 1 st layer of convolution layers being higher in resolution than the output features of the 2 nd layer of convolution layers. The feature vector is determined by the output features of the layer 1 convolutional layer and the output features of the layer 2 convolutional layer.
The feature vector is determined by the output features of the ith layer convolution layer and the output features of the jth layer convolution layer, namely the feature vector comprises node features and structural features with different resolutions, so that the feature vector comprises clear and comprehensive multi-resolution information, and the feature vector is used for determining the target quality of target software, so that the evaluation accuracy of the software quality can be improved.
Since the information amounts of the different node identifications are different, the different nodes in the node map have different information amounts. The information quantity identifies the importance degree of the node in the node diagram, and can be also understood as the influence degree of the node on the quality evaluation of the target software. For the node with less information, the importance degree of the node in the node diagram is lower, and the influence on the quality of the evaluation target software is lower. For the nodes with more information, the importance degree of the nodes in the node diagram is higher, and the influence on the quality of the evaluation target software is higher.
In one possible implementation, an attention mechanism for the amount of information included by the nodes may be set in the graph convolution model, and after the node graph is processed by each layer of the graph convolution model, attention parameters respectively corresponding to the plurality of nodes are updated by the attention mechanism.
Wherein the size of the attention parameter identifies the importance level to which the node corresponds. The attention parameter corresponding to the node is larger, which indicates that the importance degree of the node is higher; the less attention parameter a node corresponds to, indicating that the node is of less importance. In practical application, when the graph convolution model learns node characteristics of the node graph, the graph convolution model can learn according to the attention parameters corresponding to the nodes. For the node with large attention parameters, the model has large feature learning degree and attention degree for the node; for the node with small attention parameters, the model has small feature learning degree and small attention degree for the node.
Because the attention parameter marks the importance degree of the node in the node diagram, the diagram convolution model can learn the characteristics of the node with large importance degree more and learn the characteristics of the node with small importance degree less, so that the efficiency of determining the software quality by using the model is improved, and meanwhile, the evaluation precision of the software quality is improved.
For the above process of obtaining the feature vector by using the output feature, in one possible implementation manner, the output feature of the ith layer of convolution layer may be subjected to pooling processing to obtain a first pooling result, the output feature of the jth layer of convolution layer is subjected to pooling processing to obtain a second pooling result, and then the feature vector corresponding to the node map is determined according to the first pooling result and the second pooling result.
Pooling (Pooling) is an important concept in convolutional neural networks, which is actually a form of downsampling. There are many different forms of non-linear Pooling functions, such as Max Pooling, average Pooling. The pooling layer can continuously reduce the dimension of the output characteristics of the convolution layer, so that the quantity and the calculated quantity of parameters can be reduced in the process of determining the software quality by the graph convolution model, and the overfitting problem is prevented to a certain extent.
In one possible implementation, the nodes in the node map whose attention parameters meet the preset conditions may be pooled.
In practical application, a threshold may be preset, and the attention parameter corresponding to the node is compared with the threshold. If the attention parameter is larger than the threshold value, the information amount included in the node is larger, the quality evaluation of the target software is less affected by the node, and the node can be reserved and used for subsequent pooling processing. If the attention parameter is smaller than the threshold value, the information quantity included in the node is smaller, the quality evaluation of the node on the target software is greatly influenced, and the node characteristics of the node are not considered in the subsequent pooling process. Wherein, the Pooling treatment can adopt Sum Pooling (Sum Pooling). That is, the characteristics of the nodes with the attention parameter larger than the threshold value are integrated together for evaluating the software quality, so that a pooling mechanism of topk is realized. Wherein, when k is 1, it is the maximum pooling; when k is the number of all nodes in the node map, it is global pooled.
Aiming at the training process of the graph convolution model, a node diagram with a label can be utilized, and a gradient return method is adopted to perform supervised training on a pre-established initial graph convolution model to obtain the graph convolution model for determining the software quality. Wherein the tag user identification node map corresponds to the target quality of the target software.
It will be appreciated that training samples for supervised training are typically manually labeled, and are inefficient. In order to improve the training efficiency of the model, in one possible implementation manner, the initial convolution model may be pre-trained in a semi-supervised manner, and then the initial convolution model after the pre-training is trained in a supervised manner, so as to obtain the convolution model.
Because the training samples for semi-supervised training comprise the training samples with the labels and the training samples without the labels, the requirement on the training samples with the labels is reduced, so that the manual marking work can be reduced by training the graph convolution model in a semi-supervised mode, and the training efficiency of the model is improved.
Referring to fig. 7, fig. 7 is a schematic flow chart of another software quality determining method according to an embodiment of the present application. For ease of understanding, the description will be given taking the server as an execution subject for determining the quality of service of the target software. The server is provided with a trained graph convolution model.
For the target software, the server may collect user behavior data corresponding to all users within one month (S701), and then extract behavior features of feature dimensions (function type, page position and duration) corresponding to the service quality from the user behavior data. The server may divide the nodes according to the function type, the page position, and the duration, discretize the nodes, and generate a node map having a connection relationship according to a time sequence relationship between behavior features (S702). Then, the server may predict the target software quality of service using the node map as an input to the graph convolution model (S703). The graph convolution model may be implemented by a graph neural network framework (Deep Graph Library, DGL). Finally, the quality parameters of the service quality corresponding to the target software are obtained through the output of the graph convolution model (S704).
For the software quality determining method described above, the embodiment of the application also provides a corresponding software quality determining device.
Referring to fig. 8, fig. 8 is a schematic structural diagram of a software quality determining apparatus according to an embodiment of the present application. As shown in fig. 8, the software quality determination apparatus 800 includes an acquisition unit 801, an extraction unit 802, a generation unit 803, and a determination unit 804:
The acquiring unit 801 is configured to acquire user behavior data of the target software;
the extracting unit 802 is configured to extract, according to a target quality to be determined, a behavior feature of a feature dimension corresponding to the target quality from the user behavior data;
the generating unit 803 is configured to generate a node map corresponding to the target software according to the behavior feature; the node diagram comprises a plurality of nodes with connection relations, the nodes are determined according to the feature dimensions, and the connection relations among the nodes are used for reflecting jump relations among behavior features identified by the nodes;
the determining unit 804 is configured to determine a quality parameter of the target software corresponding to the target quality according to the node map.
Wherein the target quality is a service quality, the feature dimensions corresponding to the service quality include a function type, a page position, and a duration, and the extracting unit 802 is configured to:
and extracting behavior characteristics from the user behavior data according to the service quality, wherein the behavior characteristics comprise the function types related to the user behavior, a software interface responding to the user behavior and the duration of the user behavior.
Wherein the plurality of nodes are determined based on the function type and duration.
Wherein the determining unit 804 is configured to:
determining a feature vector corresponding to the node map through a graph convolution model; the graph convolution model at least comprises an ith convolution layer and a jth convolution layer, and the feature vector is determined through the output features of the ith convolution layer and the output features of the jth convolution layer;
and determining a quality parameter of the target software corresponding to the target quality according to the feature vector.
The graph rolling model is provided with an attention mechanism aiming at the information quantity included by the nodes, after the node graph is processed through each layer of the graph rolling model, attention parameters corresponding to the nodes are updated through the attention mechanism, and the attention parameters are used for identifying the importance degree of the corresponding nodes.
Wherein the determining unit 804 is configured to:
pooling the output characteristics of the ith convolution layer to obtain a first pooling result, and pooling the output characteristics of the jth convolution layer to obtain a second pooling result;
and determining the feature vector corresponding to the node map through the first pooling result and the second pooling result.
The nodes involved in the pooling process are nodes in the node diagram, wherein the attention parameters of the nodes meet preset conditions.
The graph convolution model is obtained through training in the following mode:
pre-training the initial graph rolling model in a semi-supervision mode;
training the initial graph rolling model after the pre-training in a supervision mode to obtain the graph rolling model.
Wherein the user data includes user behavior data generated by the target software during a history period.
According to the software quality determining device provided by the embodiment, when the target quality of the target software needs to be determined, the behavior characteristics of the feature dimension corresponding to the target quality can be extracted according to the user behavior data of the target software, a node diagram is generated based on the behavior characteristics, the nodes in the node diagram can be determined through the feature dimension, and the connection relationship among the nodes represents the jump relationship among the behavior characteristics identified by the nodes. Therefore, the node diagram can quantitatively show the response mode and time sequence relation of the target software to the user behavior in the process of using the target software by the user, so that the quality parameter of the target quality corresponding to the target software can be conveniently and automatically determined through the node diagram, the quality is not required to be identified by means of manpower, and the quality determining efficiency and accuracy are improved.
The embodiment of the application also provides a server and terminal equipment for software quality determination, and the server and terminal equipment for software quality determination provided by the embodiment of the application are described below from the perspective of hardware materialization.
Referring to fig. 9, fig. 9 is a schematic diagram of a server structure provided in an embodiment of the present application, where the server 1400 may vary considerably in configuration or performance, and may include one or more central processing units (central processing units, CPU) 1422 (e.g., one or more processors) and memory 1432, one or more storage media 1430 (e.g., one or more mass storage devices) that store applications 1442 or data 1444. Wherein the memory 1432 and storage medium 1430 can be transitory or persistent storage. The program stored in the storage medium 1430 may include one or more modules (not shown), each of which may include a series of instruction operations on a server. Further, the central processor 1422 may be provided in communication with a storage medium 1430 to perform a series of instruction operations in the storage medium 1430 on the server 1400.
The server 1400 may also include one or more power supplies 1426, one or more wired or wireless network interfaces 1450, one or more input/output interfaces 1458, and/or one or more operating systems 1441, such as Windows Server, mac OS XTM, unixTM, linuxTM, freeBSDTM, and the like.
The steps performed by the server in the above embodiments may be based on the server structure shown in fig. 9.
Wherein, the CPU 1422 is configured to perform the following steps:
acquiring user behavior data of target software;
according to the target quality to be determined, extracting behavior characteristics of characteristic dimensions corresponding to the target quality from the user behavior data;
generating a node diagram corresponding to the target software according to the behavior characteristics; the node diagram comprises a plurality of nodes with connection relations, the nodes are determined according to the feature dimensions, and the connection relations among the nodes are used for reflecting jump relations among behavior features identified by the nodes;
and determining a quality parameter of the target software corresponding to the target quality according to the node diagram.
Optionally, the CPU 1422 may further perform method steps of any specific implementation of the software quality determination method in the embodiments of the present application.
For the software quality determining method described above, the embodiment of the application also provides a terminal device for determining the software quality, so that the method for determining the software quality is realized and applied in practice.
Referring to fig. 10, fig. 10 is a schematic structural diagram of a terminal device according to an embodiment of the present application. For convenience of explanation, only those portions relevant to the embodiments of the present application are shown, and specific technical details are not disclosed, refer to the method portions of the embodiments of the present application. The terminal device may be any terminal device including a tablet computer, a personal digital assistant (english full name: personal Digital Assistant, english abbreviation: PDA), and the like:
fig. 10 is a block diagram showing a part of the structure related to a terminal provided in an embodiment of the present application. Referring to fig. 10, the terminal includes: radio Frequency (r.f. Frequency) circuitry 1510, memory 1520, input unit 1530, display unit 1540, sensor 1550, audio circuitry 1560, wireless fidelity (r.f. wireless fidelity, wiFi) module 1570, processor 1580, and power supply 1590. Those skilled in the art will appreciate that the tablet configuration shown in fig. 10 is not limiting of the tablet and may include more or fewer components than shown, or certain components may be combined, or a different arrangement of components.
The following describes the components of the tablet pc in detail with reference to fig. 10:
the memory 1520 may be used to store software programs and modules, and the processor 1580 implements various functional applications and data processing of the terminal by executing the software programs and modules stored in the memory 1520. The memory 1520 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, phonebook, etc.) created according to the use of the handset, etc. In addition, memory 1520 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.
The processor 1580 is a control center of the terminal, connects various parts of the entire tablet computer using various interfaces and lines, and performs various functions and processes data of the tablet computer by running or executing software programs and/or modules stored in the memory 1520 and calling data stored in the memory 1520, thereby performing overall monitoring of the tablet computer. In the alternative, processor 1580 may include one or more processing units; preferably, the processor 1580 can integrate an application processor and a modem processor, wherein the application processor primarily processes operating systems, user interfaces, application programs, and the like, and the modem processor primarily processes wireless communications. It is to be appreciated that the modem processor described above may not be integrated into the processor 1580.
In an embodiment of the present application, the memory 1520 included in the terminal may store program codes and transmit the program codes to the processor.
The processor 1580 included in the terminal may perform the software quality determining method provided in the above embodiment according to instructions in the program code.
The present application also provides a computer readable storage medium storing a computer program for executing the software quality determining method provided in the above embodiments.
Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the above method embodiments may be implemented by hardware related to program instructions, where the above program may be stored in a computer readable storage medium, and when the program is executed, the program performs steps including the above method embodiments; and the aforementioned storage medium may be at least one of the following media: read-only memory (ROM), RAM, magnetic disk or optical disk, etc., which can store program codes.
It should be noted that, in the present specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment is mainly described in a different point from other embodiments. In particular, for the apparatus and system embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, with reference to the description of the method embodiments in part. The apparatus and system embodiments described above are merely illustrative, in which elements illustrated as separate elements may or may not be physically separate, and elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
The foregoing is merely one specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (15)
1. A method of software quality determination, the method comprising:
acquiring user behavior data of target software;
according to target quality to be determined, extracting behavior characteristics of characteristic dimensions corresponding to the target quality from the user behavior data, wherein the target quality comprises at least one of service quality and content quality, the characteristic dimensions corresponding to the service quality comprise function types, page positions and duration, and the characteristic dimensions corresponding to the content quality comprise page text characteristics and page image characteristics;
generating a node diagram corresponding to the target software according to the behavior characteristics; the node diagram comprises a plurality of nodes with connection relations, the nodes are determined according to the feature dimensions, and the connection relations among the nodes are used for reflecting jump relations among behavior features identified by the nodes;
And determining a quality parameter of the target software corresponding to the target quality according to the node diagram.
2. The method of claim 1, wherein when the target quality is a quality of service, the extracting the behavior feature of the feature dimension corresponding to the target quality from the user behavior data comprises:
and extracting behavior characteristics from the user behavior data according to the service quality, wherein the behavior characteristics comprise the function types related to the user behavior, a software interface responding to the user behavior and the duration of the user behavior.
3. The method of claim 2, wherein the plurality of nodes are determined based on the function type and duration.
4. A method according to any one of claims 1-3, wherein said determining a quality parameter of said target software corresponding to said target quality from said node map comprises:
determining a feature vector corresponding to the node map through a graph convolution model; the graph convolution model at least comprises an ith convolution layer and a jth convolution layer, and the feature vector is determined through the output features of the ith convolution layer and the output features of the jth convolution layer;
And determining a quality parameter of the target software corresponding to the target quality according to the feature vector.
5. The method according to claim 4, wherein the graph rolling model is provided with an attention mechanism for the amount of information included in the nodes, and after the node graph is processed by each layer of the graph rolling model, attention parameters respectively corresponding to the plurality of nodes are updated by the attention mechanism, wherein the attention parameters are used for identifying importance degrees of the corresponding nodes.
6. The method according to claim 5, wherein determining, from the node map, a feature vector corresponding to the node map by a graph convolution model includes:
pooling the output characteristics of the ith convolution layer to obtain a first pooling result, and pooling the output characteristics of the jth convolution layer to obtain a second pooling result;
and determining the feature vector corresponding to the node map through the first pooling result and the second pooling result.
7. The method of claim 6, wherein the nodes involved in the pooling process are nodes in the node map for which an attention parameter satisfies a preset condition.
8. The method of claim 4, wherein the graph roll-up model is trained by:
pre-training the initial graph rolling model in a semi-supervision mode;
training the initial graph rolling model after the pre-training in a supervision mode to obtain the graph rolling model.
9. The method of claim 1, wherein the user behavior data comprises user behavior data generated by the target software over a historical period of time.
10. A software quality determination device, characterized in that the device comprises an acquisition unit, an extraction unit, a generation unit and a determination unit:
the acquisition unit is used for acquiring user behavior data of the target software;
the extraction unit is used for extracting behavior characteristics of characteristic dimensions corresponding to target quality from the user behavior data according to the target quality to be determined, wherein the target quality comprises at least one of service quality and content quality, the characteristic dimensions corresponding to the service quality comprise function types, page positions and duration, and the characteristic dimensions corresponding to the content quality comprise page text characteristics and page image characteristics;
The generating unit is used for generating a node diagram corresponding to the target software according to the behavior characteristics; the node diagram comprises a plurality of nodes with connection relations, the nodes are determined according to the feature dimensions, and the connection relations among the nodes are used for reflecting jump relations among behavior features identified by the nodes;
and the determining unit is used for determining a quality parameter of the target software corresponding to the target quality according to the node diagram.
11. The apparatus of claim 10, wherein the target quality is a quality of service, and the feature dimensions corresponding to the quality of service include a function type, a page position, and a duration, and the extracting unit is configured to:
and extracting behavior characteristics from the user behavior data according to the service quality, wherein the behavior characteristics comprise the function types related to the user behavior, a software interface responding to the user behavior and the duration of the user behavior.
12. The apparatus of claim 11, wherein the plurality of nodes are determined based on the function type and duration.
13. The apparatus according to any of the claims 10-12, wherein the determining unit is configured to:
Determining a feature vector corresponding to the node map through a graph convolution model; the graph convolution model at least comprises an ith convolution layer and a jth convolution layer, and the feature vector is determined through the output features of the ith convolution layer and the output features of the jth convolution layer;
and determining a quality parameter of the target software corresponding to the target quality according to the feature vector.
14. A software quality determination device, the device comprising a processor and a memory:
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is configured to perform the method of any of claims 1-9 according to instructions in the program code.
15. A computer readable storage medium, characterized in that the computer readable storage medium is for storing a computer program for executing the method of any one of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010316245.0A CN111506510B (en) | 2020-04-21 | 2020-04-21 | Software quality determining method and related device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010316245.0A CN111506510B (en) | 2020-04-21 | 2020-04-21 | Software quality determining method and related device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111506510A CN111506510A (en) | 2020-08-07 |
| CN111506510B true CN111506510B (en) | 2023-08-08 |
Family
ID=71864872
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010316245.0A Active CN111506510B (en) | 2020-04-21 | 2020-04-21 | Software quality determining method and related device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111506510B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109117380A (en) * | 2018-09-28 | 2019-01-01 | 中国科学院长春光学精密机械与物理研究所 | A kind of method for evaluating software quality, device, equipment and readable storage medium storing program for executing |
| CN110929882A (en) * | 2019-11-21 | 2020-03-27 | 腾讯科技（深圳）有限公司 | Feature vector calculation method based on artificial intelligence and related device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109214238B (en) * | 2017-06-30 | 2022-06-28 | 阿波罗智能技术(北京)有限公司 | Multi-target tracking method, device, equipment and storage medium |
-
2020
- 2020-04-21 CN CN202010316245.0A patent/CN111506510B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109117380A (en) * | 2018-09-28 | 2019-01-01 | 中国科学院长春光学精密机械与物理研究所 | A kind of method for evaluating software quality, device, equipment and readable storage medium storing program for executing |
| CN110929882A (en) * | 2019-11-21 | 2020-03-27 | 腾讯科技（深圳）有限公司 | Feature vector calculation method based on artificial intelligence and related device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111506510A (en) | 2020-08-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113518250B (en) | Multimedia data processing method, device, equipment and readable storage medium | |
| CN112633962B (en) | Service recommendation method and device, computer equipment and storage medium | |
| CN109388674B (en) | Data processing method, device, equipment and readable storage medium | |
| CN110598070B (en) | Application type identification method and device, server and storage medium | |
| CN112085541A (en) | User demand analysis method and device based on browsing consumption time series data | |
| US20190156231A1 (en) | User segmentation using predictive model interpretation | |
| CN110929806A (en) | Picture processing method and device based on artificial intelligence and electronic equipment | |
| CN113610156A (en) | Artificial intelligence model machine learning method and server for big data analysis | |
| CN116308551A (en) | Content recommendation method and system based on digital financial AI platform | |
| CN116720489A (en) | Page filling method and device, electronic equipment and computer readable storage medium | |
| CN111506510B (en) | Software quality determining method and related device | |
| CN108764489B (en) | Model training method and device based on virtual sample | |
| CN112801156B (en) | Business big data acquisition method and server for artificial intelligence machine learning | |
| CN113255231B (en) | Data processing method, device, equipment and storage medium | |
| CN114580533A (en) | Method, apparatus, device, medium, and program product for training feature extraction model | |
| US20220253690A1 (en) | Machine-learning systems for simulating collaborative behavior by interacting users within a group | |
| CN112364649B (en) | Named entity identification method and device, computer equipment and storage medium | |
| CN116502173A (en) | State identification method, device, storage medium and equipment | |
| CN115482019A (en) | Activity attention prediction method and device, electronic equipment and storage medium | |
| CN111522747A (en) | Application processing method, device, equipment and medium | |
| CN115700550A (en) | Label classification model training and object screening method, device and storage medium | |
| CN115329183A (en) | Data processing method, device, storage medium and equipment | |
| CN113569041A (en) | Text detection method and device, computer equipment and readable storage medium | |
| CN114417944B (en) | Recognition model training method and device, and user abnormal behavior recognition method and device | |
| CN111552827A (en) | Labeling method and device, and behavior willingness prediction model training method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40029273 Country of ref document: HK |
|
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |