CN111475396A

CN111475396A - Program detection method, device, electronic equipment and storage medium

Info

Publication number: CN111475396A
Application number: CN201910071945.5A
Authority: CN
Inventors: 章磊; 余岸轩; 何磊
Original assignee: Beijing Didi Infinity Technology and Development Co Ltd
Current assignee: Beijing Didi Infinity Technology and Development Co Ltd
Priority date: 2019-01-24
Filing date: 2019-01-24
Publication date: 2020-07-31

Abstract

The embodiment of the application provides a program detection method and device, electronic equipment and a storage medium, and relates to the technical field of data processing. The method comprises the following steps: in the process of testing a client program through simulation operation, a client interactive interface generated by the operation of the client program is obtained; determining whether the client interactive interface is matched with a client preset interactive interface of a client program; if not, determining that the client program is in an abnormal state. Compared with manual testing in the prior art, the testing device realizes high-efficiency and low-cost testing. And because the testing time is free, the electronic equipment can be tested before or after the client program is online, so that various size problems of the client program can be found in time through testing.

Description

Program detection method, device, electronic equipment and storage medium

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a program detection method and apparatus, an electronic device, and a storage medium.

Background

In the development of the client program, the developed client program needs to be tested by operating various functions on the client by a tester, and the client program can be on line after the test is passed, so that a user can use the client program normally after the client program is on line.

However, the time cost and the labor cost required by the large-scale test of the tester are high, and once the client program is on line, the small function of the client program is changed, and in consideration of the cost, the small change is difficult to be tested on a large scale by the tester, so that the problem existing in the small change is difficult to be found.

Disclosure of Invention

The present application provides a program detecting method, a program detecting apparatus, an electronic device, and a storage medium, so as to implement efficient and low-cost testing to find various size problems of a client program.

In a first aspect, an embodiment of the present application provides a program detection method, where the method includes: in the process of testing a client program through simulation operation, a client interactive interface generated by the operation of the client program is obtained; determining whether the client interactive interface is matched with a client preset interactive interface of the client program; if not, determining that the client program is in an abnormal state.

In the embodiment of the application, the electronic device can control the client program to operate in a simulation mode at any time, so that whether the client program is abnormal or not can be tested on the basis of matching the client interactive interface generated by the operation of the client program with the preset client interactive interface. Compared with manual testing in the prior art, the testing device realizes high-efficiency and low-cost testing. And because the testing time is free, the electronic equipment can be tested before or after the client program is online, so that various size problems of the client program can be found in time through testing.

With reference to the first aspect, in some possible implementation manners, determining whether the client-side interactive interface matches a client-side preset interactive interface of the client-side program includes: determining the matching similarity between the client-side interactive interface and a client-side preset interactive interface of the client-side program; and judging whether the matching similarity is smaller than a similarity threshold, wherein when the matching similarity is smaller than the similarity threshold, the client interactive interface is not matched with the client preset interactive interface.

In the embodiment of the application, since the electronic device can calculate the matching similarity between the client-side interactive interface and the client-side preset interactive interface, whether the client-side interactive interface is the same as the client-side preset interactive interface can be accurately reflected through the matching similarity, so that the accuracy of the test result is ensured.

With reference to the first aspect, in some possible implementation manners, determining matching similarity between a client interaction interface and a client preset interaction interface of the client program includes: performing image recognition on the client-side interactive interface to obtain at least one sub-interface containing at least one object in the client-side interactive interface, and performing image recognition on a client-side preset interactive interface of the client-side program to obtain at least one preset sub-interface containing at least one preset object in the client-side preset interactive interface, wherein each sub-interface in the at least one sub-interface contains one object, and each preset sub-interface in the at least one preset sub-interface contains one preset object; performing image recognition on the at least one sub-interface and the at least one preset sub-interface, and determining the highest similarity of each object in the at least one object, wherein the highest similarity of each object is the similarity between each object and the most similar preset object in the at least one preset object, and the matching similarity comprises the highest similarity of each object; correspondingly, judging whether the matching similarity is smaller than a similarity threshold value includes: and judging whether the highest similarity of any object in the at least one object is smaller than a similarity threshold, wherein when the highest similarity of any object is smaller than the similarity threshold, the client interactive interface is not matched with the preset client interactive interface.

In the embodiment of the application, because the electronic device can extract each object in the client-side interactive interface and judge whether each object in the client-side interactive interface is the same as one preset object in the client-side preset interactive interface or not based on the highest similarity of each object, the electronic device can refine the test result to test which object in the client-side interactive interface has a problem, and the efficiency of finding a specific problem based on the test is greatly improved.

With reference to the first aspect, in some possible implementation manners, performing image recognition on the at least one sub-interface and the at least one preset sub-interface, and determining a highest similarity of each object in the at least one object includes: performing image recognition on the at least one sub-interface to determine a pixel probability matrix of each sub-interface in the at least one sub-interface, and performing image recognition on the at least one preset sub-interface to determine at least one preset pixel probability matrix of the at least one preset sub-interface; wherein the pixel probability matrix represents whether each pixel point in the sub-interface belongs to an object in the sub-interface or a background in the sub-interface; matching each pixel probability matrix with the at least one preset pixel probability matrix respectively, and determining at least one matching similarity of the objects in each sub-interface; the highest similarity of each object is determined from the at least one matching similarity of each object.

In the embodiment of the application, the electronic device determines the highest similarity of the object by adopting a mode of matching each pixel point based on obtaining the pixel probability matrix, so that the determined highest similarity of the object is more accurate, and the accuracy of the test result is improved.

With reference to the first aspect, in some possible implementations, after determining that the client program is in an abnormal state, the method further includes: and generating prompt information for indicating the programs in the abnormal state in the client programs, and transmitting the prompt information to the terminal.

In the embodiment of the application, after the abnormality is found, the user can be timely reminded of which programs are in the abnormal state by transmitting the prompt information to the terminal, so that the programs in the abnormal state can be quickly known by the user, and the user can timely adjust the programs in the abnormal state.

With reference to the first aspect, in some possible implementations, before obtaining the client interaction interface generated by the client program, the method further includes: judging whether the client program runs to generate the client interactive interface or not in the process of testing the client program through simulated running; if yes, executing the following steps: acquiring a client interactive interface generated by the running of the client program; if not, determining that the client program is in an abnormal state.

In the embodiment of the application, the electronic device judges whether the client program generates the client interactive interface or not, if the client interactive interface cannot be generated, the client program can be shown to be in an abnormal state, so that the client program does not need to be tested based on image recognition subsequently, the testing efficiency is improved, and redundant testing is avoided.

With reference to the first aspect, in some possible implementations, before obtaining the client interaction interface generated by the client program, the method further includes: detecting the client program and judging whether the client program is updated or not; if yes, simulating the operation of the user on the client-side interactive interface, enabling the client-side program to respond to the operation and operate, and executing the following steps: and in the process of testing the client program through simulation operation, obtaining a client interactive interface generated by the operation of the client program.

In the embodiment of the application, because the update of the client program is an important factor causing a problem to the client program, the electronic device tests the client program only when determining that the client program is updated, so that the problem of the client program can be tested in a more efficient manner, and redundant testing is avoided.

In a second aspect, an embodiment of the present application provides a program detection apparatus, including:

and the interface obtaining module is used for obtaining a client interactive interface generated by the running of the client program in the process of testing the client program through simulated running.

And the interface matching module is used for determining whether the client interactive interface is matched with a client preset interactive interface of the client program.

And the state determining module is used for determining that the client program is in an abnormal state if the client program is not in the abnormal state.

In combination with the second aspect, in some possible implementations,

the interface matching module is also used for determining the matching similarity between the client interactive interface and a client preset interactive interface of the client program; and judging whether the matching similarity is smaller than a similarity threshold, wherein when the matching similarity is smaller than the similarity threshold, the client interactive interface is not matched with the client preset interactive interface.

In combination with the second aspect, in some possible implementations,

the interface matching module is further configured to perform image recognition on the client-side interactive interface to obtain at least one sub-interface including at least one object in the client-side interactive interface, and perform image recognition on a client-side preset interactive interface of the client-side program to obtain at least one preset sub-interface including at least one preset object in the client-side preset interactive interface, where each sub-interface in the at least one sub-interface includes one object, and each preset sub-interface in the at least one preset sub-interface includes one preset object; and performing image recognition on the at least one sub-interface and the at least one preset sub-interface, and determining the highest similarity of each object in the at least one object, wherein the highest similarity of each object is the similarity between each object and the most similar preset object in the at least one preset object, and the matching similarity comprises the highest similarity of each object.

Correspondingly, the interface matching module is further configured to determine whether the highest similarity of any object in the at least one object is smaller than a similarity threshold, where when the highest similarity of any object is smaller than the similarity threshold, it indicates that the client-side interactive interface is not matched with the client-side preset interactive interface.

In combination with the second aspect, in some possible implementations,

the interface matching module is further configured to perform image recognition on the at least one sub-interface, determine a pixel probability matrix of each sub-interface in the at least one sub-interface, perform image recognition on the at least one preset sub-interface, and determine at least one preset pixel probability matrix of the at least one preset sub-interface; wherein the pixel probability matrix represents whether each pixel point in the sub-interface belongs to an object in the sub-interface or a background in the sub-interface; matching each pixel probability matrix with the at least one preset pixel probability matrix respectively, and determining at least one matching similarity of the objects in each sub-interface; the highest similarity of each object is determined from the at least one matching similarity of each object.

With reference to the second aspect, in some possible implementations, the apparatus further includes:

and the information prompt module is used for generating prompt information for showing the programs in the abnormal state in the client programs and transmitting the prompt information to the terminal.

the interface judgment module is used for judging whether the client-side interactive interface is generated by the operation of the client-side program or not in the process of testing the client-side program through simulated operation; if yes, executing the following steps: acquiring a client interactive interface generated by the running of the client program; if not, determining that the client program is in an abnormal state.

the update judging module is used for detecting the client program and judging whether the client program is updated; if yes, simulating the operation of the user on the client-side interactive interface, enabling the client-side program to respond to the operation and operate, and executing the following steps: and in the process of testing the client program through simulation operation, obtaining a client interactive interface generated by the operation of the client program.

In a third aspect, an embodiment of the present application provides an electronic device, where the electronic device includes: a processor, a memory, a communication bus, and a network port; the processor, the network port and the memory are connected by the communication bus.

The memory is used for storing programs.

The processor is configured to execute the program detection method according to the first aspect and any implementation manner of the first aspect by calling a program stored in the memory.

In a fourth aspect, the present application provides a computer-readable storage medium having a computer-executable non-volatile program code, where the program code causes the computer to execute the program detection method according to the first aspect and any implementation manner of the first aspect.

In order to make the aforementioned objects, features and advantages of the present application more comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.

Fig. 1 shows a block diagram of an electronic device according to an embodiment of the present application;

FIG. 2 is a flow chart illustrating a program detection method provided by an embodiment of the present application;

fig. 3A is a schematic view illustrating a first application scenario of a program detection method according to an embodiment of the present application;

fig. 3B is a schematic diagram illustrating a second application scenario of a program detection method according to an embodiment of the present application;

fig. 4A is a schematic diagram illustrating a third application scenario of a program detection method provided in an embodiment of the present application;

fig. 4B is a schematic diagram illustrating a fourth application scenario of a program detection method provided in an embodiment of the present application;

fig. 5A is a schematic diagram illustrating a fifth application scenario of a program detection method provided in an embodiment of the present application;

fig. 5B is a schematic diagram illustrating a sixth application scenario of a program detection method provided in an embodiment of the present application;

fig. 6 shows a block diagram of a program detecting device according to an embodiment of the present application.

Detailed Description

In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are for illustrative and descriptive purposes only and are not used to limit the scope of protection of the present application. Additionally, it should be understood that the schematic drawings are not necessarily drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of the present application. It should be understood that the operations of the flow diagrams may be performed out of order, and steps without logical context may be performed in reverse order or simultaneously. One skilled in the art, under the guidance of this application, may add one or more other operations to, or remove one or more operations from, the flowchart.

In addition, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.

To enable those skilled in the art to use the present disclosure, the following embodiments are given in conjunction with a specific application scenario for processing a service order. It will be apparent to those skilled in the art that the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the application. Although the present application is primarily described in the context of processing a service order, it should be understood that this is merely one exemplary embodiment. The method and the device can be applied to testing the client on the service request side or the service providing side. The service providing terminal can provide transportation service for the service request terminal based on the client on the service providing terminal, otherwise, the service request terminal can request to provide transportation service for the service providing terminal based on the client on the service request terminal; the manner of transportation service may include, for example, land, sea, or air, etc., or any combination thereof, while the vehicle providing the transportation service may include, for example, a taxi, a private car, a windmill, a bus, a train, a bullet train, a high speed railway, a subway, a ship, an airplane, a spacecraft, a hot air balloon, or an unmanned vehicle, etc., or any combination thereof. Of course, the present application may also include any client for enabling other services, such as a client for sending and/or receiving couriers, a client for business to and from parties. Applications of the method or apparatus of the present application may include web pages, plug-ins for browsers, client terminals, customization systems, internal analysis systems, or artificial intelligence robots, etc., or any combination thereof.

It should be noted that in the embodiments of the present application, the term "comprising" is used to indicate the presence of the features stated hereinafter, but does not exclude the addition of further features.

Referring to fig. 1, some embodiments of the present application provide an electronic device 10, where the electronic device 10 may be a server or a terminal.

The server may be a single server or a server group. The set of servers can be centralized or distributed (e.g., the servers can be a distributed system). While in some embodiments, the server may be implemented on a cloud platform; by way of example only, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud (communicuted), a distributed cloud, an inter-cloud, a multi-cloud, and the like, or any combination thereof.

The terminal may be a vehicle-mounted computer, a Personal Computer (PC), a tablet computer, a smart phone, a Personal Digital Assistant (PDA), or the like.

In this embodiment, the electronic device 10 may be used to implement the program detection method of the present application. Although only one device is shown, the functionality described herein may be implemented in a distributed manner across a plurality of similar devices to balance processing load for convenience.

For example, the electronic device 10 may include a network port 11 connected to a network, one or more processors 12 for executing program instructions, a communication bus 13, and different forms of memory 14, such as a disk, ROM, or RAM, or any combination thereof. Illustratively, the computer platform may also include program instructions stored in ROM, RAM, or other types of non-transitory storage media, or any combination thereof. The method of the present application may be implemented in accordance with these program instructions. The electronic device 10 also includes an Input/Output (I/O) interface 15 between the computer and other Input/Output devices (e.g., keyboard, display screen).

For ease of illustration, only one processor is depicted in the electronic device 10. It should be noted, however, that the electronic device 10 in the present application may also include multiple processors, and thus the steps performed by one processor described in the present application may also be performed by multiple processors in combination or separately. For example, if the processor of the electronic device 10 executes steps a and B, it should be understood that steps a and B may also be executed by two different processors together or separately in one processor. For example, a first processor performs step a and a second processor performs step B, or the first processor and the second processor perform steps a and B together.

Referring to fig. 2, some embodiments of the present application provide a program detection method, which may be executed by an electronic device, and the program detection method includes: step S100, step S200, and step S300.

Step S100: and in the process of testing the client program through simulation operation, obtaining a client interactive interface generated by the operation of the client program.

Step S200: and determining whether the client interactive interface is matched with a client preset interactive interface of the client program.

Step S300: if not, determining that the client program is in an abnormal state.

The flow of the program detecting method of the present application will be described in detail below with reference to fig. 2 and 5.

The client program may be a program required by the client to run, and the running of the client program may implement various functions of the client, for example, the running of the client program may implement the client to implement reservation, delivery, payment, and the like of the network contract vehicle order through interaction with the server or interaction with other clients through the server, or any combination thereof.

The client program may include a front-end program of the client and a back-end program of the client, and the devices that store and operate the front-end program of the client and the back-end program of the client are different at different time points. Generally, if the client program is in a testing stage after development or update, the client program is not generally pushed online for each user to download, and at this time, the client program may be stored on the server in the background as a whole, that is, both the front-end program of the client and the back-end program of the client may be stored on the server in the background. However, if the client program is pushed to the online for each user to download through the developed or updated test, the front-end program of the client can be downloaded to the terminal of the user, and the back-end program of the client can also be stored in the server, so that when the front-end program of the client is operated by the terminal of the user, the terminal can interact with the server, and the back-end program of the client stored in the server is operated in a matching manner. For example, the front-end program of the client may be a request program for loading an order payment page, and the front-end program of the client may be a program for pushing the order payment page, and then the execution of the front-end program of the client causes the terminal to send a page loading request to the server, so that the server executes the back-end program of the client based on sending the page loading request, thereby pushing the order payment page to the terminal, and causing the terminal to load the order payment page.

In this embodiment, if the client program is in a test stage after development or update, the program detection method may be executed by a server storing the client program, that is, the electronic device may be a server; however, if the client program has been pushed online by a developed or updated test, the program detection method may be executed not only by the server storing the client program, but also by a terminal downloading the front-end program of the client, that is, the electronic device may also be a terminal.

Of course, the electronic device, whether it is a server or a terminal, may execute the program detection method to test whether the client program is normal or not.

In this embodiment, before testing the client program, the electronic device may determine whether it is necessary to test the client program at present. That is, the client program is generally relatively prone to a problem after being updated, the electronic device may determine whether the client program can be detected based on determining whether the client program has been updated.

As an optional way to determine whether the client program is updated, the electronic device may intermittently detect whether the version number of the client program is updated at certain time intervals, so as to determine whether the client program is updated by detecting whether the version number of the client program is updated.

If the electronic device detects that the version number of the client program is updated, the electronic device can determine that the client program is updated, so that the electronic device can determine to test the client program.

If the electronic device detects that the version number of the client program is not updated, the electronic device may determine that the client program is not updated, and wait until the next time point to continue detecting the version number of the client program.

It can be understood that detecting the version number of the client program is an implementation manner of this embodiment, and should not be limited to this embodiment, and other implementation manners may also be adopted in this embodiment, for example: and comparing the client program acquired each time with the last client program, and determining the update of the client program when the difference of the programs is obtained by comparison.

It can also be understood that, if the client program is a program developed for the first time, the electronic device may determine that the client program is the program developed for the first time by detecting that the version number of the client program is a new version number, or by confirming that the client program is a new program through program comparison, and determine that the client program can be detected.

Then, after determining that the client program can be detected, the electronic device may perform step S100.

Step S100: and in the process of testing the client program, obtaining a client interactive interface generated by the running of the client program.

The electronic device may perform simulated operation on the client program, and optionally, the method of performing simulated operation on the client program may be as follows: the electronic device may be installed with a script for simulating user operation, and the electronic device may simulate the operation of the user on the client-side interactive interface by running the script, for example, simulate the operation of the user clicking a key of the client-side interactive interface for making a reservation for the internet contract carrier, or simulate the operation of the user clicking a key of the client-side interactive interface for paying a taxi-taking order. Therefore, the front-end program of the client can detect and respond to the simulated operations, so that the programs related to the operations in the client program can run under the coordination of the front-end program of the client and the back-end program of the client, and the related program running can enable the client to display the client interactive interface related to the operations.

Of course, the manner of using the script is one manner of the present application, and should not be taken as a limitation of the present application; for example, it may also set corresponding detection programs in the client program, so that the client can implement the client itself based on running these detection programs and also can simulate the operation of the user on the client interactive interface.

It should be noted that, in order to ensure the comprehensiveness of detecting the client program, the electronic device may simulate all client interactive interfaces that can be generated by the operation of the client program. However, the processing manner of the electronic device for each of all the client interactive interfaces may be the same, so that the embodiment is described by taking the example that the electronic device processes one of all the client interactive interfaces, so as to facilitate understanding of the present solution, but the present invention is not limited thereto.

In the process of testing the client program through simulation operation, the electronic device can acquire the client interactive interface generated by the operation of the client program in a screenshot sampling mode and store the client interactive interface for subsequent matching.

As an optional way for facilitating subsequent matching, the script in the electronic device also presets the relevant information of each client interactive interface in all the client interactive interfaces of the client, and the relevant information of each client interactive interface may represent: each client interactive interface is an interface corresponding to what function in the client, and which objects are contained in each client interactive interface, and what the function of each object is. For example, an order payment button of the client-side interactive interface may be an object in the client-side interactive interface, or a map interface in the client-side interactive interface may also be an object in the client-side interactive interface, or a client icon in the client-side interactive interface may also be an object in the client-side interactive interface. Based on this, while the electronic device obtains the client-side interactive interface, the electronic device may further establish an association relationship between the obtained client-side interactive interface and the relevant information of the client-side interactive interface, so that the relevant information may be searched and used based on the association relationship in the following.

It should also be noted that, in order to provide efficiency for testing the client program, the electronic device may further determine whether the simulated operation of the client program enables the client to correspondingly generate and display a corresponding client interaction interface when the electronic device performs a test process on the client program through the simulated operation.

If the electronic equipment judges that the corresponding client-side interactive interface is generated, the electronic equipment can capture the screenshot of the client-side interactive interface and continue to execute the subsequent image matching process.

If the electronic device determines that the corresponding client-side interactive interface is not generated, the electronic device may not execute the subsequent image matching process, but may directly determine that the client-side program is in an abnormal state.

Then to ensure a complete description of the scenario, the present embodiment will proceed with the principle of the scenario as the electronic device determines that a corresponding client-side interactive interface is generated. That is, after the electronic device obtains the client-side interactive interface, the electronic device may continue to perform step S200.

All the client preset interactive interfaces generated by the client program running in the normal state can be stored in the electronic equipment in advance. The preset interactive interface of the client can be an image drawn on third-party software by a developer according to functions which can be realized after development or update of a client program.

That is to say, the client preset interactive interface table shows what the developer expects that the client program can generate the client interactive interface in actual operation, and if the client interactive interface is not matched with the client preset interactive interface, it indicates that the client program does not reach the function that the developer expects to be implemented, and thus it may indicate that the client program is in an abnormal state. Therefore, the electronic device may determine whether the client program is abnormal based on whether the client interactive interface matches the client preset interactive interface.

In detail, the electronic device also establishes an association relationship between each client preset interactive interface in all the client preset interactive interfaces and the related information. Therefore, after the electronic equipment obtains the client-side interactive interface and establishes the association relationship between the client-side interactive interface and the related information, the electronic equipment can previously establish the relationship with one client-side preset interactive interface based on the related information of the established association relationship, so that the client-side preset interactive interface corresponding to the client-side interactive interface is searched from all the client-side preset interactive interfaces. And then, the electronic equipment can carry out similarity matching on the client-side interactive interface and the client-side preset interactive interface.

As a first way of performing similarity matching between the client-side interactive interface and the client-side preset interactive interface, a trained similarity matching model may be preset in the electronic device, so that the electronic device may invoke the similarity matching model and input the client-side interactive interface and the client-side preset interactive interface into the similarity matching model for similarity matching calculation.

The similarity matching model can utilize algorithms of some deep neural networks to respectively calculate the client interactive interface and the client preset interactive interface. For example, the similarity matching model may perform histogram processing on the client-side interactive interface and the client-side preset interactive interface respectively, so as to obtain a grayed and straight-square client-side interactive interface and further obtain a grayed and straight-square client-side preset interactive interface. The similarity matching model can carry out convolution and pooling treatment on the gray and straight-square client interactive interface, so that multi-dimensional characteristic data of the client interactive interface can be obtained. Correspondingly, the similarity matching model can also perform convolution and pooling on the client preset interactive interface which is gray and is in a straight direction, so that multi-dimensional characteristic data of the client preset interactive interface can be obtained. The similarity matching model may perform similarity calculation on the multidimensional feature data of the client-side interactive interface and the multidimensional feature data of the client-side preset interactive interface based on the link layer, so as to calculate matching similarity between the client-side interactive interface and the client-side preset interactive interface, where the matching similarity may be a percentage value greater than 0 and less than or equal to 1.

The electronic device is also preset with a similarity threshold, and the similarity threshold may also be a percentage value greater than 0 and less than or equal to 1, for example, the similarity threshold may be 90%. Then, the electronic device may compare the matching similarity output by the similarity matching model with a similarity threshold to determine whether the matching similarity is smaller than the similarity threshold. When the matching similarity is smaller than the similarity threshold, it can indicate that the client interactive interface is not matched with the client preset interactive interface, otherwise, it indicates matching.

As a second way of performing similarity matching between the client-side interactive interface and the client-side preset interactive interface, the electronic device may further refine to determine a difference between the client-side interactive interface and the client-side preset interactive interface.

The electronic device may be preset with an object detection model, and the object detection model may extract the sub-image including the object in the image by using the methods of FasterRCNN, RFCN, SSD, RetinaNet, refindedt, YO L Ov2, and the like.

In detail, the object detection model may perform object recognition on the client interactive interface, so as to segment at least one sub-interface including at least one object from the client interactive interface, where, to improve accuracy of subsequent object matching, each sub-interface in the at least one sub-interface may include one object. Correspondingly, the object detection model may also perform object recognition on the preset interactive interface of the client, so as to segment at least one preset sub-interface including at least one preset object from the preset interactive interface of the client, wherein, in order to improve the accuracy of subsequent object matching, each preset sub-interface in the at least one preset sub-interface may also include one preset object.

After obtaining the at least one sub-interface and the at least one preset sub-interface, the electronic device may obtain the highest similarity of each object in each sub-interface based on the at least one sub-interface and the at least one preset sub-interface.

In this embodiment, the electronic device may be provided with similarity matching, where the similarity matching model in the second manner may be different from the similarity matching model in the first manner. An FCN network (full convolution network) in the similarity matching model may calculate each sub-interface in the at least one sub-interface to obtain a pixel probability matrix of each sub-interface, and the FCN network in the similarity matching model calculates each preset sub-interface in the at least one preset sub-interface, thereby obtaining a preset pixel probability matrix of each preset sub-interface and obtaining at least one preset pixel probability matrix in total.

It should be noted that the pixel probability matrix may be used to indicate whether each pixel in the sub-interface belongs to an object in the sub-interface or belongs to a background in the sub-interface, that is, the pixel probability matrix may be represented as C × L × W, where C is 0 or 1, where 1 may indicate that each pixel in the sub-interface belongs to an object in the sub-interface, where 0 may indicate that each pixel belongs to a background in the sub-interface, L may be a row number of pixels on the sub-interface, and W may be a column number of pixels on the sub-interface.

In this way, the similarity matching model can match each pixel probability matrix with at least one preset pixel probability matrix, respectively, so as to determine at least one matching similarity of the objects in each sub-interface. The more pixel points of the pixel probability matrix of each sub-interface are matched with the preset pixel probability matrix of each preset sub-interface, the higher the matching similarity between the object in each sub-interface and the preset object in each preset sub-interface is, and the lower the matching similarity is otherwise.

After the electronic device obtains the at least one matching similarity of each object, the electronic device may select, from the at least one matching similarity of each object, a highest similarity of each object that is the highest in similarity, where the highest similarity of each object may be a percentage value greater than 0 and less than or equal to 1. Then, the highest similarity of each object represents a similarity between each object and a most similar preset object of the at least one preset object, and the matching similarity between the client-side interactive interface and the client-side preset interactive interface may include the highest similarity of each object, that is, whether the client-side interactive interface and the client-side preset interactive interface are matched is represented by the highest similarity of each object.

The electronic device is also preset with a similarity threshold, and the similarity threshold may also be a percentage value greater than 0 and less than or equal to 1, for example, the similarity threshold may be 90%. Then, the electronic device may compare the highest similarity of each object output by the similarity matching model with the similarity threshold to determine whether the highest similarity of any object in the at least one object is smaller than the similarity threshold. When any matching similarity is smaller than the similarity threshold, the client interactive interface can be indicated to be not matched with the preset client interactive interface; conversely, a match may be indicated when the similarity of each match is greater than or equal to the similarity threshold.

Then, after the electronic device determines whether the client-side interactive interface matches the client-side preset interactive interface, the electronic device may continue to perform step S300.

Step S300: if not, determining that the client program is in an abnormal state.

If the electronic equipment determines that the client-side interactive interface is matched with the client-side preset interactive interface, the electronic equipment determines that the client-side program corresponding to the client-side interactive interface is normal, so that the prompt of program abnormity is not needed.

If it is determined that the client-side interactive interface is not matched with the client-side preset interactive interface based on the first manner, because the client-side interactive interface may be generated based on the running of a corresponding program in the client-side program, the electronic device may further determine, based on the unmatched client-side interactive interface, a program required to generate the unmatched client-side interactive interface in the client-side program, and determine that the program required to generate the unmatched client-side interactive interface is a program in an abnormal state in the client-side program.

If it is determined that the client-side interactive interface is not matched with the client-side preset interactive interface based on the second manner, because the unmatched object in the client-side interactive interface may also be generated based on the corresponding program running in the client-side program, the electronic device may further determine, in the client-side program, a program required to generate the unmatched object in the client-side interactive interface based on the unmatched object in the client-side interactive interface, and also determine that the program required to generate the unmatched object in the client-side interactive interface is a program in an abnormal state in the client-side program.

Therefore, the electronic equipment can also generate prompt information for indicating the programs in the abnormal state in the client programs, and the prompt information is transmitted to the terminal of the developer, so that the developer can quickly and accurately know that the programs in the client programs are in the abnormal state based on the prompt information.

As shown in fig. 3A to 5B, the processing flow of the foregoing second mode will be described below by way of an example.

The electronic device may obtain the client interactive interface a1 as shown in fig. 3A, and the electronic device is further preset with the client preset interactive interface a2 as shown in fig. 3B. The client-side interactive interface a1 includes an object B1 serving as a selection box control and also includes an object C1 serving as a picture, and the client-side preset interactive interface a2 includes a preset object B2 serving as a selection box control and also includes a preset object C2 serving as a picture.

The electronic device processes the client interactive interface a1 through the object detection model, and then obtains the sub-interface a11 shown in fig. 4A and the sub-interface a12 shown in fig. 5A. Sub-interface A11 contains the object B1, which is a checkbox control, and sub-interface A12 contains the object C1, which is a picture. Accordingly, the electronic device also processes the client-side interactive interface a2 through the object detection model, and then obtains the sub-interface a21 shown in fig. 4B and the preset sub-interface a22 shown in fig. 5B. The default sub-interface A21 includes the default object B2 as the selection box control, and the default sub-interface A22 includes the default object C2 as the picture.

In this way, the electronic device may match the sub-interface a11 with the preset sub-interface a21 and the preset sub-interface a22, respectively, based on the similarity matching model calculation, so that it may be found that the similarity of the object B1, which is the selection frame control, is 70% compared to the preset object B2, which is the selection frame control, and the similarity of the object B1, which is the selection frame control, is 10% compared to the preset object C2, which is the picture, so that the highest similarity of the object B1, which is determined to be the selection frame control, is 70%. Correspondingly, the electronic device may also match the sub-interface a12 with the preset sub-interface a21 and the preset sub-interface a22, respectively, based on the similarity matching model calculation, so that the similarity of the object C1 as a picture compared with the preset object B2 as a selection frame control may be 10%, and the similarity of the object C1 as a picture compared with the preset object C2 as a picture may be 98%, thereby determining that the highest similarity of the object C1 as a picture is 70%.

Then, in the case of a similarity of 90%, the electronic device may determine that the program corresponding to object B1 of the selection box control is an abnormal program.

Referring to fig. 6, some embodiments of the present application provide a program detecting apparatus 100, where the program detecting apparatus is run on an electronic device, and the program detecting apparatus 100 may include:

the interface obtaining module 110 is configured to obtain a client interaction interface generated by running a client program in a process of testing the client program through simulated running;

an interface matching module 120, configured to determine whether the client-side interactive interface matches a client-side preset interactive interface of the client-side program;

and the state determining module 130 is configured to determine that the client program is in an abnormal state if the client program is not in the abnormal state.

Optionally, the interface matching module 120 is further configured to determine a matching similarity between a client interaction interface and a client preset interaction interface of the client program; and judging whether the matching similarity is smaller than a similarity threshold, wherein when the matching similarity is smaller than the similarity threshold, the client interactive interface is not matched with the client preset interactive interface.

Optionally, the interface matching module 120 is further configured to perform image recognition on the client interaction interface, obtain at least one sub-interface including at least one object in the client interaction interface, perform image recognition on a client preset interaction interface of the client program, and obtain at least one preset sub-interface including at least one preset object in the client preset interaction interface, where each sub-interface in the at least one sub-interface includes one object, and each preset sub-interface in the at least one preset sub-interface includes one preset object; and performing image recognition on the at least one sub-interface and the at least one preset sub-interface, and determining the highest similarity of each object in the at least one object, wherein the highest similarity of each object is the similarity between each object and the most similar preset object in the at least one preset object, and the matching similarity comprises the highest similarity of each object.

And the interface matching module 120 is further configured to determine whether the highest similarity of any object in the at least one object is smaller than a similarity threshold, where when the highest similarity of any object is smaller than the similarity threshold, it indicates that the client-side interactive interface is not matched with the client-side preset interactive interface.

Optionally, the interface matching module 120 is further configured to perform image recognition on the at least one sub-interface, determine a pixel probability matrix of each sub-interface in the at least one sub-interface, perform image recognition on the at least one preset sub-interface, and determine at least one preset pixel probability matrix of the at least one preset sub-interface; wherein the pixel probability matrix represents whether each pixel point in the sub-interface belongs to an object in the sub-interface or a background in the sub-interface; matching each pixel probability matrix with the at least one preset pixel probability matrix respectively, and determining at least one matching similarity of the objects in each sub-interface; the highest similarity of each object is determined from the at least one matching similarity of each object.

Optionally, the program detecting apparatus 100 may further include:

and the information prompt module 140 is configured to generate prompt information for indicating a program in an abnormal state in the client program, and transmit the prompt information to the terminal.

The interface judgment module 150 is configured to judge whether the client-side interactive interface is generated by the operation of the client-side program in the process of testing the client-side program through simulated operation; if yes, executing the following steps: acquiring a client interactive interface generated by the running of the client program; if not, determining that the client program is in an abnormal state.

An update determining module 160, configured to detect the client program and determine whether the client program is updated; if yes, simulating the operation of the user on the client-side interactive interface, enabling the client-side program to respond to the operation and operate, and executing the following steps: and in the process of testing the client program through simulation operation, obtaining a client interactive interface generated by the operation of the client program.

It should be noted that, as those skilled in the art can clearly understand, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

Some embodiments of the present application also provide a computer-readable storage medium of a non-volatile program code executable by a computer, the computer-readable storage medium having a program code stored thereon, the program code, when executed by the computer, performing the steps of the program detecting method of any of the above embodiments.

Specifically, the storage medium can be a general-purpose storage medium, such as a removable disk, a hard disk, or the like, and when the program code on the storage medium is executed, the steps of the program detection method of the above embodiment can be executed to implement efficient and low-cost test for finding various size problems of the client program

The program code product of the program detection method provided in the embodiment of the present application includes a computer-readable storage medium storing the program code, and instructions included in the program code may be used to execute the method in the foregoing method embodiment, and specific implementation may refer to the method embodiment, which is not described herein again.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In summary, the embodiments of the present application provide a program detection method, an apparatus, an electronic device, and a storage medium. The method comprises the following steps: in the process of testing a client program through simulation operation, a client interactive interface generated by the operation of the client program is obtained; determining whether the client interactive interface is matched with a client preset interactive interface of a client program; if not, determining that the client program is in an abnormal state.

The electronic equipment can control the client program to operate in a simulation mode at any time, so that whether the client program is abnormal or not can be tested on the basis of matching of a client interactive interface generated by the operation of the client program and a client preset interactive interface. Compared with manual testing in the prior art, the testing device realizes high-efficiency and low-cost testing. And because the testing time is free, the electronic equipment can be tested before or after the client program is online, so that various size problems of the client program can be found in time through testing.

The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A program detection method, the method comprising:

in the process of testing a client program through simulation operation, a client interactive interface generated by the operation of the client program is obtained;

determining whether the client interactive interface is matched with a client preset interactive interface of the client program;

if not, determining that the client program is in an abnormal state.

2. The program detection method of claim 1, wherein determining whether the client interactive interface matches a client preset interactive interface of the client program comprises:

determining the matching similarity between the client-side interactive interface and a client-side preset interactive interface of the client-side program;

and judging whether the matching similarity is smaller than a similarity threshold, wherein when the matching similarity is smaller than the similarity threshold, the client interactive interface is not matched with the client preset interactive interface.

3. The program detection method according to claim 2, wherein determining the matching similarity between the client-side interactive interface and the client-side preset interactive interface of the client-side program comprises:

performing image recognition on the client-side interactive interface to obtain at least one sub-interface containing at least one object in the client-side interactive interface, and performing image recognition on a client-side preset interactive interface of the client-side program to obtain at least one preset sub-interface containing at least one preset object in the client-side preset interactive interface, wherein each sub-interface in the at least one sub-interface contains one object, and each preset sub-interface in the at least one preset sub-interface contains one preset object;

performing image recognition on the at least one sub-interface and the at least one preset sub-interface, and determining the highest similarity of each object in the at least one object, wherein the highest similarity of each object is the similarity between each object and the most similar preset object in the at least one preset object, and the matching similarity comprises the highest similarity of each object;

correspondingly, judging whether the matching similarity is smaller than a similarity threshold value includes:

and judging whether the highest similarity of any object in the at least one object is smaller than a similarity threshold, wherein when the highest similarity of any object is smaller than the similarity threshold, the client interactive interface is not matched with the preset client interactive interface.

4. The program detection method according to claim 3, wherein performing image recognition on the at least one sub-interface and the at least one preset sub-interface to determine a highest similarity of each object in the at least one object comprises:

performing image recognition on the at least one sub-interface to determine a pixel probability matrix of each sub-interface in the at least one sub-interface, and performing image recognition on the at least one preset sub-interface to determine at least one preset pixel probability matrix of the at least one preset sub-interface; wherein the pixel probability matrix represents whether each pixel point in the sub-interface belongs to an object in the sub-interface or a background in the sub-interface;

matching each pixel probability matrix with the at least one preset pixel probability matrix respectively, and determining at least one matching similarity of the objects in each sub-interface;

the highest similarity of each object is determined from the at least one matching similarity of each object.

5. The program detection method of any of claims 1-4, wherein after determining that the client program is in an abnormal state, the method further comprises:

and generating prompt information for indicating the programs in the abnormal state in the client programs, and transmitting the prompt information to the terminal.

6. The program detection method according to any one of claims 1 to 4, wherein before obtaining the client interactive interface generated by the client program, the method further comprises:

judging whether the client program runs to generate the client interactive interface or not in the process of testing the client program through simulated running;

if yes, executing the following steps: acquiring a client interactive interface generated by the running of the client program; if not, determining that the client program is in an abnormal state.

7. The program detection method according to any one of claims 1 to 4, wherein before obtaining the client interactive interface generated by the client program, the method further comprises:

detecting the client program and judging whether the client program is updated or not;

if yes, simulating the operation of the user on the client-side interactive interface, enabling the client-side program to respond to the operation and operate, and executing the following steps: and in the process of testing the client program through simulation operation, obtaining a client interactive interface generated by the operation of the client program.

8. A program detecting apparatus, characterized in that the apparatus comprises:

the interface obtaining module is used for obtaining a client interactive interface generated by the running of a client program in the process of testing the client program through simulated running;

the interface matching module is used for determining whether the client interactive interface is matched with a client preset interactive interface of the client program;

9. The program detecting device according to claim 8,

10. The program detecting device according to claim 9,

the interface matching module is further configured to perform image recognition on the client-side interactive interface to obtain at least one sub-interface including at least one object in the client-side interactive interface, and perform image recognition on a client-side preset interactive interface of the client-side program to obtain at least one preset sub-interface including at least one preset object in the client-side preset interactive interface, where each sub-interface in the at least one sub-interface includes one object, and each preset sub-interface in the at least one preset sub-interface includes one preset object; performing image recognition on the at least one sub-interface and the at least one preset sub-interface, and determining the highest similarity of each object in the at least one object, wherein the highest similarity of each object is the similarity between each object and the most similar preset object in the at least one preset object, and the matching similarity comprises the highest similarity of each object;

11. The program detecting device according to claim 10,

12. The program detecting device according to any of claims 8-11, wherein the device further comprises:

13. The program detecting device according to any of claims 8-11, wherein the device further comprises:

14. The program detecting device according to any of claims 8-11, wherein the device further comprises:

15. An electronic device, characterized in that the electronic device comprises: a processor, a memory, a communication bus, and a network port; the processor, the network port and the memory are connected by the communication bus;

the memory is used for storing programs;

the processor, configured to execute the program detection method according to any one of claims 1 to 7 by calling a program stored in the memory.

16. A computer-readable storage medium having computer-executable nonvolatile program code, the program code causing the computer to execute the program detection method according to any one of claims 1 to 7.