TW202303388A - Device and method for identifying errors in a software application - Google Patents

Abstract

Aspects concern a method for identifying errors in a software application comprising obtaining a log of an execution of a plurality of test cases for a software application under test, searching for matches between regular expressions with strings in the log, wherein each regular expression is associated with an error type and, if a match of a regular expression with a string the log has been found, outputting information about the test for which the log contains the string and outputting the error type associated with the regular expression.

Description

Device and method for identifying errors in software applications

Various aspects of the present disclosure relate to devices and methods for identifying errors in software applications.

Validation and testing is an important part of developing software applications. In particular, errors causing application failures should be identified and corrected. A variety of tools currently exist to run test cases for applications under development. Such testing tools typically generate logs indicating how the application behaved under certain test cases. Often, a large number of test cases must be run to ensure that the application functions correctly under various conditions. Therefore, a large number of logs are created, and reviewing these logs to find and correct the cause of the failure requires a lot of effort.

Therefore, it is desirable to have a method that allows the identification and correction of errors in software applications to be more efficient.

Various embodiments relate to a method of identifying errors in a software application, comprising obtaining a log of the execution of a plurality of test cases of a software application under test, searching for a match between a regular expression and a string in the log, wherein each regular expression is associated with For an error type, if a match between the regular expression and the string in the log is found, output the test-related information whose log contains the string, and output the error type associated with the regular expression.

According to an embodiment, obtaining the log includes receiving test result data of a plurality of test cases, and parsing the log from the test result data.

According to one embodiment, obtaining the logs includes testing the software application according to a plurality of test cases.

According to an embodiment, the above method includes judging the relationship between the error type and the regular expression.

According to one embodiment, the test-related information includes an indication of the test case, or a test category, or both.

According to an embodiment, outputting the test-related information and outputting the error type includes writing the test-related information and the error type into a report log.

According to an embodiment, outputting test-related information and outputting error types includes uploading a report log to a server.

According to an embodiment, outputting the test-related information and outputting the error type includes presenting the test-related information and the error type.

According to one embodiment, presenting test-related information includes presenting information associated with tests in the test list.

According to one embodiment, presenting test-related information includes presenting root causes of failures assigned to tests.

According to an embodiment, the above method includes assigning a root cause in response to user selection of a predefined list of root causes.

According to one embodiment, presenting the test-related information includes obtaining a screenshot of the user interface output of the software application under test when an error occurs, and presenting the screenshot.

According to an embodiment, the above method includes presenting the screenshot in response to a user request for the screenshot.

According to one embodiment, the presentation test related information includes an excerpt of the presentation log, the excerpt including the found matches in the log.

According to an embodiment, the above method includes presenting the excerpt of the diary in response to a user request for the excerpt of the diary.

According to one embodiment, for at least one error type, a plurality of regular expressions are associated with the error type.

According to an embodiment, the above-mentioned regular expression is a regular expression of the first set of regular expressions, and the above-mentioned method further includes searching for a match between the second set of regular expressions and the strings in the log, wherein each of the second set of regular expressions The regular expression is associated with at least one passed test case, one failed test case, and a start of a test case.

According to one embodiment, searching for matches of the regular expressions of the first set of regular expressions to strings in the log includes first searching for matches of the regular expressions of the second set of regular expressions associated with the beginning of the test case, searching The beginning of the test case, and then search for the match between the regular expression of the first set of regular expressions and the string in the log for the test case.

According to one embodiment, searching for a match between the regular expressions of the first set of regular expressions and the strings in the log includes first searching for the fault by searching for a match of the regular expressions of the second set of regular expressions associated with the fault test case test case, and then searches the test case for a match between the regular expression of the first set of regular expressions and the string in the log containing the test case.

According to one embodiment, searching for a match between the regular expression of the first set of regular expressions and the string in the log includes consulting the log, and stopping the review when a match between the string in the log and the regular expression associated with the end of the log is found Logs, which are related to searching for matches between regular expressions and log strings.

According to an embodiment, the regular expressions include at least one regular expression associated with a preset error type, wherein the method includes outputting the default error type if a match between at least one regular expression and a line of the log is found, unless Also find a match for the regular expression associated with the particular error type in that line of the log.

According to an embodiment, a server computer is provided, which includes a communication interface, a memory, and a processing unit, configured to execute the method according to one of the above embodiments.

According to one embodiment, there is provided a computer program element comprising program instructions which, when executed by one or more processors, cause the one or more processors to perform a method according to one of the above-mentioned embodiments.

According to one embodiment, there is provided a computer-readable medium comprising program instructions which, when executed by one or more processors, cause the one or more processors to perform a method according to one of the above-mentioned embodiments.

The following description refers to the accompanying drawings, which show specific details and examples in which the disclosure can be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the disclosure. Other embodiments may be utilized and structural and logical changes may be made without departing from the scope of the present disclosure. The various embodiments are not necessarily mutually exclusive, and some embodiments can be combined with one or more other embodiments to form new embodiments.

Embodiments described in the description of one of the devices or methods are similarly valid for the other devices or methods. Likewise, what is described in the description of an apparatus is similarly valid for a system or method, and vice versa.

Features described in the description of one embodiment can be correspondingly applied to the same or similar features in other embodiments. Features described in the description of one embodiment can be correspondingly applied to other embodiments even if they are not described in these other embodiments. In addition, the additions and/or combinations and/or substitutions described for features in the description of one embodiment can be correspondingly applied to the same or similar features in other embodiments.

In the description of various embodiments, the use of "a" and "the" in relation to a feature or element includes references to one or more features or elements.

As used herein, "and/or" includes any and/or all combinations of one or more of the listed items.

Examples will be described in detail below.

FIG. 1 shows a computer 100 for developing software applications.

The computer 100 includes a central processing unit (CPU) 101 and a system memory (RAM) 102 . The system memory 102 is used to load program codes (ie loaded from the hard disk 103 ), and the CPU 101 executes the program codes.

In this example, it is assumed that the user intends to use the computer 100 to develop software applications.

To this end, the user executes a software development environment 104 , such as an integrated development environment like Xcode (for iOS), on the CPU 101 .

The software development environment 104 allows users to develop applications 105 for various devices 106 (eg, smartphones), especially smartphones. To this end, the CPU 101, as a part of the software development environment 104, runs a simulator (simulator) to simulate the device corresponding to the application program, such as an Iphone.

When the user has successfully developed the application program, the user can distribute the application program to the corresponding device 106 via the communication network 107, such as uploading the application program to an app store to distribute it to a smart phone.

Before doing so, however, the user should test the application 105 to avoid distributing an application that does not work correctly to the device 106 .

To this end, the user further runs a test tool 108 on the CPU 101 , such as an XC test. With the testing tool 108, the user can run multiple test cases. Therefore, the test tool 108 outputs the test result data in one or more test result files, such as XCRResult files, which the user can use to find errors in the application program 105 and finally correct them.

Typically, a large number of tests are run. Therefore, a large amount of test result data will be generated, and it is very heavy for the user to check all the result data.

Accordingly, according to various embodiments, a system for processing test result data is provided that allows for more efficient analysis of test results.

FIG. 2 shows a test system 200 according to one embodiment.

The main components can be found in the Report Parser (or Results Parser) 201, and the Reporting Front End (FE) Application 202 (also known as the Root Cause Analyzes (RCA) FE Application) and The reporting server 204 of the reporting backend (BE) server 203 (also referred to as the RCA backend server). The reporting server 204 can be, for example, a part of a cloud, for example, coupled to the computer 100 via the communication network 100 .

The report parser 201 receives the test result data 205 (such as XCResult file) generated by the test tool 206 to test the software application program on a specific device (such as the above-mentioned simulated device). The test tool 206 and the report parser 201 can be implemented by the computer 100, for example, but can also be distributed to multiple machines. These machines can be implemented, for example, by Jenkins nodes. The report parser 201 uploads the results to the report server 204 .

The report parser 201 and the report server 204 perform test result reporting and test result analysis, and provide options for reporting root causes of failures by combining multiple techniques (excluding simple HTML reports). In particular, according to various embodiments, the report parser parses information from the test result data to help the user 208 easily understand the root cause of software application failures. The results of the report parser 204 can be obtained by the user 208 via the report server 204 .

For example, the report parser parses XCResult files and prepares reports in JSON or XML format. The RCA FE application 202 presents reports in tabular format. For example, the RCA FE application 202 can continue to be online and running, so that the user 208 can continue to access it.

The RCA BE server 203 uses a database to store reports. The RCA BE server 203 may have sufficient memory to store scrrenshots and logs, and may have sufficient processing power to serve many requests simultaneously. The RCA BE server 203 can also go online and run continuously, so that the user 208 can keep accessing.

Hereinafter, the operation of the report parser 201 will be described in more detail with reference to FIG. 3 .

FIG. 3 shows a report parser 300 .

As explained with reference to FIG. 2 , report parser 300 receives test result file 301 . The test result file 301 may be a result file of a software application test on a different device (which may be a simulated device), or a result file of a different software application test. Each test result file can be regarded as the original version of the test report. For example, after the execution of the test case, Xcode (a tool for iOS development and testing) generates a test report in XCResult format. Test reports can be analyzed by Xcode UI and command line tools.

Each test result file 301 may contain one or more logs of a test case. The report parser 300 extracts test case logs from a test result file 301 (such as an XCResult file), and parses the logs at operation 302 . For example, for an XCResult file, the report parser 300 internally uses the Xcode command line interface (CLI) to parse the log and obtain details such as test type, method, status, duration, etc. of each test case. In operation 303, the report parser 300 uploads the logs to the RCA server 309 (corresponding to the report server 204).

The report parser 300 reads the log of the test case more line by line, and finds out the type of error in the application (if the test case encounters a failure in the application) and the operation when the test case encounters a failure (to help identify the cause of the failure) Possible Causes). The report parser 300 uses a regular expression (regex) mechanism to do so in operation 304 , that is, finds a match of the regular expression in the log. Then, the report parser 300 prepares a report 311 (including the result of parsing using the regex mechanism), and uploads the report 311 to the RCA server 309 , eg in JSON format. The report parser 300 can further generate Junit XML for reporting to the Jenkins server 310 .

The report parser 300 can search for the screenshot attachment in the test result data of the fault case in operation 307 , map the screenshot attachment to the respective test case in operation 308 , and upload the screenshot to the RCA server 309 .

According to various embodiments, the RCA FE application 202 is a web application (eg, based on React) that displays the reports 311 uploaded by the report analyzer 300 in a tabular format with an interactive user interface (UI). For example, it can be based on Material UI, but any other UI framework can also be used.

For example, the RCA FE application 202 has a summary screen for displaying Reports Summary and RCA Summary. The summary screen displays a list of report 311 with summary (date and time, test category, task name, number of passed test cases, number of failed test cases, and total number of test cases), supports pagination, and uses RCA summary ( How many are due to server issues, UI issues, script issues...etc) displays a list of reports with a button to delete each report.

The RCA FE application 202 further has a report detail screen to display the detailed information of each test case in the report 311 .

FIG. 4 shows an example of a report detail screen 400 presented by the RCA FE application 202 .

The report details screen shows key information of each test case, such as class name 401 , method name 402 , status 403 , and duration 404 .

In addition, the report detail screen 400 emphasizes the fault tab 405 of the fault case, such as UI, functional plane, framework, . . . . The user can hover the cursor over the status to make the RCA FE application 202 display the cause of the failure.

The report details screen 400 further provides a button 406 for each test case to present a drop-down menu for selecting a root cause.

Figure 5 shows a portion of the report detail screen in Figure 4 with a drop-down menu presented for a certain test case.

The report detail screen 400 further has a button 407 for each test case to display a screenshot of the test case, and a button 408 to display a log of the test case.

Figure 6 shows a part of the report detail screen in Figure 4 with a screenshot presented for a certain test case.

Screenshot 600 shows the user interface of the tested application at the time of failure.

Similarly, a log of a test case may be presented.

Additionally, the report details screen 400 has a button 409 to allow the user to enter comments for the test case.

According to various embodiments, the RCA BE server 203 is a web server. The RCA BE server 203 exposes an Application Programming Interface (API) for Representational State Transfer (REST) calls to upload and obtain reports. For example, RCA BE server 203 uses Django as framework, Django REST API framework, Gunicorn plus Nginx as deployment setup, and Mysql as database.

For example, the RCA BE server 203 provides the following APIs: api/reports => provide a list of reports with detailed information (including case details) api/report => provide case information for a specific report api/summary => only provide test summary and RCA summary of each component (build) api/update_rca => PUT call to update RCA for each test case api/upload_logs => POST call to upload the test log text file to the server api/upload_screen_shot => POST call to upload screenshot to server media => call to access previously uploaded media (image or text)

For example, a Jenkins task that executes a test case on a node machine looks for test result files (such as XCResult files) from different test fixtures. Once the raw report (i.e. the test result file) is found, the Jenkins job starts to parse the raw report, merges the reports from all devices, and uploads the merged report to the RCA server using the above API. Finally, users can access reports by loading the web application into a browser. The report browser supports various platforms and can provide more information than the above.

As mentioned above, using the regex mechanism at operation 304 to identify the type of error that caused the application under test to fail in the test case is an important feature of various embodiments.

FIG. 7 shows a flowchart 700 illustrating identifying error types from test logs.

Once the test result file 701 (such as the XCResult file) is received, the report parser 300 parses the test result file 701 to find logs of test cases in the test result file 701 at operation 702 . In the case where the test result file 701 is an XCResult file, the report parser 300 can use XCParse for it.

When the report parser 300 has found the log, it processes the log line by line, that is, starts processing each line in an (outer) loop at operation 703 . On each iteration of the (outer) loop, report parser 300 processes the current line using regular expressions in the following manner.

In this example, the following regular expressions are used, please refer to Regex 1 to 7 below. Regex: 1. test_start_regex = r'Test [Cc]ase \'\-\[(.+?)\s(.+?)\]\' started' 2. passed_test_regex = r'Test [Cc]ase \'\-\[(.+?)\s(.+?)\]\' passed\s\((.+?)\)' 3. failed_test_regex = r'Test [Cc]ase \'\-\[(.+?)\s(.+?)\]\' failed\s\((.+?)\)' 4. assert_fail_regex = r'(.+?)\serror:\s(.+?) [Ff]ailed(.*)' 5. cs_cmd_fail_regex = r'(.+?)\serror code=\s(.+?)' 6. tear_down_regex = r'(.+?)Tear Down(.+?)' 7. log_end_failed_regex = r'Test Suite \'RetryTestSuite\' failed at (.+?)' 8. log_end_passed_regex = r'Test Suite \'RetryTestSuite\' passed at (.+?)'

The report parser 300 checks whether there is a match of Regex 1 in operation 704 , checks whether there is a match of Regex 7 in operation 705 , and checks whether there is a match of Regex 8 in operation 706 .

If there is a Regex 7 match, the report parser 300 stops looping because the end of the log has been reached (indicating a failure).

For example, there is a line in the test log "Test Suite 'RetryTestSuite' failed at 2020-11-04 16:49:06.060", so there is a Regex 7 match.

If there is a match for Regex 8, the report parser 300 stops looping because the end of the log has been reached (indicating a pass).

For example, there is a line "Test Suite 'RetryTestSuite' passed at 2020-11-04 16:49:06.060" in the test log, so there is a Regex 8 match.

If there is a match of Regex 1, the report parser 300 parses the test category and test case from the current line in operation 707 . The report parser writes the results to a dictionary 722 containing the content of the current test case to upload the report to the RCA server 309 .

For example, there is a line "Test Case '-[TestMicrositeTextComponent testTextContentAsSet]' started" in the test log, so there is a match of Regex 1.

When the report parser 300 has found the beginning of the test case in the log (by looking for a Regex 1 match), the report parser 300 starts an inner loop at operation 708 to process the next line. In each iteration of the inner loop, the report parser 300 checks at operation 709 for a Regex 4 match, at operation 710 for a Regex 5 match, and at operation 711 for a Regex 2 match, In operation 712 it is checked whether there is a match of Regex 3, and in operation 713 it is checked whether there is a match of Regex 6.

If a match of Regex 4 is found, the report parser 300 checks at operation 714 whether one of the following UI error regular expressions matches the current row: ● query_fail_regex = r'Get number of matches for: Elements matching predicate \'(.+?)\'' ●query_fail_regex2 = r'Get all elements bound by index for: Elements matching predicate \'(.+?)\'' ● exits_fail = r'Checking existence of (.+?)' ● find_fails = r'Find: Elements matching predicate \'(.+?)\'' ● find_fails2 = r'Find the\s(.+?)' ●snap_shot_fails = r'Using snapshot previously cached by application'

If one of these regular expressions matches, the report parser 300 considers this as a UI error that has occurred and writes “ui_err” to the dictionary 722 at operation 715 .

If none of these regular expressions match, report parser 300 treats this as another type of error at operation 716 and writes “other” to dictionary 722 .

For example, there is a line in the test log " t = 282.14s Assertion Failure: TestMicrositeTextComponent.m:54: Failed to get matching snapshot: No matches found for Elements matching predicate 'identifier ==[c] "viewText"' from input { (", so there is a match in Regex 4.

If a match of Regex 5 is found, the report parser 300 regards this as a core server (Core Server; CS) error in operation 717 and writes "cs_err" into the dictionary.

For example, there is a line in the test log "2021-06-04 09:37:55.615466+0800 ShopeeSG-Runner[6735:1581092] Updating shop info failed , error code= 19", so there is a Regex 5 match.

If a match of Regex 2 is found, the report parser 300 considers the test case passed. Report parser 300 parses the duration from the log and writes the status and duration to dictionary 722 at operation 718 .

For example, there is a line "Test Case '-[TestMicrositeTextComponent testTextContentAsSet]' passed (44.243 seconds)" in the test log, so there is a match for Regex 2.

If a match of Regex 3 is found, the report parser 300 treats the test case as a failure. Report parser 300 parses the duration from the log and writes the status and duration to dictionary 722 at operation 719 . In addition, the report parser 300 parses the screenshot corresponding to the current test case (presented when the tested application fails) at operation 720 . Then, the report parser 300 leaves the inner loop and continues with the outer loop operation 703 (to the next line down, ie, the line next to the line processed by the inner loop). When the entire set of test cases has been parsed, the contents of the dictionary 722 (eg, converted to JSON and/or XML at operation 721 ) and screenshots are uploaded to the RCA server 309 at operation 723 .

For example, there is a line "Test Case '-[TestMicrositeTextComponent testTextContentAsSet]' failed (283.255 seconds)" in the test log, so there is a Regex 3 match.

The report parser 300 determines that the Regex 3 matches and can parse out the duration from this line.

If a match of Regex 6 is found, the report parser 300 leaves the inner loop, and proceeds to the outer loop operation 703 (to the next line that goes down, that is, the next line of the line processed by the inner loop). For example, there is a line "t = 282.85s Tear Down" in the test log, so there is a Regex 6 match.

The above embodiments can add more features, such as ● Execute tests (i.e. actively test instead of just receiving test results) ● Integration with JIRA ● Email report filled with RCA ● Graphical test suite health ● User authentication

Overall, according to various embodiments, a method is provided, as illustrated in FIG. 8 .

FIG. 8 shows a flowchart 800 illustrating a method for identifying errors in a software application.

In step 801, execution logs of a plurality of test cases of the software application under test are obtained.

In step 802, matches of regular expressions and strings in the log are searched, wherein each regular expression is associated with an error type.

If a match between the regular expression and the string in the log is found, then in step 803, output the test-related information whose log contains the string, and output the error type associated with the regular expression.

According to various embodiments, in other words, regular expressions are used to find strings in test logs that allow for the type of error that caused the software application under test to fail. When a regular expression match is found, the corresponding error type is output along with test-related information.

The method in Fig. 8 can be used in the context of the above-mentioned embodiment, for example, as a tool for converting original test result files (such as XCResult files) into web-based test reports, and the above-mentioned web-based test reports can be obtained from other test frameworks (such as XCResult files) Espresso + Junit and Selenium + Junit) test reports into a single web application, where the tool provides the ability to analyze and collect failure causes, and the ability to publish test reports. With this, the tool implements algorithms to convert XCResult files into web-based reports, as well as consolidate test reports from multiple platforms into a single source.

In this way, the tool can unify reports from automation platforms such as iOS, Android, and web automation, and can analyze and find possible causes of failures. In addition, the tool provides options for users to select predefined root causes and provide custom comments for each failure. The tool can, for example, present the results (of various test cases) in a table of reports and root causes, and can support maintenance of old reports.

Root cause analysis that supports testing of software applications reduces the need for manual work and thereby aids in the automation of the software development cycle. The tool provides detailed test reports to help developers (users) understand problems very quickly and fix them when executing tests. The tool automatically identifies certain failures by reading the logs itself to look for signs of errors in the logs (for example, using regex (regular expression) validation), thus reducing the time for report analysis. According to the regular expression defined by the log type, it can be used to parse the log to quickly find the required information.

The method in Figure 8 can be used for different types of software application testing, such as UI automation for different platforms, and API automation. According to various embodiments, a tool is provided to improve the productivity and efficiency of an automated testing life cycle by providing a platform capable of serving as a unified test automation reporting platform for visualizing and recording test results. The tool can be used by any software company utilizing cross-platform (web, mobile, API) automated testing.

For example, the method in FIG. 8 is implemented by the server computer shown in FIG. 9 .

FIG. 9 shows a server computer system 900 according to one embodiment.

The server computer system 600 includes a communication interface 601 (for example, configured to receive test result data or test logs, and configured to output test-related information whose logs contain character strings and error types). The server computer 600 further includes a processing unit 602 and a memory 603 . The memory 603 can be used by the processing unit 602 to store, for example, data to be processed, such as logs. The server computer is set up to perform the method in Figure 8. It should be noted that the server computer system 900 may be a distributed system including a plurality of computers.

The methods described herein may be performed by one or more circuits, and various processing or computing units, devices and computing entities described herein (such as the processing unit 602 ) may be implemented by one or more circuits. In one embodiment, a "circuit" can be understood as any kind of logical implementation entity, which can be hardware, software, firmware or any combination thereof. Thus, in one embodiment, a "circuit" may be a hard-wired logic circuit or a programmable logic circuit, such as a programmable processor, such as a microcontroller. A "circuit" may also be software executed or implemented by a processor, such as any computer program, such as a computer program coded using a virtual machine. According to alternative embodiments, any other implementation of the respective functions described herein may also be understood as a "circuit".

Although the present disclosure has been shown and described with reference to specific embodiments, it should be understood by those skilled in the art that various embodiments may be made therein without departing from the spirit and scope of the invention as defined in the appended claims. changes and details. Accordingly, the scope of the present invention is indicated by the appended claims and is intended to embrace all changes that come within the meaning and range of equivalents of the claims.

100: computer 101:Central Processing Unit 102: System memory 103: hard disk 104:Software development environment 105: Apps 106: Device 107: Communication network 108:Test tools 200: Test System 201:Report Parser 202:Report front-end application 203:Report backend server 204:Report server 205: Test result information 206:Test tool 207: Device 208: user 301:Test result file 302-308: Operation 309:RCA server 310:Jenkins server 311: Report 400:Report detail screen 401: Category name 402: method name 403: Status 404: Duration 405: Fault label 406-409: button 500: Part of the report detail screen 600: screenshot 700: Flowchart 701:Test result file 702-721: Operation 722:Dictionary 723:Operation 800: flow chart 801-803: Steps 900: server computer system 901: communication interface 902: processing unit 903: Memory

The present invention will be better understood by referring to the embodiments in the specification together with non-limiting examples and accompanying figures, wherein: Figure 1 shows a computer used to develop software applications. Figure 2 shows a test system according to one embodiment. Figure 3 shows a report parser, according to one embodiment. FIG. 4 shows an example of a report detail screen displayed according to one embodiment. Figure 5 shows a portion of the report details screen of Figure 4, with the drop-down menu displayed for a test case. Figure 6 shows a portion of the report detail screen of Figure 4, with the screen displayed for a test case. FIG. 7 shows a flowchart illustrating identifying error types from test logs according to an embodiment. FIG. 8 shows a flowchart illustrating a method for identifying errors in a software application according to an embodiment. Figure 9 shows a server computer system according to one embodiment.

800: flow chart

801-803: Steps

Claims (18)

A method for identifying errors in a software application, comprising: obtaining a log of the execution of a plurality of test cases of a software application under test; searching for matches of regular expressions with strings in the log, where each regular expression is associated with an error type; If a match between a regular expression and a character string of the log is found, output the test-related information whose log contains the character string, and output the error type associated with the regular expression; Obtaining the log includes receiving test result data of a plurality of test cases, and analyzing the log from the test result data; wherein the regular expressions are regular expressions of the first set of regular expressions, and the method further includes searching for matches of the second set of regular expressions with strings in the log; and associating each regular expression of the second set of regular expressions with a passed test case, a failing test case, and the beginning of a test case, wherein if the regular expression is associated with the beginning of the test case, the method further Search for this error type. The method of claim 1, wherein obtaining the log includes testing the software application according to the plurality of test cases. The method as claimed in claim 1 or 2, including judging the relationship between the error type and the regular expressions. The method according to any one of claims 1 to 3, wherein the test-related information includes an indication of the test case, or a test category, or both. The method according to any one of claims 1 to 4, wherein outputting the test-related information and outputting the error type includes writing the test-related information and the error type into a report log. The method of claim 5, wherein outputting the test-related information and outputting the error type includes uploading the report log to a server. The method according to any one of claims 1 to 6, wherein outputting the test-related information and outputting the error type includes presenting the test-related information and the error type. The method of claim 7, wherein presenting the test-related information includes presenting information associated with tests in a test list. The method of claim 7 or 8, wherein presenting the test-related information includes presenting a root cause of a failure assigned to the test. The method of claim 9, comprising assigning the root cause in response to a user selection of a predefined list of root causes. The method according to any one of claims 7 to 10, wherein presenting the test-related information includes obtaining a screenshot of the user interface output of the software application under test when an error occurs, and presenting the screenshot. The method of claim 11, comprising presenting the screenshot in response to a user request for the screenshot. The method according to any one of claims 7 to 12, wherein presenting the test-related information comprises presenting an excerpt of the log, the excerpt including the string of the match found in the log. The method of claim 13, comprising presenting the excerpt of the diary in response to a user request for the excerpt of the diary. The method of any one of claims 1 to 14, wherein for at least one error type, a plurality of regular expressions are associated with the error type. A server computer, including a communication interface, a memory, and a processing unit, configured to execute the method of any one of claim items 1 to 15. A computer program element, including program instructions, which, when executed by one or more processors, causes the one or more processors to perform the method of any one of claim items 1 to 15. A computer-readable medium comprising program instructions which, when executed by one or more processors, cause the one or more processors to perform the method of any one of claims 1 to 15.

Images