WO2013121394A1

WO2013121394A1 - Remote debugging service

Info

Publication number: WO2013121394A1
Application number: PCT/IB2013/051247
Authority: WO
Inventors: André Daniel MOREIRA PINTO RIBOIRA; Rui Filipe LIMA MARANHÃO DE ABREU
Original assignee: Universidade Do Porto
Priority date: 2012-02-15
Filing date: 2013-02-15
Publication date: 2013-08-22

Abstract

The present disclosure relates to a debugging module and method, suitable for a network use, to debug distributed systems and/or technologically heterogeneous software implementations. In particular, the disclosure comprises a debugging service module comprising: a remote debugging sub-module configured for: receiving source code coverage messages from the system or systems-under-test through a network; processing the received source code coverage messages to perform a fault localization analysis; and generating debugging reports; and further comprising an API sub-module for communicating with the remote debugging sub-module and for communicating with the system or systems-under-test through said network, in particular for receiving and/or sending said messages.

Description

D E S C R I P T I O N

"REMOTE DEBUGGING SERVICE"

Technical field

[0001] The present disclosure relates to a debugging module and method, suitable for a network use, to debug distributed systems and/or technologically heterogeneous software implementations.

Background Art

[0002] Software is created by source code written in a given programming language. Being a manual process, either directly or indirectly, even with automated code production e.g. CASE, software may have several faults, mainly on large projects. When a fault is activated during the software run-time, the result of the software is not as expected, producing in this way a failure. Typically, software projects are not failure-tolerant, which leads to a system error. These failures are usually easy to identify, because the user realizes that the software is producing unexpected results. One of the most cumbersome and expensive tasks during software development is the localization of the root cause of these failures.

[0003] The currently most used methods to localize faults are mainly manual, by altering the software to produce intermediate outputs (to a standard output or a log file) during test executions, or to run some test executions step-by-step (using breakpoints) to analyze the system status at each point. This approach is very useful mainly when the programmer can reproduce the exact system state that leads to a system failure, and when the software project is simple enough to analyze with a step- by-step approach, or when the programmer already has a rather clear idea of the fault location. This is not what usually happens on large software projects, mainly those that use many different modules and technologies. [0004] Manual fault localization techniques are shown not to be the most effective when dealing with large and/or complex software projects. Because of this drawback, there were developed some techniques that automatically pinpoints the most probable software faults locations, based on sets of test executions. The most effective techniques use statistical analysis to calculate the failure probability of each software section.

[0005] Among the techniques that automatically pinpoint software faults locations, one of the most effective is the Spectrum-based Fault Localization (SFL) [1]. This technique uses as input data obtained from test executions of the System Under Test (SUT). These data includes the software source code coverage data record, which pinpoints which sections were used during a given test execution, and the respective test execution result data record. Having these data, an algorithm is executed to process the failure probability of each software section. Some of the most effective algorithms are Ochiai, Barinel and Tarantula [3].

[0006] There are already some tools that explores this technique, such as Tarantula [10], Zoltar [9] and Gzoltar [11,12]. Although these tools are useful when dealing with local and technologically homogeneous systems, they cannot be used directly to test distributed systems, nor systems that integrate different technologies, because they need to make changes at compilation level, to obtain the needed information about the test executions. Because these changes are dependent of the compiler that was used to build the software, the tools that are currently available do not support a global analysis of a system composed by multiple technologies.

Summary

[0007] An aspect of the disclosure comprises a debugging service module comprising: a remote debugging sub-module configured for: receiving source code coverage messages from the system or systems-under-test through a network; processing the received source code coverage messages to perform a fault localization analysis; and generating debugging reports;

and further comprising an API sub-module for communicating with the remote debugging sub-module and for communicating with the system or systems-under- test through said network, for receiving and/or sending said source code coverage messages.

[0008] Some embodiments comprise a front-end sub-module for remote access by users through a network to debugging reports generated by the remote debugging sub-module.

[0009] In some embodiments the remote debugging sub-module is also configured for receiving through a network a message noticing a test execution start by the system or systems-under-test.

[0010] In some embodiments the remote debugging sub-module is also configured for receiving through a network a message noticing a test execution end, with or without the test result, by the system or systems-under-test.

[0011] In some embodiments the remote debugging sub-module is also configured for receiving through a network a message with additional information on the system or systems-under-test and/or the test execution by the system or systems-under-test, in particular: the time measurements of execution of a given software section, the system resources used by a given software section, the entities a given software section communicated with, and/or metrics on the communications by a given software section.

[0012] In some embodiments the performed fault localization analysis is spectrum- based fault localization, and the remote debugging sub-module is further configured for storing, and accumulating by processing said source code coverage messages, a hit spectra matrix and an error vector, wherein the hit spectra matrix comprises the code coverage hit spectra of a predefined number of tests over a predefined number of code coverage items, i.e. software components, and wherein the error vector comprises the information in which of said tests errors were detected. [0013] In some embodiments the source code coverage messages comprises a unique identifier of the code coverage item, i.e. software component, that is executed by the system or systems-under-test and that triggered the sending of said message.

[0014] In some embodiments the unique identifier of the code coverage item, i.e. software component, is unique across machines even if running different instances of the same software component.

[0015] In some embodiments the remote debugging sub-module is also configured for receiving any of said received messages, including the source code coverage messages, with encrypted contents.

[0016] In some embodiments the debugging reports comprise probability failure of each code coverage item, i.e. software component.

[0017] In some embodiments the messages are transmitted over a network by using a reliable transport protocol or a protocol based on a reliable transport protocol, or one or more of HTTP, HTTPS, SNMP, TCP, UDP, IP protocols.

[0018] Another aspect of the disclosure comprises a debugging service method comprising the steps of: a remote debugging sub-module receiving source code coverage messages from the system or systems-under-test through a network;

the remote debugging sub-module processing the received source code coverage messages to perform a fault localization analysis; and

the remote debugging sub-module generating debugging reports; and further comprising

an API sub-module communicating with the remote debugging sub-module and also communicating with the system or systems-under-test through said network, for receiving and/or sending said source code coverage messages.

[0019] Any of the above described embodiments, in isolation or combination with other embodiments, is also applicable to the methods aspect of the disclosure. [0020] An aspect of the disclosure comprises a computer readable medium comprising the computer program comprising computer program code adapted to perform any of the above methods when said program is run on a data processor.

[0021] Localizing software faults is a cumbersome and rather expensive task [8]. Some advances were made to attempt to localize faults automatically. One of the most efficient techniques for automatic fault localization is Spectrum-based Fault Localization [3]. The present disclosure describes a module and method to allow the use of this technique on systems under test that can be composed by multiple different technologies and can use distributed architectures. This versatility can be achieved by using a remote network service.

[0022] This disclosure relates to a debugging module and method, implementing a debugging service. Debugging typically refers to the task of fixing software faults. Particularly, but not exclusively, the disclosure relates to a debugging service that is suitable for a network use, to debug distributed systems and/or technologically heterogeneous software implementations. This service is also intended to enhance the performance and accuracy of currently, state-of-the-art automatic debugging techniques. It explores the parallelizable processing property of the majority of the algorithms used in these techniques, by namely taking advantage of GPGPU techniques [6]. It also reduces some processing weight from local machines, because the heavy processing is done on a dedicated server or set of servers. The accuracy of the currently available techniques is also improved by using historic data analysis techniques [7].

[0023] According to one of the main aspects of the present disclosure there is provided a network service that detects remotely the location of faulty software source code. This service is capable of dealing with distributed systems and systems that integrates different technologies.

[0024] It uses as input an aggregation of data obtained by test executions of the SUT. The data collected during the executions is information about whether the execution was successful or not and the code coverage for the execution. Other complementary information can also be sent.

[0025] This information is sent by the SUT as messages through a network to the debugging service module of the present disclosure. The SUT can be manually configured to send these messages, or can be automatically changed to do that task. When the SUT is manually configured to send these messages, the software developer inserts hit functions directly into the application's source code. This function sends a message to our service or to a proxy server, containing a unique identifier of the software component where the function is located. It may also contain additional data such as the identification of the test execution or the customer id. These hit functions may also be automatically inserted into the source code using external tools. Moreover, they can be inserted directly into the application bytecode, also using external tools or, can be inserted into machine code, or even more generally, at any intermediary level between source code and machine code. Insertion can be carried out in known ways e.g. by make-file configuration, compiling, and/or linking whether static or dynamic.

[0026] The debugging service module of the present disclosure can return some results directly to the SUT to enhance the testing task, and can offer debugging reports for local and remote access.

[0027] The results are generally a ranking with the failure probability of each software component. This data may be automatically used by the testing system to vary the instrumentation granularity, or to automatically generate debugging reports.

[0028] For the present disclosure, the SUT should be able to gather information related to the test executions and send it as messages to the debugging service module of the present disclosure. This information can be, but not limited to, some of the following:

- Test Execution Start, to inform the start of a new test of the SUT;

- Test Execution End, to inform the end of the current test of the SUT;

- Test Execution Result, to inform if the current test passed or failed; - Test Execution Code Coverage, to inform that a given section of the software was executed;

- Test Execution Performance Records, to inform the time measurements related with the execution of each software section;

- Test Execution Used Resources, to inform which system resources were used by a given software section;

- Test Execution Communications, to inform with which entities a given software section communicates, and some metrics about that communication.

[0029] According to an aspect of the disclosure, the messages sent from the client to the debugging service generally follow this example structure:

[0030] Where "Customerjd" identifies the service customer, the "Projectj^'d" identifies the current System Under Test, the "Versionj^'d" identifies the version of the current System Under Test, the "Testj^'d" identifies the current test execution, and the "<content>" field has the message content, which may be the test start tag, a covered component during the test, or the test end tag with the test result.

[0031] The message may contain additional data fields, and is sent using network protocols such as TCP/IP or UDP/IP. Each protocol has its advantages and disadvantages, and is selected according to the implementation.

[0032] The preferred protocols are in general TCP/IP using HTTPS. However, for certain implementations, the security and reliability level of these protocols may be disposed over performance, by selecting different network protocols.

[0033] The "<content>" data field may be encrypted because it may contain sensitive data related to the covered component. In this case it remains encrypted during the message transmissions and processing (in the service). Due to the nature of Spectrum based Fault Localization, this data does not need to be decrypted to be processed. The statistical methods for Spectrum based Fault Localization can be analogously applied to encrypted data. However, the debugging report will also include this data encrypted and is necessary to be decrypted in the client. This processing may require a proxy server managed by the client and located between the System Under Test's modules and the debugging service, to encrypt and decrypt the messages and debugging reports. This guarantees the SUT confidentiality.

[0034] The debugging service module of the present disclosure collects the information sent by the SUT and processes it to perform a fault localization analysis. This analysis can be made, but not limited by, some of the following artificial intelligence techniques:

- SFL

- Data mining

- Machine learning

- Model-based

[0035] Using the information collected by the messages received from the SUT, and optionally information present on the debugging service such as databases, the present system processes an indicator representing the failure disposition of the SUT sections. This module can be from a project to a line of code, depending on the information sent by the SUT to the debugging service module of the present disclosure.

[0036] The processed information can be sent directly to the SUT, as messages, to help the testing process. The processed information can also be used in automatic report generation, to offer relevant debugging information for a local and remote access.

[0037] The automatically generated debugging reports can be integrated in IDE's, to allow a direct access to the source code of the software areas identified as problematic.

Brief Description of Drawings

[0038] The following figure provides preferred embodiments for illustrating the description and should not be seen as limiting the scope of disclosure. [0039] Figure 1: Schematic representation of the main components of the disclosure.

Detailed Description

[0040] The present disclosure provides a mechanism that can be used to debug software projects. This mechanism is able to debug technological heterogeneous software projects and software that runs in a distributed environment. The mechanism acts as a service that is accessible through a network.

[0041] Figure 1 illustrates model of an example of the present disclosure in use. The model is composed by three main components: the SUT 1, the Debugging Service 16 and the Debugging Service Front-End 19.

[0042] The SUT 1 can be a distributed software, having different sections running on different machines. On this example it is represented two different machines: Machine A 2 and Machine B 3. These machines communicate through a Network A 7. The SUT can also use different technologies. On this example there are depicted three different modules, each one using a different technology. These modules are Module A 8, Module B 9 and Module C 10. These modules can communicate with each other through communication channels represented as 8, 9 and 10. Each module is prepared to include a hit function to send a message to the debugging service indicating that it was executed, providing a unique identifier.

[0043] By sending messages that follow the same structure, messages coming from very different machines, e.g. in terms of languages, operating systems, manufacturers, software versions, can be accommodated easily. Even incorporating data from machines executing in third-party facilities is possible, e.g. at client facilities, cloud data centres, upstream sub-module developers, downstream integrators, etc...

[0044] The present system and method is able to deal with replicated systems (such as ghosts and load balancers) where it is not usually possible, in some embodiments, to obtain a unique identifier because in these cases the faults will also be replicated and these systems may be treated as a single instance presenting the same results. In other embodiments, a second identifier may be used for differentiating between machines running different instances of the same source code, or, alternatively, the unique identifier may combine source code component and also machine identification.

[0045] The message with this data may be sent from every software component able to send messages through a computer network.

[0046] The Debugging Service 16 is a remote service that is able to send and receive messages to/from the SUT 1. These messages can be delivered through a Network B 11. This network can be or not the same as Network A 7. These messages can be a notice of a test execution start 12, a set of messages with notices of software code coverage 13, a notice of a test execution end 14 with its respective result, and/or an intermediate debugging report 15. The SUT 1 can use the intermediate debugging reports 15 to aid on test selection and code coverage detail level.

[0047] The Debugging Service 16 processes debugging reports and generates outputs (Debugging Service Front-End 19). These outputs are displayed on a web browser, an IDE or other tool, that accesses remotely the Debugging Service 16. To access the debugging reports, the tools that displays the generated outputs uses a Network C 17, that can be, or not, the same as Network A 7 and/or Network B 11. The communication between Debugging Service 16 and Debugging Service Front-End 19, through Network C 17, is done using a network communication protocol 18.

[0048] In some embodiments of the present disclosure a debugging service is provided through a network. This service can be provided by a single server or a distributed system connected to one or many networks. This debugging service uses data obtained from many Systems Under Test and data obtained from its own database to perform a fault localization analysis, generating debugging reports with the failure disposition of the SUT sections. These reports are accessible remotely through a device that can be a web browser, an IDE or other tool.

[0049] In an example of operation of the debugging service according to an embodiment of the present disclosure, it was created a software project that is our SUT. This software project runs in two different machines. The first machine runs Apple MacOS X and the second machine runs Microsoft Windows. The first machine executes two main modules, developed in different technologies. The first module was developed using Oracle Java and the second module was developed using Perl. The second machine executes one main module, developed in Microsoft .NET technology. The first and the second machines communicate through a TCP/IP network.

[0050] It was also created a simple Debugging Service, which is able to receive TCP/IP messages [5], with notices of a test start, code coverage and test end with the rest result. With this data is executed the Ochiai Algorithm [2] to calculate the failure disposition of the SUT's sections. After 5 executions an intermediate debugging report is sent to the SUT.

[0051] To allow remote access to the debugging reports data, it was created a web application that is accessible using the HTTP protocol. A user is able to access this web server, provided by The Apache HTTP Server Project, and have the failure disposition of the SUT's sections, using a traditional web browser, like Mozilla Firefox.

[0052] An Eclipse plug-in [4] was also developed to allow the remote access to the debugging reports. This plug-in shows the web pages generated by the web server in an Eclipse View. It also listens the page activity to allow the user to jump directly to a given line of code.

[0053] See as reference for the above the following: http://www.apple.com/macosx/; http://windows.microsoft.com/; http://www.java.com/; http://www.perl.org/; http://www.microsoft.com/net/; http://tools.ietf.org/html/rfc2616; http://httpd.apache.org/; http://www.mozilla.com/firefox/; http://www.eclipse.org/;

[0054] The created SUT and the created Debugging Service communicates through a TCP/IP network, using standard TCP/IP messages. The three created modules of the SUT communicate with the Debugging Service using TCP/IP messages. The first module, developed in Oracle Java sends the test start message. After that it sends code coverage messages indicating the respective source code path and line of code (the SUT can be manually configured to send these messages using directly the communication protocol, using an API offered by the debugging service, or can be automatically changed to do that task, following a set of predefined conditions). The SUT calls the module developed in Perl, which also sends code coverage messages. The Perl script calls a remote server, created in Microsoft .NET technology, that runs on the second machine. This Microsoft .NET module also sends code coverage messages. It returns a value to the Perl script, and the Perl script returns a value to the Oracle Java module. Finally, the Oracle Java module sends a test end message, with the test result (if the processed result is the expected or not). The debugging reports generated by the created Debugging Service are accessible through a TCP/IP network, using standard TCP/IP messages. The reports are deployed using the HTTP - Hypertext Transfer Protocol.

[0055] Overall, a debugging service according to the present disclosure provides a powerful debugging tool that is able to deal with distributed software systems that combines different technologies. This debugging service is able to, but is not limited to, use the SFL technique, independently of the technologies used on the SUT, or its architecture.

[0056] As described some embodiments comprise a debugging service able to pinpoint faulty source code of the software used in a SUT. The SUT can be technologically homogeneous and/or heterogeneous, and can be mono-location or distributed. The debugging service comprises:

- An API that facilitates the communication between a SUT and a remote debugging service.

- A remote debugging service. This service receives messages from software test executions, sent through a network. The remote debugging service processes the received information and is able to complement it with extra information present on its own databases. That process results in a report with indicators representing the failure disposition of the SUT sections.

- A front-end to allow a user to access remotely to the debugging reports generated by the debugging service.

[0057] In some embodiments, the source code coverage messages are sent through a network. [0058] In some embodiments, a message noticing the test execution start is sent through a network.

[0059] In some embodiments, a message noticing the test execution end is sent through a network, with or without the test result.

[0060] In some embodiments, a message with complementary information about the test execution is sent through a network.

[0061] In some embodiments, a message with complementary information about the SUT is sent through a network.

[0062] In some embodiments, a message with complementary information about the SUT environment is sent through a network.

[0063] In some embodiments, debugging reports are sent by messages through a network to a SUT.

[0064] In some embodiments, debugging reports are accessible remotely through a network.

[0065] In some embodiments, the debugging service executes the Spectrum-based fault localization technique.

[0066] In some embodiments, software projects that use multiple different technologies are analyzed simultaneously. In some embodiments, software projects that use distributed architectures are analyzed simultaneously.

[0067] In some embodiments, generated debugging reports are integrated into IDE's.

[0068] The above described embodiments are combinable. The following dependent claims set out particular embodiments of the disclosure.

Citations

[1] R. Abreu. Spectrum-based Fault Localization in Embedded Software. Phd thesis, Delft University of Technology, 2009. [2] R. Abreu, P. Zoeteweij, and A. Gemund. An Evaluation of Similarity Coefficients for Software Fault Localization. In Proceedings of the 12th Pacific Rim International Symposium on Dependable Computing (PRDC 2006), pages 39-46, Riverside, California, USA, 2006. IEEE.

[3] R. Abreu, P. Zoeteweij, R. Golsteijn, and A. Gemund. A practical evaluation of spectrum-based fault localization. Journal of Systems and Software, 82(11):1780-1792, November 2009.

[4] E. Clayberg and D. Rubel. Eclipse Plug-ins. Addison-Wesley Professional, 2008.

[5] B. Forouzan. TCP/IP Protocol Suite. McGraw-Hill Higher Education, 2002.

[6] N. Govindaraju, B. Lloyd, W. Wang, M. Lin, and D. Manocha. Fast computation of database operations using graphics processors. In Proceedings of the ACM SIGGRAPH

2005 Courses (SIGGRAPH 2005), volume 2004, Los Angeles, California, USA, 2005.

ACM.

[7] R. Grossman and Y. Gu. Data mining using high performance data clouds: experimental studies using sector and sphere. In Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining (KDD 2008), pages 920-927, Las Vegas, Nevada, USA, 2008. ACM.

[8] B. Hailpern and P. Santhanam. Software debugging, testing, and verification. IBM Systems Journal, 41(1):4-12, 2002.

[9] T. Janssen, R. Abreu, and A. Gemund. Zoltar: A Toolset for Automatic Fault Localization. In Proceedings of the 24th IEEE/ACM International Conference on Automated Software Engineering (ASE 2009), pages 662-664, Auckland, NZ, November 2009.

[10] J. Jones, M. Harrold, and J. Stasko. Visualization of Test Information to Assist Fault Localization. In Proceedings of the 24th international conference on Software engineering (ICSE 2002), number May, pages 467-477, Buenos Aires, AR, 2002. ACM.

[11] A. Riboira. GZoltar: A Graphical Debugging Interface. Msc thesis, Universidade do Porto, 2011. [12] A. Riboira, R. Abreu, and R. Rodrigues. Interactive visualizations of automatic debugging reports. In Proceedings of the V IberoAmerican Symposium in Computer Graphics (SIACG 2011), Faro, PT, 2011.

Claims

C L A I M S

1. Debugging service module comprising:

a. a remote debugging sub-module configured for:

i. receiving source code coverage messages from the system or systems- under-test through a network;

ii. processing the received source code coverage messages to perform a fault localization analysis; and

iii. generating debugging reports; and further comprising

b. an API sub-module for communicating with the remote debugging sub- module and for communicating with the system or systems-under-test through said network, for receiving and/or sending said source code coverage messages.

2. Debugging service module according to the previous claim further comprising a front-end sub-module for remote access by users through a network to debugging reports generated by the remote debugging sub-module.

3. Debugging service module according to any previous claim wherein the remote debugging sub-module is also configured for receiving through a network a message noticing a test execution start by the system or systems-under-test.

4. Debugging service module according to any previous claim wherein the remote debugging sub-module is also configured for receiving through a network a message noticing a test execution end, with or without the test result, by the system or systems-under-test.

5. Debugging service module according to any previous claim wherein the remote debugging sub-module is also configured for receiving through a network a message with additional information on the system or systems-under-test and/or the test execution by the system or systems-under-test, in particular: the time measurements of execution of a given software section, the system resources used by a given software section, the entities a given software section communicated with, and/or metrics on the communications by a given software section.

6. Debugging service module according to any previous claim wherein the performed fault localization analysis is spectrum-based fault localization, and the remote debugging sub-module is further configured for storing, and accumulating by processing said source code coverage messages, a hit spectra matrix and an error vector, wherein the hit spectra matrix comprises the code coverage hit spectra of a predefined number of tests over a predefined number of code coverage items, i.e. software components, and wherein the error vector comprises the information in which of said tests errors were detected.

7. Debugging service module according to any previous claim wherein the source code coverage messages comprises a unique identifier of the code coverage item, i.e. software component, that is executed by the system or systems-under-test and that triggered the sending of said message.

8. Debugging service module according to any previous claim wherein the unique identifier of the code coverage item, i.e. software component, is unique across machines even if running different instances of the same software component.

9. Debugging service module according to any previous claim wherein the remote debugging sub-module is also configured for receiving any of said received messages, including the source code coverage messages, with encrypted contents.

10. Debugging service module according to any previous claim wherein the debugging reports comprise probability failure of each code coverage item, i.e. software component.

11. Debugging service module according to any previous claim wherein the messages are transmitted over a network by using a reliable transport protocol or a protocol based on a reliable transport protocol, or one or more of HTTP, HTTPS, SNMP, TCP, UDP, IP protocols.

12. Debugging service method comprising:

a. a remote debugging sub-module receiving source code coverage messages from the system or systems-under-test through a network;

b. the remote debugging sub-module processing the received source code coverage messages to perform a fault localization analysis; and

c. the remote debugging sub-module generating debugging reports; and further comprising

d. an API sub-module communicating with the remote debugging sub-module and also communicating with the system or systems-under-test through said network, for receiving and/or sending said source code coverage messages.

13. Debugging service method according to the claim 12 further comprising providing a front-end sub-module for remote access to users through a network to debugging reports generated by the remote debugging sub-module.

14. Debugging service method according to any previous claims 12-13 wherein the remote debugging sub-module also receives through a network messages noticing a test execution start sent by the system or systems-under-test.

15. Debugging service method according to any previous claims 12-14 wherein the remote debugging sub-module also receives through a network messages noticing a test execution end, with or without the test result, sent by the system or systems-under-test.

16. Debugging service method according to any previous claims 12-15 wherein the remote debugging sub-module also receives through a network messages with additional information on the system or systems-under-test and/or the test execution, sent by the system or systems-under-test, in particular: the time measurements of execution of a given software section, the system resources used by a given software section, the entities a given software section communicated with, metrics on the communications by a given software section.

17. Debugging service method according to any previous claims 12-16 wherein the performed fault localization analysis is spectrum-based fault localization, and the remote debugging sub-module stores, and accumulates by processing said source code coverage messages, a hit spectra matrix and an error vector, wherein the hit spectra matrix comprises the code coverage hit spectra of a predefined number of tests over a predefined number of code coverage items, i.e. software components, and wherein the error vector comprises the information in which of said tests errors were detected.

18. Debugging service method according to any previous claim 12-17 wherein the source code coverage messages comprises a unique identifier of the code coverage item, i.e. software component, that is executed by the system or systems-under-test and that triggered the sending of said message.

19. Debugging service method according to any previous claim 12-18 wherein the unique identifier of the code coverage item, i.e. software component, is unique across machines even if running different instances of the same software component.

20. Debugging service method according to any previous claim 12-19 wherein the remote debugging sub-module receives any of said received messages, including the source code coverage messages, with encrypted contents.

21. Debugging service method according to any previous claim 12-20 wherein the debugging reports comprise probability failure of each code coverage item, i.e. software component.

22. Debugging service method according to any previous claim 12-21 wherein the messages are transmitted over a network by using a reliable transport protocol or a protocol based on a reliable transport protocol, or one or more of HTTP, HTTPS, SNMP, TCP, UDP, IP protocols.

23. Computer readable medium comprising the computer program comprising computer program code adapted to perform the method of any of the claims 12 - 22 when said program is run on a data processor.