KR20230050602A - Test automation system and method for testing system-on-chip design validation - Google Patents

Abstract

The present invention relates to a test automation system and method for system on chip design verification. The test automation system according to the present invention includes a configuration management unit having a repository for storing a source code file for verifying a system-on-chip design that is committed and pushed from a customer terminal in which an integrated development environment is installed, and when the source code file is updated in the repository, It includes an automation unit that controls an automated test for an updated source code file to be performed, and a build server unit that performs an automated test under the control of the automation unit. According to the present invention, it is possible to automate a test for verifying a system on chip design through cooperation with a customer company.

Description

Test automation system and method for testing system-on-chip design validation

The present invention relates to a test automation system and method, and more particularly, to a test automation system and method capable of automating tests for system-on-chip design verification in cooperation with a customer company.

System on Chip is a system composed of devices with multiple functions made into a single chip. For example, main semiconductor elements such as a CPU, a memory element, and a digital signal processing element are implemented in one chip so that the chip itself becomes one system.

Long-term design and various functions are required for system-on-chip development, and IP (Intellectual Property) blocks are used to easily develop large-scale circuits at low development costs by securing standardized function blocks. An IP block is an intellectual property function block that can be commonly reused in system-on-chip design. When using an IP block, system-on-chip design efficiency is increased, performance is improved, and development period is shortened.

Therefore, an appropriate IP block is required for efficient design and verification of the system on chip, but it is not easy to secure ownership of the IP block. In other words, there is no problem in using the IP block that you own, but if it is an IP block that you do not own, you have to know who has the IP block you need, and you also have to pay the cost to purchase the IP block. Occurs.

On the other hand, DevOps is a compound word of software development and operation, and is a concept that refers to a development environment or culture that emphasizes communication, collaboration, and integration between software developers and technical experts. The benefits of devops are that you work faster, so you can innovate faster for your customers, adapt better to market changes, and create business outcomes more efficiently.

Therefore, it is necessary to automate the test process by establishing a development environment in which the same devops concept is applied to the system on chip design process, and also to use various IP blocks and verified tools for system on chip design verification. also need

Accordingly, an object of the present invention is to provide a test automation system and method capable of automatically performing a test for verifying a system-on-chip design in cooperation with a customer company.

A test automation system according to the present invention for achieving the above object is a configuration management unit having a repository for storing a source code file for verifying a system-on-a-chip design that is committed and pushed from a customer terminal in which an integrated development environment is installed. When the source code file is updated, an automation unit for controlling an automated test on the source code file to be performed, and a build server unit for performing the automated test under the control of the automation unit.

In the build server unit, an IP (Intellectual Property) block that can be used to verify the system on chip design in the customer terminal may be installed, and the automation unit, after completion of the automated test, results files related to the automated test results. can provide.

The configuration management unit may store the result file and manage storage and sharing of the source code file, and the automation unit may include a CI (Continuous Integration) tool module implemented in Jenkins to control the automated test. there is.

In addition, the test automation method according to the present invention for achieving the above object includes the steps of committing a source code file for a system on chip design test in a customer terminal where an integrated development environment is installed and pushing it to a storage provided in a configuration management unit, automation When the source code file is updated in the repository in the unit, controlling an automated test for the source code file to be performed, and performing the automated test in a build server unit under the control of the automation unit .

And, in order to achieve the above object, in the present invention, it is possible to provide a processor-readable recording medium on which a program for executing the test automation method in a processor is recorded.

According to the present invention, it is possible to provide a test automation system capable of automating tests for system-on-chip design verification by establishing a development environment to which the devops concept is applied and collaborating with a customer. It can provide a collaboration tool that automates source configuration management, quality inspection, and distribution. In addition, it is possible to save resources by automating repetitive tasks through the test automation system, and to reduce test time by quickly providing feedback on test results and improvements to customers, and to use IP blocks and proven tools. It can also be used to improve test accuracy. In addition, the operator can strengthen security by separating the build server part, and can protect IP and license.

1 is a view referenced in the description of the configuration of a test automation system according to an embodiment of the present invention;
2 is a block configuration diagram of the automation unit shown in FIG. 1;
3 is a flow chart provided to explain a process of updating a source code file in a test automation method according to an embodiment of the present invention;
4 is a flowchart provided for explanation of an automated test process in a test automation method according to an embodiment of the present invention;
5 is a diagram showing an example of a project main screen, and
6 shows an example of a project build history screen.

In this specification, when a component is referred to as "connected" or "connected" to another component, a component may be directly connected or connected to another component, but other components in the middle It should be understood that elements may be present. Other expressions describing the relationship between components, such as "between" or "adjacent to", and expressions such as "transmitting" a signal from one component to another, should be interpreted similarly.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

1 is a diagram referenced for description of the configuration of a test automation system according to an embodiment of the present invention.

Referring to FIG. 1 , a customer terminal 100 , an operator terminal 150 , and a test automation system 200 are communicatively connected through a communication network 10 .

The communication network 10 refers to all networks in which communication is possible between the customer terminal 100, the operator terminal 150, and the test automation system 200, such as a mobile communication network, the Internet, or a network combined therewith.

The client company terminal 100 is a terminal of a customer company using the test automation system 200 , and the operator terminal 150 is a terminal of an operator operating the test automation system 200 .

An integrated development environment (IDE) for system-on-chip design verification is installed in the customer terminal 100 and the operator terminal 150, respectively. The integrated development environment is software for building applications that combine common developer tools into a single graphical user interface (GUI).

The test automation system 200 may include a configuration management unit 210 , an automation unit 230 and a build server unit 250 .

The configuration management unit 210 includes a repository 213 that stores source code files for verifying system-on-chip designs committed and pushed in the integrated development environment of the customer terminal 100 in units of projects. The storage 213 is accessible from the customer terminal 100 and can be managed by separating into a customer use storage that stores only the codes necessary for the customer company and an operation use storage that stores the entire project of the operator company and is accessible only to the operator company. In addition, the operator company and the customer company may collaborate by allowing the operator terminal 150 and the customer terminal 100 to share the source code files stored in the storage 213 .

When a source code file for system on chip design verification is updated in the storage 213, the automation unit 230 controls the build server unit 250 to perform an automated test on the updated source code file. After the automation test is completed in the build server unit 250, the automation unit 230 may provide a result file related to the automated test result to the customer terminal 100 and the operator terminal 150. Result files related to automated test results may be stored and managed in the configuration management unit 210 .

The build server unit 250 performs an automated test for system on chip design verification under the control of the automation unit 230 . The build server unit 250 may perform an automated test by compiling the updated source code file in the customer terminal 100 and then simulating the source code file.

In the build server unit 250, an IP (Intellectual Property) block and simulation program owned by the operator are installed, and the customer terminal 100 performs system-on-chip design verification using the IP block and program owned by the operator. can

In addition, the build server unit 250 is configured so that it cannot be directly accessed from the client company terminal 100, thereby increasing security stability and protecting licenses for IP blocks and various programs.

The build server unit 250 may also perform an automated test by a daily build performed according to a predetermined schedule or a manual build manually performed by a developer when necessary.

With this configuration, it is possible to provide an automated system for system on chip design verification to customers.

FIG. 2 is a block configuration diagram of an automation unit shown in FIG. 1 .

Referring to FIG. 2 , the automation unit 230 may include a CI (Continuous Integration) tool module 231, a result providing module 233, a messenger module 255, and an issue management module 257. When these components are implemented in actual applications, two or more components may be combined into one component, or one component may be subdivided into two or more components as needed.

The CI tool module 231 can be implemented with Jenkins or the like, and controls the execution of automated tests. Jenkins is a tool that provides continuous integration services when developing software. When multiple developers develop a program, Jenkins enables continuous integration by uploading their work to a repository in a shared area to prevent version conflicts.

The CI tool module 231 includes an operational use CI tool capable of accessing an operational use repository and performing a full regression test, and a customer use CI tool that accesses a customer use repository and performs unit tests, etc. includes

The CI tool module 231 is a commit build performed when a source code file for system-on-chip design verification is committed and pushed, a daily build performed according to a pre-determined schedule, and a manual build performed manually when needed by a developer. You can set the type of build, such as, and control its execution.

The result providing module 233 provides automated test results, and may provide a log file or a waveform diagram file as the test result.

The messenger module 255 notifies the customer company terminal 100 or operator terminal 150 of the state when a simulation fails or a specific event occurs.

The issue management module 257 manages issues generated during build and automated test processes. Issues include almost everything related to the system-on-chip design verification process, such as errors, bugs, new features, and work requests. The issue management module 257 can be implemented using Jira or the like.

With such configuration modules, the automation unit 230 can be configured.

3 is a flowchart provided to explain a source code file update process in a test automation method according to an embodiment of the present invention.

Referring to FIG. 3 , the customer terminal 100 executes a web browser, accesses the login screen provided to the test automation system 200, and enters the user name and password set in advance on the login screen to access the system. (S300).

When accessing the system for the first time, a process of adding a Secure Shell (SSH) public key may be performed in order to work in the local environment of the customer terminal 100 . SSH is a cryptographic network protocol for safely operating network services on unsecured networks.

When logged in and connected to the system, a screen listing all the projects that the customer has access to is displayed. The user of the customer terminal 100 selects and duplicates a desired project (S310).

Then, a new source code file for system on chip design verification is created in the workspace of the customer terminal 100, or a source code editing process of modifying an existing source code file is performed (S310).

When editing of the source code file is completed, a new commit is created and then pushed to the storage 213 (S315 and S320). In the process of pushing after committing the source code, the target test branch can be designated. The system-on-chip design process can generally be verified by creating a test branch for each block before simulating the entire chip.

Since the customer company terminal 100 and the operator terminal 150 can share a designated area in the storage 213, they can collaborate by connecting to a synchronized working space. In addition, every source code file update to the repository 213 triggers a new build in the test automation system 200 to perform automated tests.

Through this process, the storage 213 may be updated with a source code file for system on chip design verification.

4 is a flowchart provided to explain an automated test process in a test automation method according to an embodiment of the present invention.

Referring to Figure 4, first, a preparation process is performed (S400). The preparation process prepares the environment for a new build, which includes setting up build descriptions and user notifications.

Next, source code files to be built are updated (S405).

Then, the updated source code file is compiled (S410). Depending on the compilation, files in the format of *.elf, *.bin, *.hex, etc. may be created from the newly updated source code file.

After the compilation is performed, the simulation is executed using the file generated as a result of the compilation (S420). Simulation may be performed with a newly compiled test branch from a previously built RTL (Register Transfer Level) file.

When the build and simulation are completed, the simulation result is output to the customer terminal 100 or the like (S425). When build and simulation are completed, simulation completion may be notified to a designated terminal through a messenger program or the like.

If the simulation fails or a problem occurs in the automated test process, a process of solving the problem may proceed through collaboration between the customer company and the operator using the customer terminal 100 and the operator terminal 150.

5 and 6 show an example of a screen displayed in an automated test process.

5 shows an example of a project screen.

Referring to FIG. 5 , automation tests can be executed in units of projects. In the project screen 500, a general description part 503 displays a general description of the project, and a stage view part 505 shows each build record step by step. Indicates status, and step-level logs are supported.

The side panel portion 507 displays menu items that can be selected by the user.

Here, if the 'Status' item is selected, the status of the project is displayed, and if the 'Changes' item is selected, changes in source code management are displayed. If you select the 'Build' item, manual build is executed, and if you select the 'Full Stage View' item, the screen is displayed focusing on the stage view. And, if 'Open Blue Ocean' is selected, the current project is opened in blue ocean mode. Blue Ocean is one of the Jenkins plugins for graphical visualization and diagnostics of Jenkins pipeline projects.

6 shows an example of a project build history screen.

Referring to FIG. 6 , the project build history screen 520 shows information on previous and current tasks.

In the project build history screen 520, a general description portion 523 describes general information of the build record. This includes the node used, the time it started, and the cause of the operation.

The side panel portion 525 displays menu items that can be selected by the user.

Here, if you select the 'Status' item, it indicates the status of the build history, and if you select the 'Changes' item, it indicates changes made to commits in Jenkins source code management. If you select the 'Console Output' item, the real-time log of the Jenkins project is displayed, and if you select the 'View Build Information' item, information about a specific job number is displayed. If you select the 'See Fingerprints' item, the fingerprint of the *.jar file created by Jenkins is displayed, which indicates when and why the file was created. If you select the 'Open Blue Ocean' item, the current page is displayed in blue ocean mode, which is useful for visualizing the build history.

If you select the 'Restart from Stage' item, it restarts at a specific stage. Selecting 'Reply' will replay the current build with the same build configuration, but if the build configuration is not updated, this has no special meaning. If you select the 'Pipeline Steps' item, the pipeline log for each pipeline step is displayed. If you select the 'Previous build' item, the previous build is displayed. If there is no previous build, this item is not displayed. And, if you select the 'Next build' item, the next build is displayed. If there is no next build, this item is not displayed.

In addition, various menu screens can be provided during the automated test process.

On the other hand, the test automation system and method according to the present invention can not be applied limitedly to the configuration of the embodiments described above, but the above embodiments are all or part of each embodiment so that various modifications can be made. It may be selectively combined and configured.

In addition, the present invention can be implemented as a computer program on a programmable computer. Such computers may include processors, storage devices, input devices, and output devices. In order to implement the contents described in the present invention, the program code may be input with a mouse or keyboard input device. These programs can be implemented in high-level languages or object-oriented languages. It can also be implemented as a computer system implemented in assembly or machine code.

The present invention can also be implemented as processor-readable codes on a processor-readable recording medium. The processor-readable recording medium includes all types of recording devices in which data readable by the processor is stored. Examples of the processor-readable recording medium include ROM, RAM, CD-ROM, and an optical data storage device. In addition, the processor-readable recording medium is distributed in computer systems connected through a network, so that processor-readable codes can be stored and executed in a distributed manner.

In addition, although the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. Of course, various modifications are possible by those skilled in the art, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

100: customer terminal 150: operator terminal
200: test automation system 210: configuration management unit
230: automation unit 250: build server unit

Claims (15)

a configuration management unit having a repository for storing a source code file for system-on-chip design verification that is committed and pushed from a customer terminal in which an integrated development environment is installed;
When the source code file is updated in the storage, an automation unit for controlling an automated test on the source code file to be performed; and
A test automation system including a build server unit that performs the automated test under the control of the automation unit. According to claim 1,
Test automation system, characterized in that in the build server unit, an IP (Intellectual Property) block that can be used for verifying the system-on-chip design in the customer terminal is installed. According to claim 1,
The automation unit, after completion of the automated test, provides a result file related to the automated test result. According to claim 3,
The test automation system, characterized in that the configuration management unit stores the result file and manages the storage and sharing of the source code file. According to claim 1,
The automation unit is a test automation system including a CI (Continuous Integration) tool module implemented in Jenkins to control the automated test. According to claim 1
The test automation system, characterized in that the build server unit performs the automated test even according to a preset schedule or manual request. According to claim 1,
The test automation system further includes an operator terminal that cooperates with the customer terminal by sharing the source code file stored in the storage. According to claim 1,
The test automation system, characterized in that the storage is separated and managed into a storage shared with the customer terminal and a storage accessible only through the operator terminal Committing a source code file for a system on chip design test in a customer terminal where an integrated development environment is installed and pushing the source code file to a storage provided in a configuration management unit;
Controlling an automated test on the source code file to be performed when the source code file is updated in the repository by an automation unit; and
A test automation method comprising the step of performing the automated test in a build server unit according to the control of the automation unit. According to claim 9,
The test automation method further comprising providing, by the automation unit, a result file related to the automated test result after completion of the automated test. According to claim 10,
The test automation method further comprising the step of storing the result file in the configuration management unit and controlling the saving and sharing of the source code file. According to claim 9,
The test automation method further comprising a step of collaborating with the customer terminal by sharing the source code file stored in the storage from the operating company terminal. According to claim 9,
The test automation method further comprising the step of performing the automated test by the build server unit according to a preset schedule or manual request. According to claim 9,
The test automation method further comprising notifying a state to the customer terminal when the automated test fails or a specific event occurs. A processor-readable recording medium recording a program for executing the test automation method according to any one of claims 9 to 14 in a processor.

Images