KR102602534B1 - Test automation system and method for testing system-on-chip design validation - Google Patents

Abstract

The present invention relates to a test automation system and method for system-on-chip design verification. The test automation system according to the present invention includes a configuration management unit that has a storage for storing source code files for system-on-chip design verification committed and pushed from a customer terminal with an integrated development environment installed, and when the source code file is updated in the storage, It includes an automation unit that controls automated testing on updated source code files, and a build server unit that performs automated testing under the control of the automation unit. According to the present invention, testing for system-on-chip design verification can be automated through collaboration with customer companies.

Description

Test automation system and method for testing system-on-chip design validation}

The present invention relates to a test automation system and method, and more specifically, to a test automation system and method that can automate tests for system-on-chip design verification in collaboration with a customer company.

System on Chip is a system composed of devices with multiple functions made into a single chip. For example, major semiconductor elements such as computing elements (CPUs), memory elements, and digital signal processing elements are implemented on one chip so that the chip itself becomes one system.

System-on-chip development requires long-term design and various functions, and IP (Intellectual Property) blocks are used to easily develop large-scale circuits at low development costs by securing standardized functional blocks. IP blocks are intellectual property function blocks that can be commonly reused in system-on-chip design. When IP blocks are used, the design efficiency of system-on-chip increases, and performance improvement and development period can be shortened.

Therefore, an appropriate IP block is required for efficient design and verification of a system-on-chip, but securing ownership of the IP block is not easy. In other words, if you own an IP block, there is no problem in using it, but if it is an IP block that you do not own, you need to know who holds the necessary IP block, and you also have to pay a fee to purchase the IP block, etc. Occurs.

Meanwhile, DevOps is a compound word of software development and operations, and is a concept that refers to a development environment or culture that emphasizes communication, collaboration, and integration between software developers and technical experts. The benefit of DevOps is that you work faster, so you can innovate faster for your customers, adapt better to market changes, and drive business outcomes more efficiently.

Therefore, it is necessary to automate the testing process by establishing a development environment that applies the same DevOps concept to the system-on-chip design process, and in addition, a plan to use various IP blocks and verified tools to verify the system-on-chip design. is also needed.

Therefore, the purpose of the present invention is to provide a test automation system and method that can automatically perform tests for system-on-chip design verification in collaboration with a customer company.

The test automation system according to the present invention for achieving the above purpose includes a configuration management unit that has a storage for storing source code files for system-on-chip design verification committed and pushed from a customer terminal with an integrated development environment installed, and the storage. It includes an automation unit that controls an automated test to be performed on the source code file when the source code file is updated, and a build server unit that performs the automated test under the control of the automation unit.

In the build server unit, an IP (Intellectual Property) block that can be used to verify the system-on-chip design in the customer terminal may be installed, and the automation unit may provide a result file related to the automated test results after completing the automated test. can be provided.

The configuration management unit may store the result file and manage storage and sharing of the source code file, and the automation unit may include a CI (Continuous Integration) tool module implemented in Jenkins to control the automated test. there is.

In addition, the test automation method according to the present invention to achieve the above object includes the steps of committing a source code file for system-on-chip design testing at a customer terminal with an integrated development environment installed and pushing it to a repository provided in the configuration management unit, When the source code file is updated in the repository, the unit controls to perform an automated test on the source code file, and performs the automated test in the build server unit under the control of the automation unit. .

In order to achieve the above object, the present invention can provide a processor-readable recording medium on which a program for executing the test automation method on the processor is recorded.

According to the present invention, it is possible to provide a test automation system that can automate tests for system-on-chip design verification by establishing a development environment to which the DevOps concept is applied and collaborating with customers. In addition, the present invention provides a consistent form Collaboration tools that automate source configuration management, quality inspection, and distribution can be provided. In addition, the test automation system saves resources by automating repetitive tasks, reduces test time by quickly providing feedback on test results and improvements to customers, and provides IP blocks and proven tools. Test accuracy can also be improved by using it. Additionally, the operator can strengthen security by separating the build server unit and protect IP and licenses.

1 is a diagram referenced in the description of the configuration of a test automation system according to an embodiment of the present invention;
Figure 2 is a block diagram of the automation unit shown in Figure 1;
Figure 3 is a flowchart provided to explain the update process of the source code file in the test automation method according to an embodiment of the present invention;
Figure 4 is a flow chart provided to explain the automated test process in the test automation method according to an embodiment of the present invention;
5 is a diagram showing an example of the project main screen, and
Figure 6 shows an example of a project build history screen.

In this specification, when a component is referred to as being “connected” or “connected” to another component, the component may be directly connected or connected to the other component, but may be connected to another component in the middle. It should be understood that elements may exist. Other expressions that describe the relationship between components, such as “between” or “neighboring to,” and expressions such as “transmitting” a signal from a component to another component, should be interpreted similarly.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

1 is a diagram referenced in the description of the configuration of a test automation system according to an embodiment of the present invention.

Referring to FIG. 1 , a customer terminal 100, an operator terminal 150, and a test automation system 200 are communicatively connected through a communication network 10.

The communication network 10 refers to any network capable of communicating between the customer terminal 100, the operator terminal 150, and the test automation system 200, such as a mobile communication network, the Internet, or a network combined thereof.

The customer terminal 100 is a terminal of a customer company that uses the test automation system 200, and the operator terminal 150 is a terminal of an operator that operates the test automation system 200.

An integrated development environment (IDE) for system-on-chip design verification is installed in the customer terminal 100 and the operator terminal 150, respectively. An integrated development environment is software for building applications that combines common developer tools into one graphical user interface (GUI).

The test automation system 200 may include a configuration management unit 210, an automation unit 230, and a build server unit 250.

The configuration management unit 210 is provided with a repository 213 that stores source code files for system-on-chip design verification committed and pushed in the integrated development environment of the customer terminal 100 on a project basis. The repository 213 is accessible from the customer terminal 100 and can be managed by dividing it into a customer repository that stores only the codes needed by the customer, and an operational repository that stores the entire project of the operating company and is accessible only to the operating company. In addition, the operator terminal 150 and the customer company terminal 100 can share the source code file stored in the storage 213, so that the operator company and the customer company can collaborate.

When the source code file for system-on-chip design verification is updated in the storage 213, the automation unit 230 controls the build server unit 250 to perform an automated test on the updated source code file. After completing the automated test in the build server unit 250, the automation unit 230 may provide a result file related to the automated test results to the customer terminal 100 and the operator terminal 150. Result files related to automated test results can be stored and managed in the configuration management unit 210.

The build server unit 250 performs automated testing to verify the system-on-chip design under the control of the automation unit 230. The build server unit 250 may compile the source code file updated in the customer terminal 100 and then perform automated testing through a simulation process.

In the build server unit 250, IP (Intellectual Property) blocks and simulation programs owned by the operator are installed, and in the customer terminal 100, system-on-chip design verification can be performed using the IP blocks and programs owned by the operator. You can.

In addition, the build server unit 250 is configured so that it cannot be directly accessed from the customer terminal 100, thereby increasing security stability and protecting IP blocks and licenses for various programs.

The build server unit 250 can also perform automated testing through a daily build performed according to a predetermined schedule or a manual build performed manually when necessary by a developer.

With this configuration, it is possible to provide customers with an automated system for system-on-chip design verification.

Figure 2 is a block diagram of the automation unit shown in Figure 1.

Referring to FIG. 2, the automation unit 230 may include a CI (Continuous Integration) tool module 231, a result provision module 233, a messenger module 255, and an issue management module 257. When these components are implemented in actual applications, two or more components may be combined into one component, or one component may be subdivided into two or more components, as needed.

The CI tool module 231 can be implemented with Jenkins, etc., and controls automated test performance. Jenkins is a tool that provides continuous integration services during software development. It enables continuous integration by uploading each developer's work to a repository in a shared area to prevent version conflicts when multiple developers develop one program.

The CI tool module 231 is a CI tool for operational use that can access the operational repository and perform overall regression testing, etc., and a customer-use CI tool that accesses the customer-use repository and performs unit tests, etc. Includes.

The CI tool module 231 is a commit build performed when the source code file for system-on-chip design verification is committed and pushed, a daily build performed according to a predetermined schedule, and a manual build performed manually when necessary by developers. You can set the build type, etc., and control its execution.

The result providing module 233 provides automated test results and can provide log files or waveform files as test results.

When a simulation fails or a specific event occurs, the messenger module 255 notifies the customer terminal 100 or the operator terminal 150 of the status.

The issue management module 257 manages issues that occur during the build and automation testing process. Issues include almost everything related to the system-on-chip design verification process, including errors, bugs, new features, and work requests. The issue management module 257 can be implemented using Jira, etc.

The automation unit 230 can be configured using such configuration modules.

Figure 3 is a flowchart provided to explain the source code file update process in the test automation method according to an embodiment of the present invention.

Referring to FIG. 3, you can run a web browser on the customer terminal 100, access the login screen provided to the test automation system 200, and enter the user name and password preset in the login screen to access the system. (S300).

When connecting to the system for the first time, a process of adding a Secure Shell (SSH) public key may be performed to work in the local environment of the customer terminal 100. SSH is an encryption network protocol for safely operating network services on insecure networks.

Once logged in and connected to the system, a screen listing all projects accessible to the customer is displayed. The user of the customer terminal 100 selects and replicates the desired project (S310).

Then, in the workspace of the customer's terminal 100, a source code editing process is performed to create a new source code file for system-on-chip design verification or to modify an existing source code file (S310).

When editing of the source code file is completed, a new commit is created and pushed to the repository 213 (S315, S320). During the push process after committing the source code, you can specify the target test branch. The system-on-chip design process can generally be verified by creating a test branch for each block before simulating the entire chip.

Since the customer terminal 100 and the operator terminal 150 can share a designated area in the storage 213, they can collaborate by connecting to a synchronized work space. Additionally, any source code file update to the repository 213 triggers a new build in the test automation system 200 so that automated testing is performed.

Through this process, the source code file for system-on-chip design verification can be updated in the storage 213.

Figure 4 is a flowchart provided to explain the automated test process in the test automation method according to an embodiment of the present invention.

Referring to Figure 4, first, a preparation process is performed (S400). The preparation process prepares the environment for a new build, and includes setting up a build description and user notifications.

Next, update the source code file to be built (S405).

Then, compilation is performed on the updated source code file (S410). Depending on the compilation, files in formats such as *.elf, *.bin, *.hex, etc. can be created from the newly updated source code file.

After compilation, the simulation is executed using the file generated as a result of compilation (S420). Simulations can be performed with a newly compiled test branch from a previously built Register Transfer Level (RTL) file.

When the build and simulation are completed, the simulation results are output to the customer terminal 100 (S425). When the build and simulation are completed, the completion of simulation can be notified to a designated terminal through a messenger program, etc.

If the simulation fails or a problem occurs during the automated testing process, the process of solving the problem may proceed through collaboration between the customer and the operator using the customer terminal 100 and the operator terminal 150.

Figures 5 and 6 show an example of a screen displayed during an automated test process.

Figure 5 shows an example of a project screen.

Referring to Figure 5, automated tests can be executed on a project basis, and on the project screen 500, the general description part 503 displays a general description of the project, and the stage view part 505 shows each step of each build record. It indicates the status, and step-level logs are supported.

The side panel portion 507 displays menu items that the user can select.

Here, selecting the 'Status' item displays the status of the project, and selecting the 'Changes' item displays changes in source code management. If you select the 'Build' item, a manual build is executed, and if you select the 'Full Stage View' item, the screen is displayed focusing on the stage view. And, if you select the 'Open Blue Ocean' item, the current project will be opened in blue ocean mode. Blue Ocean is one of the Jenkins plugins for graphical visualization and diagnosis of Jenkins pipeline projects.

Figure 6 shows an example of a project build history screen.

Referring to FIG. 6, the project build history screen 520 displays information about previous and current work.

In the project build history screen 520, the general description portion 523 explains general information about the build record. This includes the nodes used, the time it was started, and the cause of the operation.

The side panel portion 525 displays menu items that the user can select.

Here, selecting the 'Status' item shows the status of the build history, and selecting the 'Changes' item shows commit changes in Jenkins source code management. If you select the 'Console Output' item, the real-time log of the Jenkins project is displayed, and if you select the 'View Build Information' item, information about a specific job number is displayed. Selecting the 'See Fingerprints' item displays the fingerprint of the *.jar file created in Jenkins, which indicates when and why the file was created. If you select the 'Open Blue Ocean' item, the current page is displayed in blue ocean mode, which is useful for visualizing build history.

If you select the 'Restart from Stage' item, it will restart at a specific stage. Selecting the 'Reply' item will replay the current build with the same build configuration, but if the build configuration is not updated, this has no particular meaning. If you select the 'Pipeline Steps' item, the pipeline log for each pipeline step is displayed. If you select the 'Previous build' item, the previous build will be displayed. If there is no previous build, this item will not be displayed. Also, if you select the 'Next build' item, the next build will be displayed. If there is no next build, this item will not be displayed.

In addition, various menu screens can be provided during the automated testing process.

Meanwhile, the test automation system and method according to the present invention cannot be applied limited to the configuration of the embodiments described above, and the above embodiments are all or part of each embodiment so that various modifications can be made. It may also be configured by selective combination.

Additionally, the present invention can be implemented as a computer program on a programmable computer. These computers may include processors, storage, input devices, and output devices. To implement the content described in the present invention, program code can be input with a mouse or keyboard input device. These programs can be implemented in a high-level language or an object-oriented language. It can also be implemented as a computer system implemented in assembly or machine code.

The present invention can also be implemented as processor-readable code on a processor-readable recording medium. Processor-readable recording media includes all types of recording devices that store data that can be read by a processor. Examples of recording media that can be read by a processor include ROM, RAM, CD-ROM, and optical data storage devices. In addition, the processor-readable recording medium is distributed in a computer system connected to a network, so that the processor-readable code can be stored and executed in a distributed manner.

In addition, although preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the invention pertains without departing from the gist of the present invention as claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be understood individually from the technical idea or perspective of the present invention.

100: Customer terminal 150: Operator terminal
200: Test automation system 210: Configuration management department
230: automation unit 250: build server unit

Claims (15)

A configuration management unit including a storage for storing source code files for system-on-chip design verification that are committed and pushed through the project screen from a customer terminal on which an integrated development environment is installed and a project screen is displayed;
When the source code file is updated in the repository, an automation unit that controls automated testing on the source code file and includes a CI (Continuous Integration) tool module implemented in Jenkins to control the automated testing; and
An IP (Intellectual Property) block that can be used to verify the system-on-chip design, owned by an operator who operates the system for the automated test, is installed, and under the control of the automation unit, the automated test is performed using the IP block. A test automation system that performs and includes a build server unit that is not accessible from the customer terminal. delete According to paragraph 1,
A test automation system wherein the automation unit provides a result file related to the automated test results after completion of the automated test. According to paragraph 3,
The configuration management unit stores the result file and manages storage and sharing of the source code file. delete In paragraph 1
A test automation system wherein the build server unit performs the automated testing according to a preset schedule or manual request. According to paragraph 1,
A test automation system characterized in that the operator terminal of the operator can collaborate with the customer terminal by sharing the source code file stored in the repository. According to paragraph 1,
A test automation system characterized in that the storage is managed separately into a storage shared with the customer terminal and a storage accessible only through the operator's terminal. Committing the source code file for system-on-chip design testing through the project screen at the customer terminal where the integrated development environment is installed and displaying the project screen and pushing it to a repository provided in the configuration management unit;
When the source code file is updated in the repository, an automation unit including a CI (Continuous Integration) tool module implemented in Jenkins and controlling automated testing controls the automated testing on the source code file to be performed;
Providing a build server unit in which an IP (Intellectual Property) block that is owned by an operator who operates the system for the automated testing and can be used to verify the system-on-chip design is installed and that is not accessible from the customer terminal; and
A test automation method comprising performing the automated test using the IP block in the build server unit under control of the automation unit. According to clause 9,
A test automation method further comprising providing, in the automation unit, a result file related to the automated test result after completion of the automated test. According to clause 10,
A test automation method further comprising the step of storing the result file in the configuration management unit and controlling storage and sharing of the source code file. According to clause 9,
A test automation method further comprising the step of collaborating with the customer terminal by sharing the source code file stored in the repository at the terminal of the operator. According to clause 9,
A test automation method further comprising the step of the build server unit performing the automated test according to a preset schedule or manual request. According to clause 9,
A test automation method further comprising notifying the status to the customer terminal when the automated test fails or a specific event occurs. A processor-readable recording medium recording a program for executing the test automation method of any one of claims 9 to 14 on a processor.

Images