KR20240040654A - Systems and Methods for Testing CXL Enabled Devices in Parallel - Google Patents

Abstract

An efficient and effective test system and method are provided. In one embodiment, the test system includes a user interface configured to enable user interaction with the system, and a test board configured to be communicatively coupled to a plurality of devices under test (DUTs), where the DUTs are CXLs. (compute express link) protocol - includes a tester configured to direct testing of multiple DUTs, including managing flexible, independent parallel testing across multiple DUTs. Manage tests. In one example implementation, a tester independently creates and manages workloads for DUTs included in a plurality of DUTs. The DUTs may be memory devices, and the tester is configured to test different memory spaces in parallel. The different memory spaces may have various implementations (eg, included in multiple DUTs, different memory spaces within one of the DUTs included in the multiple DUTs, etc.). Workloads can be created and individually managed based on the individual characteristics of DUTs. Testing may include performance testing (e.g., bandwidth testing, latency testing, error testing, etc.).

Description

Systems and Methods for Testing CXL Enabled Devices in Parallel}

Related applications

This application claims the benefit and priority of the provisional application as follows:

No. 63/408,788 (Attorney Docket No. ATSY-0113-00.00US), entitled “CXL PROTOCOL ENABLEMENT FOR TEST ENVIRONMENT SYSTEMS and METHODS,” filed September 21, 2022;

No. 63/439,460 (Attorney Docket No. ATSY-0113-01.01US), entitled “CXL PROTOCOL ENABLEMENT FOR TEST ENVIRONMENT SYSTEMS and METHODS,” filed Jan. 17, 2022;

No. 63/408,795 (Attorney Docket No. ATSY-0112-00.00US), entitled “CXL PROTOCOL ENABLEMENT FOR TEST ENVIRONMENT SYSTEMS and METHODS,” filed September 21, 2022;

No. 63/439,434, filed January 17, 2022, entitled “MANAGEMENT OF HOT ADD IN A TESTING ENVIRONMENT FOR DUTS THAT ARE CXL PROTOCOL ENABLED” document number ATSY-0112-01.01US),

No. 63/408,801, filed September 21, 2022, entitled "SYSTEMS AND METHODS UTILIZING DAX MEMORY MANAGEMENT FOR TESTING CXL PROTOCOL ENABLED DEIVCES" document number ATSY-0115-00.00US),

63/439,470, filed January 17, 2022, entitled "SYSTEMS AND METHODS UTILIZING DAX MEMORY MANAGEMENT FOR TESTING CXL PROTOCOL ENABLED DEIVCES" document number ATSY-0115-01.01US),

No. 63/408,809, filed September 21, 2022, entitled “SYSTEMS AND METHODS FOR TESTING CXL ENABLED DEVICES IN PARALLEL” (Attorney Docket No. ATSY-0114-00.00) US), and

No. 63/439,464, filed Jan. 17, 2022, entitled “SYSTEMS AND METHODS FOR TESTING CXL ENABLED DEVICES IN PARALLEL” (Attorney Docket No. ATSY-0114-01.01) US).

All of which are incorporated herein by reference.

field of invention

The present disclosure relates to systems and methods configured to provide test capabilities for a device under test (DUT) that is compliant with the compute express link (CXL) protocol.

Since Compute Express Link (CXL) is a new communication (e.g., bus, etc.) protocol for the electronics industry (e.g., memory, etc.), there are many challenges that must be addressed before the CXL protocol can be used in the test environment of CXL-enabled DUTs. In particular, during normal non-test operation, CXL devices are often used to provide shared storage resources in system memory. Prior art Automatic Test Equipment (ATE) did not efficiently and effectively address these issues (e.g., CXL system memory, etc.), and traditional test systems and methods were associated with the testing of CXL-compatible devices under test (DUTs). Provided little or no testing capabilities.

The basic general purpose of a CXL Type 3 device is to serve as an additional shared main system memory resource, and this characteristic makes testing the device particularly problematic. Without the present invention, the CXL DUT's memory space could be considered a shared portion of the tester host system memory space. As a shared part of system memory, the CXL DUT can be accessed by various entities coupled to the system (e.g., devices, applications, processes, threads, etc.). In shared system memory mode, or "kernel memory" mode, one device coupled to the test system can typically write to and corrupt the memory space of another device. These results in a test environment can cause disruption to test operations and produce unreliable test results.

Additionally, there are several highly desirable test system characteristics that are often considered necessary for proper testing. However, traditional test systems and methods generally cannot realistically provide this capability. For example, it is highly desirable and often necessary for test systems and methods to be able to hot add/swap DUTs and test multiple DUTs in parallel. The shared access aspect of a CXL DUT makes it very difficult for the system to properly test multiple DUTs in parallel without interfering with the testing of other DUTs. Additionally, there are currently versions of CXL that do not have the ability to conveniently add/replace CXL devices to the system. Version 1.1 of the CXL protocol does not allow non-disruptive addition/replacement of devices while the device is running because device enumeration is only performed through the BIOS (requiring a system reboot). When the device first boots, the BIOS initializes it to operate as a CXL device. This effectively prevents non-disruptive additions of devices (e.g., during testing), as the host requires a reboot to recognize the new device. To add/replace a CXL device, traditionally the entire system would have to be shut down, a new CXL device would be added/replaced, and then the system would be restarted to start testing of the DUT from scratch. This prior art condition of having to start over again sacrifices/wastes a lot of time and resources to test other DUTs in the system (including, for example, loss of control data, etc.).

Because the CXL protocol is currently primarily designed for general computational use on non-test devices, and is not specifically intended for testing multiple CXL devices in parallel, many challenges have traditionally had to be addressed to allow for efficient and effective testing.

An efficient and effective test system and method are provided. In one embodiment, the test system includes a user interface configured to enable user interaction with the system, and a test board configured to be communicatively coupled to a plurality of devices under test (DUTs), where the DUTs are CXLs. (compute express link) protocol - includes a tester configured to direct testing of multiple DUTs, including managing flexible, independent parallel testing across multiple DUTs. Manage tests. In one example implementation, a tester independently creates and manages workloads for DUTs included in a plurality of DUTs. The DUTs may be memory devices, and the tester is configured to test different memory spaces in parallel. The different memory spaces may have various implementations (eg, included in multiple DUTs, different memory spaces within one of the DUTs included in the multiple DUTs, etc.). Workloads can be created and individually managed based on the individual characteristics of DUTs. Testing may include performance testing (e.g., bandwidth testing, latency testing, error testing, etc.).

The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention. Unless otherwise noted, drawings may not be drawn to scale.
1A is a block diagram of an example test system according to one embodiment.
1B is a block diagram of an example test system according to one embodiment.
Figure 2 is a block diagram of an example test method according to one embodiment.
Figure 3 is a block diagram of a non-disruptive additional test system according to one embodiment.
4 is a block diagram of an exemplary device non-disruptive additional testing method according to an embodiment.
Figure 5 is a block diagram of an example test system according to one embodiment.
Figure 6 is a block diagram of an example device test management method according to one embodiment.
Figure 7 is a block diagram of an example test system according to one embodiment.
Figure 8 is a block diagram of an example parallel testing method according to one embodiment.
9 is a block diagram of an example electronic system that can be used as a platform for implementing and controlling an exchange process, according to one embodiment.
Figure 10 is a block diagram of an example test system according to one embodiment.
Figure 11 is a block diagram of an example test system according to one embodiment.
Figure 12 is a block diagram of an example test system according to one embodiment.

Reference will now be made to various embodiments of the present disclosure, examples of which are shown in the accompanying drawings. Although described in connection with these embodiments, it will be understood that the disclosure is not intended to be limited to these embodiments. Rather, the present disclosure is intended to cover alternatives, modifications, and equivalents that may be included within the spirit and scope of the present disclosure. Additionally, in the following detailed description of the disclosure, numerous specific details are set forth to provide a thorough understanding of the disclosure. However, it will be understood that the present disclosure may be practiced without these specific details. In other instances, well-known methods, procedures, components and circuits have not been described in detail so as not to unnecessarily obscure aspects of the disclosure.

The accompanying drawings, which are incorporated in and form a part of this specification, are included for illustrative purposes of the principles of the invention and are not intended to limit the invention to the specific implementations set forth herein. Unless otherwise specified, drawings are not to scale.

The drawings are not necessarily drawn to scale and show not only portions of the devices and structures depicted, but also the various layers that form these structures. To simplify discussion and description, only one or two devices or structures may be described, but in practice, more devices or structures may be provided or formed. Additionally, although specific elements, components and layers are discussed, embodiments according to the invention are not limited to such elements, components and layers. For example, there may be other elements, components, layers, etc. than those discussed.

The novel systems and methods presented provide efficient and effective testing of CXL-compliant devices. In one embodiment, the system and method provide convenient test capabilities that address new and rapidly expanding needs associated with the technological paradigm shift to CXL protocol compliant memory DUTs. In one embodiment, the system and method enable efficient and cost-saving testing of CXL protocol-enabled DUTs. In one example implementation, the system and method cover devices using the new CXL protocol features on a DUT under test conditions.

It is recognized that there are many different new aspects and approaches to the systems and methods provided to address problematic issues associated with the testing of CXL protocol compliant devices. In one embodiment, the provided systems and methods enable efficient and effective implementation of CXL-related activities during DUT testing. In one example implementation, the provided systems and methods include hardware and software enhancements.

In one embodiment, the system and method cover aspects of creating and managing the environment necessary to implement testing of a DUT using the CXL protocol. It is recognized that a CXL DUT may include various types of devices. In one embodiment, the CXL DUT may include various types of memory (eg, DRAM, persistent memory, etc.) that are often used to provide storage resources for system memory under normal non-test operation. In one example implementation, the DUT is typically allocated to a separate memory range within the system memory space during normal operation.

1A is a block diagram of an example test method 10 according to one embodiment.

In block 1, the device detection process is performed. In one embodiment, the detection process includes determining whether the device is a CXL device and, if so, what the characteristics/characteristics of the CXL device are and to what portion/address of host CPU memory the CXL device is assigned. The device detection process may include enumeration. In one embodiment, the device detection process completely determines that the device is a CXL device and requires a reboot to complete/finalize the enumeration. In one example implementation, the device detection process is performed by the basic input/output system (BIOS).

In Block 2, the CXL device test organization process is performed. The CXL Device Test Organization process organizes and configures test systems to perform testing of CXL devices. In one embodiment, the CXL device test organization process includes isolating the CXL device from harmful interference from other components. In one example implementation, the device test organization process includes preventing other components from accessing portions of the memory range allocated to the CXL device (e.g., allocated in block 1 above). In one embodiment, the device test organization process includes retrieving device detection process information and establishing a CXL test reference or name for a CXL device based on test board coupling location (eg, DUT slot, etc.). In one example implementation, the reference or name is consistent or persistent while the application/thread process that created/instantiated it is running. In one example implementation, the device test organization process includes a device mapping process that maps CXL devices to test board mating location indicators (e.g., DUT slot identifiers, etc.). In one example implementation, user test specification requirements are mapped from the host CPU instruction set to available hardware. Portions of the memory range allocated to a CXL device may correspond to a name/identifier.

In Block 3, the CXL device test process is performed/instructed. In one embodiment, a test workload (eg, data patterns, instructions, etc.) is generated. Generation may include converting user-indicated functions into appropriate CXL device commands that achieve user desired functions. In one embodiment, the test process is a functional performance test (eg, bandwidth, latency, errors, etc.). In one embodiment, directing the CXL device test process includes generating a test workload. In one embodiment, the system and method do not allow harmful test system properties/features (eg, host CPU optimizations, etc.) to inadvertently/unintentionally corrupt test operation. In one embodiment, multiple independent parallel test processes are performed on multiple CXL DUTs.

In one embodiment, converting user-indicated functions into appropriate CXL device commands includes generating an appropriate number of read commands and write commands according to user-indicated preferences. In one example implementation, the user indicates a desired 50% read and 50% write. However, in practice the system also utilizes a read operation in response to a write command from the host. Accordingly, the systems and methods may be used to determine the correct ratio of actual reads and writes (e.g., 2/3 or 66% reads and 2/3 or 33%) to obtain a user requested ratio (e.g., 50% reads and 50% writes, etc.). records, etc.) automatically generates appropriate workload instructions.

In one embodiment, the CXL device test organization process includes a device test organization process non-disruptive add/replace feature.

In one embodiment, the device test organization process includes a flushing operation. In one example implementation, the memory locations associated with the isolation and mapping operations of block 2 are flushed when the corresponding CXL DUT is removed from the tester system. The flush operation may include flushing the corresponding cache entry. In one embodiment, the system and method ensure that the sequence of commands going to the CXL DUT is appropriate, and that the host CPU (e.g., as part of an optimization process, etc. ) Prevents issuing alternative commands.

In this explanation, it should be understood that the CXL device is generally the device under test (DUT). In one embodiment, the test system host CPU memory storage capacity can be expanded by a CXL device without the CXL device becoming a DUT, where the CXL device acts as a memory expansion for the host.

In one embodiment, the systems and methods include an API. In one example implementation, the systems and methods include three API groups. The first API group provides functions to automatically assist users when turning on and off the power of the CXL DUT in the test system. The second API group automatically handles the translation and management of user test requirements and control descriptions into appropriate CPU instruction set commands and CXL device instructions/instructions. The third API group includes debugging controls. In one example implementation, the API can communicate with the user through a convenient and intuitive user GUI.

Figure 1B is a block diagram of an example test system 100 according to one embodiment. The test system 100 includes a CXL DUT (121, 122, ..., 129) and a tester (110) compatible with CXL. The tester 110 includes a CXL host 111 that includes a processor (eg, CPU 112, etc.) and memory (eg, DRAM 113, etc.). In Figure 1, CXL DUT 121 is allocated to system memory range 131, CXL DUT 122 is allocated to system memory range 132, and CXL DUT 129 is allocated to system memory range 139. . This aspect of the CXL protocol can be particularly problematic during testing because the CXL DUT may be considered part of the host's (e.g. tester CPU, etc.) memory space. For example, without the new systems and methods provided, the CXL DUT could be accessed by a variety of entities (e.g., applications, threads, processes, etc.), thereby interfering with test operations (e.g., improperly ignored data, consistency issues, etc.). may cause and produce unreliable test results. In one example implementation, the provided test systems and methods prevent interference (e.g., data improperly overwritten by different entities, erroneous read operations, etc.).

In one embodiment, the novel systems and methods provided provide coordination and management of test accesses to different memory resources of a CXL DUT. It is recognized that there are many different approaches to coordinating and managing CXL tests (e.g., DAX interface, tracking DUT allocated memory ranges, etc.). In one example implementation, the provided systems and methods prevent undesirable interference with testing of specific DUTs. The new systems and methods provided can also be used to solve/resolve many other issues associated with the testing of CXL DUTs. In one device test mode embodiments, the systems and methods address issues related to aspects such as payload generation of a CXL protocol-supported DUT environment, device enumeration, support for non-disruptive addition/replacement of devices, and unique test environment constraints associated with the CXL protocol. Solve problems efficiently and effectively.

Figure 2 is a block diagram of an example test method 200 according to one embodiment.

In block 210, a process is performed to direct testing of a device under test (DUT), which includes CXL protocol communication with the DUT. In one embodiment, directing testing includes determining whether a particular device in the DUT is a CXL compliant device. Determining whether a particular device among the DUT is CXL-compliant can be performed as part of the enumeration process. In one example implementation, basic input/output system (BIOS) operation determines whether a particular device in the DUT is a CXL compatible device. Directing a test may include determining the characteristics of the DUT.

In block 220, test operations of the DUT are prevented from detrimentally interfering with each other. In one embodiment, preventing test operations of a DUT from detrimentally interfering with each other includes allocating separate memory ranges to the DUTs. In one example implementation, user configuration requirements are mapped to characteristics of the DUT. In one example implementation, cache access is tracked and adjusted according to user requirements.

Managed non-disruptive addition/replacement

The provided systems and methods provide user-friendly, efficient and effective managed hot add/swap capability for DUTs (also known as CXL devices) compatible with the CXL protocol, wherein the addition/swap capability of a specific DUT is provided. Replacement does not interfere with testing of other DUTs. The provided systems and methods also allow for non-disruptive addition/replacement of CXL protocol supporting devices during testing without loss of control data. In one embodiment, the system and method include modifications necessary to allow CXL-enabled DUTs to be added or replaced non-disruptively in a manner that is transparent to technicians performing test procedures without interfering with other DUT test operations. There are many reasons why devices may need to be replaced during testing. For example, being able to remove a DUT and insert another DUT (e.g., for binning purposes, when an error occurs, etc.) can increase overall test efficiency. In one embodiment, the system and method allow CXL DUTs to be non-disruptively added/replaced in a tester system without interrupting (or disrupting) the test operation of other CXL DUT devices. In one example implementation, addition/replacement of CXL DUT devices is performed without loss of associated control data. According to one embodiment, while a host device tester is testing multiple CXL-enabled DUTs in parallel, new DUTs can be added (e.g., "add-on-the-fly", etc.) and new DUTs can be added to the host device in a manner that is nearly transparent to the test technician. It will be automatically recognized by .

In one embodiment, the test technician interfaces with a graphical user interface (GUI) implemented on a host tester computer system. In one embodiment, upon need/demand for non-disruptive additions, the user/technician uses the GUI to temporarily trigger a test pause, then a new DUT is installed by the technician, and the technician interfaces with the GUI again to test it. A restart is triggered. All the technician needs to do is implement easy pause and easy start triggers (e.g., pressing a GUI button/icon, etc.) of testing for a specific DUT, and the system and method conveniently automatically directs other complex aspects of non-disruptive additions/replacements. .

Figure 3 is a block diagram of a non-disruptive additional test system 300 according to one embodiment. The test system 300 includes a DUT (321, 322) and a DUT (329) compatible with CXL and a tester (310). The tester 310 includes a CXL host 311 that includes a processor (eg, CPU 312, etc.) and memory (eg, DRAM 313, etc.). Tester 310 also includes an automatic non-disruptive addition/replacement management component 371 that is used to direct host addition/replacement of DUTs. Tester 310 may also be used to direct the implementation of GUI 391, which includes pause button 392 and resume button 393. 3, the technician presses the Pause GUI button for DUT 2 testing, inserts a new CXL device into the test board DUT 2 location, and then presses the Resume GUI button to disconnect the CXL device from the test board DUT 2 location/slot (e.g. 322, etc.) are added without interruption.

The system automatically performs several operations transparent to the user (e.g. mounting, enumeration, etc.), which saves the current test data set associated with all DUTs, then reboots the entire system (e.g. newly added DUTs are may be enumerated by BIOS, etc.), initiating communication between the DUT (e.g., including newly added DUTs, etc.) and the host. Then, test data synchronization also occurs automatically and transparently. After resetting and booting, testing continues. In one example implementation, reset and other initialization steps implemented to enable managed, non-disruptive additions are performed automatically and transparently to the technician by the novel system and method provided. The GUI (see, e.g., Figure 1, etc.) displays convenient on-screen instructions to “walk”/guide the technician through the few manual steps required to perform a non-disruptive addition. In one embodiment, the technician triggers a pause by pressing a button/icon on the GUI. Technicians can also similarly use the GUI to notify the system that a new DUT has been inserted and resume testing. In one example implementation, no test control data is lost when performing a managed non-disruptive addition, and the details required to implement the non-disruptive addition are advantageously hidden from the technician and performed automatically by the novel system and method provided. do.

Additionally, there are additional aspects of the systems and methods related to other improvements to the graphical user interface (GUI), including test flows, test programs, etc. In one example implementation, switching between the first and second flows can be conveniently triggered by simple interaction with the GUI, with the system automatically performing other operations associated with the implementation of the switch (e.g., 1 automatically directs retention of appropriate data from the flow, etc.).

In one embodiment, the system and method can quickly and efficiently address the technological paradigm shift to CXL protocol-compliant DUTs and respond to customer demands for non-disruptive addition/replacement test capabilities associated with new CXL-enabled DUTs.

Figure 4 is a block diagram of an example device non-disruptive addition test method 400 according to one embodiment. At block 410, instructions associated with the non-disruptive addition of a device under test (DUT) are received, where the test includes compute express link (CXL) protocol communication with the DUT, and the DUT is included in a plurality of devices under test (DUT). .

At block 420, a testing suspension process begins, wherein the test suspension process suspends testing of a plurality of DUTs, and the testing suspension process is automatically performed in response to receipt of an instruction associated with the non-disruptive addition of a DUT. . Initiating a test grace process may involve communication between a plurality of DUTs and a host. In one embodiment, initiating a test grace process includes saving a current test data set associated with a plurality of DUTs.

At block 430, instructions are received to seamlessly add the DUT to the test system and then resume testing.

At block 440, the test deferral process ends, where terminating the test deferral process is performed automatically in response to receiving an instruction to resume testing. In one embodiment, terminating the test grace process includes retrieving the stored test data sets associated with the plurality of DUTs at block 420. In one embodiment, ending the test grace process includes synchronizing test data, resetting, and rebooting the system to allow continued testing. In one example implementation, operations associated with the test grace process, including mounting and enumeration (eg, termination of the test grace process includes BIOS enumeration operations, etc.), are transparent to the user.

At block 450, testing of multiple DUTs resumes.

In one embodiment, non-disruptive addition of a DUT involves non-disruptive replacement with another DUT, where the other DUT is removed from the test system.

DAX

The new systems and methods presented provide DAX management and test isolation capabilities for individual DUTs that are compatible with the CXL protocol. In one embodiment, systems and methods relate to novel test operation management techniques for Linux Direct Access Device (DAX) related features. In parallel testing by a host tester, management between the host tester and multiple DUTs (DUTs support CXL) enables efficient and effective testing of CXL DUTs. The CXL DUT may include various types of memory (e.g., DRAM, persistent memory, non-persistent memory, etc.). Instead of sharing the CXL DUT's memory within the tester system's system memory space, memory test management technology advantageously provides protection against memory corruption between DUTs.

It is recognized that there are many different new aspects that leverage DAX capabilities to solve problematic issues associated with the new CXL protocol. In one embodiment, the provided systems and methods are directed to isolating the testing of a particular CXL-enabled DUT from undesirable interference and damage (e.g., improper writing to the DUT's memory, etc.). In one example implementation, the provided systems and methods also relate to other aspects of CXL device testing (e.g., workload generation for parallel operation, etc.). It is recognized that the provided systems and methods may focus on various aspects of testing (e.g., testing based on individual DUTs, management/coordination of parallel testing based on multiple DUTs/DUT systems, etc.).

During normal operation of one embodiment, each DUT is assigned a separate memory range within the system memory space. Figure 5 is a block diagram of an example test system 500 according to one embodiment. The test system 500 includes a DUT (521, 522) and a DUT (529) compatible with CXL and a tester (510). The tester 510 includes a CXL host 511 that includes a processor (eg, CPU 512, etc.) and memory (eg, DRAM 513, etc.). In one embodiment, CPU 512 directs the operation of DAX components 550, including DAX interface 551, DAX interface 552, and DAX interface 559. The DAX interface 551 manages interaction with the independent DAX memory range 571 allocated to the DUT 521. DAX interface 552 manages interaction with independent DAX memory ranges 572 allocated to DUT 522. The DAX interface 550 manages interaction with independent DAX memory ranges 579 allocated to the DUT 529. In one example implementation, CXL DUT 521 is allocated to system memory range 531, CXL DUT 522 is allocated to system memory range 532, and CXL DUT 129 is allocated to system memory range 539. ) is assigned to.

Without the provided systems and methods, during testing, a CXL-enabled DUT may be considered part of the tester host system shared memory space, which can disrupt test operations and produce unreliable test results. However, the use/exploitation of the DAX interface in the new test environment provided allows the presented test system and method to overcome the testing problems/issues of the prior art.

In one embodiment, DAX is a memory management interface technology where a host (e.g., a tester host, etc.) runs Linux and CXL devices (e.g., a DUT, etc.) reside in a separate “device space” of the Linux operating system. Therefore, the CXL DUT's memory resources are considered to reside in device space memory and not simply as an extension of the host tester's system memory. One advantage of using DAX for DUT memory management is that DAX allows the host to communicate with and perform tasks with each device individually. This protects the DUT's memory from external damage caused by any other device on the system (e.g., overwriting DUT memory, etc.). Therefore, while the DUT is managed using the DAX memory interface, the DUT memory is protected and no other device can write to the "shared" memory portion of the DUT.

That is, the DAX interface advantageously prevents other connected devices from writing to or corrupting the memory space of the connected DUT. Another advantage of the DAX memory interface is that it makes troubleshooting and debug much easier by directly managing the mapping between physical and virtual addresses and optionally making the virtual addresses the same as the physical addresses. Additionally, because the DUT memory is not directly mapped to OS system memory, the DAX memory interface prevents higher levels of system crashes when the DUT memory becomes bad (e.g. corruption that causes a silent crash). The provided systems and methods allow devices that will be part of shared system memory to be considered separate and tested individually.

Additionally, leveraging the DAX interface reduces the potential for other test delays/crashes associated with memory allocation. The presented systems and methods efficiently and effectively provide convenient CXL device test capabilities that respond to the rapidly growing demand for test capabilities associated with the technological paradigm shift to new CXL-enabled DUTs. The provided systems and methods do this in a way that avoids/minimizes the development and production costs associated with the complex management/coordination features required to implement testing of CXL-compliant DUTs.

Figure 6 is a block diagram of an example device test management method 600 according to one embodiment.

At block 610, a location-based persistent device representation process is performed on a compute express link (CXL) protocol compliant device under test (DUT). In one embodiment, performing a persistent device representation process includes allocating and tracking a CXL-compatible DUT, including characteristics of the CXL-compatible DUT, based on a reference indicator that is unique to the CXL-compatible DUT, where the reference indicator is a CXL-compatible DUT. Lasts as long as a compatible DUT is communicatively coupled to the test board. Test board location-based naming ensures that the tester can target a CXL-compatible DUT and persists as long as the CXL-compatible DUT is communicatively coupled to the load board. In one example implementation, the reference indicator is based on the test board location slot used to communicatively couple the CXL compatible DUT. Persistent device representation includes naming the DUT, where the name is automatically generated at the top of the LINUX enumeration.

At block 620, a device power up cleanup process is performed. In one embodiment, the power-up process includes waiting for the BIOS enumeration process without scanning the device.

At block 630, a device power down cleanup process is performed. In one embodiment, the power down process includes a flushing process for the central processing unit (CPU) and cache included in the host within the tester. In one example implementation, flushing includes flushing data from an address that was written in an operation associated with testing a CXL-compatible DUT, where the address is included in a range mapped to the CXL-compatible DUT.

parallel

The novel systems and methods presented provide systems and methods for testing CXL-enabled devices in parallel (also known as multiple device management). The new systems and methods provided provide and coordinate/manage complex independent workload generation and parallel test capabilities for multiple DUT/DUT systems compatible with the CXL protocol. The CXL protocol is a new protocol, and as such, there are many new challenges that must be addressed to use the CXL protocol in a test environment. Prior art did not address these issues efficiently and effectively. Especially during normal non-test operation, CXL compatible devices are often used to provide shared storage resources in system memory. This can be especially problematic when trying to test multiple CXL DUTs in parallel. Without the present invention, the memory space of a CXL-enabled DUT could be considered a shared portion of the tester host system memory space, and coordinating the testing of multiple CXL DUTs in parallel and managing each workload independently would typically be problematic. may cause memory corruption and produce unreliable test results.

The new systems and methods presented are directed to providing convenient CXL device test capabilities for new and rapidly expanding needs associated with the technological paradigm shift to CXL protocol-compliant DUTs. The novel systems and methods presented address an efficient and cost-saving ability to test multiple CXL-enabled devices in parallel. Additionally, multiple different memory spaces of the same CXL-enabled device can be tested in parallel.

The novel systems and methods presented relate to techniques developed to establish and increase independent parallel testing across multiple DUT/DUT systems, where the DUTs support the CXL protocol. In one embodiment, multiple CXL DUTs are tested in parallel by a host tester system, and workloads are created and managed independently for multiple CXL DUTs being tested in parallel. Additionally, different memory spaces of the same DUT may be tested in parallel. In one embodiment, the novel systems and methods provided provide maximum flexibility for testing DUTs in parallel and, with respect to this flexibility, push the testing of DUTs to their limits. Flexible, independent parallel test features provide fast and cost-effective CXL DUT test capabilities.

It is recognized that there are many different problematic issues associated with the new CXL protocol. The novel systems and methods presented relate to managing/coordinating parallel testing within the same DUT and across multiple DUT/DUT systems where independent workloads are applied to the DUTs. The novel systems and methods provided may also relate to other aspects of CXL device testing (e.g., isolating specific individual CXL DUTs from memory overwrites, etc.). It is recognized that the provided systems and methods may focus on various aspects of testing (e.g., testing based on individual DUTs, management/coordination of parallel testing based on multiple DUT/DUT systems, etc.).

Figure 7 is a block diagram of an example test system 700 according to one embodiment. The test system 700 includes a DUT (721, 722) and a DUT (729) compatible with CXL and a tester (710). The tester 710 includes a CXL host 711 that includes a processor (eg, CPU 712, etc.) and memory (eg, DRAM 713, etc.). In one embodiment, CPU 712 directs the operation of workload generation component 790 and DAX component 750. Workload generation component 790 generates workload instructions and includes workload interfaces 791, 792, and 791. The workload interface 791 manages workload testing of the DUT 721. Workload interface 792 and workload interface 793 manage workload testing of DUT 722. In one embodiment, workload interface 792 manages the workload for a first portion of memory in DUT 722 and workload interface 793 manages workload for a second portion of memory in DUT 722. Manage your workload. Workload interface 799 manages workload testing of DUT 729. In one example implementation, CXL DUT 721 is assigned to system memory range 731, CXL DUT 722 is assigned to system memory range 732, and CXL DUT 729 is assigned to system memory range 739. ) is assigned to.

In one embodiment, the workload defines how the tester tests the CXL DUT. In one example implementation, CXL support where multiple workloads defining different parameters (e.g., memory extent, read/write rate, randomness rate, number of loads and stores, etc.) are being tested in parallel by the same host tester system. Can be assigned independently and individually to the DUT. In one example implementation, multiple host threads can be set up in parallel (e.g., running on a test host), with each thread operating on a specific memory range of a different DUT or on the same DUT. It is recognized that memory ranges may be implemented in a variety of configurations (eg, memory ranges may exist in different DUTs, different ranges of a single DUT, etc.). This provides maximum flexibility in setting up test coverage and independent workloads for testing CXL DUTs.

In one embodiment, the DAX interface is used to manage the communication coupling of multiple CXL-enabled DUTs to a test system. In one example implementation, the CXL DUT's memory resources are considered to reside "individually" in a device space instantiation rather than simply a "shared" extension of the host tester's system memory. These configuration features can facilitate efficient and effective CXL DUT testing in parallel with independent workloads. In one example implementation, many CXL DUTs (e.g., 8, 18, etc.) may be combined into a single host tester system.

It is recognized that the provided systems and methods can be utilized/modified for use in a variety of parallel and independent test configuration scenarios. In one embodiment, different workloads may be applied in parallel to different DUTs, or different instances of the same workload may be applied in parallel to different DUTs. In one embodiment, multiple workloads may be assigned to a single DUT in parallel with each workload operating on a different range of DUT memory. Multiple instances of a similar workload definition can be used in parallel (e.g., to test multiple CXL DUTs, etc.). The DUT may include various types of memory (eg, DRAM, persistent memory, etc.).

Regarding increased parallelism, workloads can be generated and assigned to devices both individually and in parallel across multiple CXL DUTs, enabling a high level of flexibility for parallel individual/independent testing. Another advantage is that each CXL DUT can be individually assigned its own workload, allowing many tests to be queued in parallel.

The provided systems and methods can address the need for test capabilities related to new CXL-enabled DUTs. The flexible and independent parallel testing characteristics of the provided systems and methods provide an opportunity to address/satisfy user requests/desires for efficient and effective test resources in terms of performance and cost.

Figure 8 is a block diagram of an example parallel testing method 800 according to one embodiment.

Block 810 directs parallel testing of a device under test (DUT), where the test includes compute express link (CXL) protocol communication with the DUT, where the DUT is included in a plurality of DUTs. A DUT is one of multiple DUTs, and directing the test of a DUT involves independently managing the test workload. Testing is performed at the system level rather than component level testing.

In block 820, test operations of multiple DUTs are prevented from detrimentally interfering with each other. In one embodiment, testing includes performance testing. In one example implementation, testing is performed at user-directed read and write rates.

9 is a block diagram of an example electronic system 900 that can be used as a platform for implementing and controlling an exchange process according to one embodiment. Electronic system 900 may be a “server” computer system. Electronic system 900 includes central processor(s) 910, system memory 915, bulk memory 925 (e.g., hard drive, external memory, etc.), input/output (I/O) devices 930, Includes communication components/ports 940 and bus 950. Bus 950 connects other components (e.g., central processor(s) 910, system memory 915, bulk memory 925, input/output (I/O) devices 930, communication components/ It is configured to communicatively combine ports (940, etc.) and transfer information between them. Central processor(s) 910 is configured to process information and instructions. System memory 921 (e.g., read only memory (ROM), random access memory (RAM), etc.) and bulk memory(s) 925 are configured to store information and instructions for the central processor complex 915. I/O device(s) 930 may convey information to a system (e.g., central processor 910, memory 925, etc.). I/O devices 930 may include electronic systems (e.g., keyboards, buttons, joysticks, trackballs, audio transducers, microphones, touch-sensitive digitizer panels, eye scanners, display components, light-emitting diode (LED) displays, plasma display devices, etc. ) may be any suitable device for conveying information and/or commands to the device. The communication port 940 is configured to exchange/communicate information with an external device/network (not shown). Communication port 940 may have various configurations (e.g., limited RS-232 port, universal asynchronous receiver transmitter (UART), USB port, infrared transceiver, Ethernet port, IEEE 13394, synchronous port, etc.) , can communicate with external networks.

Figure 10 is a block diagram of an example test system 1000 according to one embodiment. Test system 1000 includes an electronics compartment 1010 and a test chamber 1090 having tester electronics 1020, a loadboard 1030, a DUT 1070, and a door 1091. Electronics compartment 1010 includes a controller 1011 and environmental components 1012. Enhanced loopback components (e.g., tester electronics 1020, etc.) may be inserted in place of the DUT for diagnostic analysis.

Figure 11 is a block diagram of an example test system 1100 according to one embodiment. It consists of a large controlled environment chamber or oven (71) containing an oven rack (10) and heating and cooling elements (11). The oven rack 10 includes a device under test (DUT) in a number of loadboard trays 31, 32, 33, 34, 41, 42, 43, and 44. The environmental test chamber 71 has a solid wall surrounding the test rack 15 and a solid door 72. Heating and cooling elements 11 may have a wide temperature range (eg, -10 to 120° C.). The tester or test head 81 includes various rack-type components including a system controller network switch 52, system power components 53, and tester slices 50 (the tester slices contain tester electronics). do. Loadboard trays (eg, 30, 31, etc.) are connected to tester slices 50 (multiple loadboard trays may be combined into a single tester slice). There is also a block diagram of the tester tray 30 and the device under test (e.g., 91, 92, etc.). The loadboard tray is manually filled with the devices under test. The entire tester tray (eg, 30, 31, etc.) is manually inserted into environmental chamber 71 and manually connected to tester electronics (eg, 50, 52, 53, etc.). This process can be labor intensive and cumbersome (e.g., it requires opening the door 72 of the environmental chamber 71 and attempting to manually insert the tray into the appropriate location through the door 72). An enhanced loopback component can be inserted in place of the DUT for diagnostic analysis of the tester electronics.

In one embodiment, the test system includes tester electronics and a device interface board that controls test operations. The tester electronics are located within an enclosure and may together be referred to as primitives. The device interface board has a device under test access interface that allows physical manipulation (eg, manual manipulation, robotic manipulation, etc.) of the device under test. A device under test can be physically operated independently with little or no disruption or effect on the test operation of other devices under test. The device interface board and its loadboard can be conveniently configured to accommodate different device form factors. In one embodiment, the loadboard consists of a device under test interface and a general-purpose primitive interface. In one example implementation, a device interface board may control the surrounding environment of the device under test.

Figure 12 is a block diagram of an example test system 1200 according to one embodiment. Test system 1200 includes test primitives 1290 (including, e.g., test control hardware and power components for a device under test) and a device interface board (DIB) disposed in front of and coupled to primitives 1290 ( 1210). In one embodiment, device interface board 1210 is a partial enclosure. The loadboard also couples to and electrically interfaces with primitive 1290 to obtain power and high-speed electrical signals for testing the device under test 1220. The device interface board may include airflow channels 1244 that allow airflow to and from the device under test environment. Air flow channels 1244 may include baffles. A partial enclosure of device interface board 1210 includes a device under test access interface 1270 that enables easy physical access (e.g., unobstructed, unobstructed, etc.) to the device under test. Environmental control components (not shown) control and maintain the environmental conditions surrounding the device under test (e.g., temperature, air flow rate, etc.). The environmental control component may create an environmental envelope that prevents or mitigates the interference of external environmental conditions with the operation of the device under test. (e.g. in primitive 1290, etc.) An enhanced loopback component can be inserted in place of the DUT for diagnostic analysis of the tester electronics.

Although the invention has been described in connection with preferred embodiments, it will be understood that the invention is not intended to be limited to these embodiments. Rather, the invention is intended to cover alternatives, modifications, and equivalents. It is not intended to be exhaustive or to limit the invention to the precise form disclosed, and obviously many modifications and variations are possible.

Some portions of the detailed description are presented in terms of procedures, logical blocks, processing, and other symbolic representations of operations on data bits within computer memory. These descriptions and representations are commonly used means by those skilled in the art of data processing to effectively convey the substance of their work to others skilled in the art. A procedure, logical block, process, etc. is considered here and generally as a coherent sequence of instructions or steps leading to a desired result. The steps involve physical manipulation of physical quantities. Typically, but not necessarily, these quantities take the form of electrical, magnetic, optical, or quantum signals that can be stored, transmitted, combined, compared, and otherwise manipulated in a computer system. Primarily for reasons of general usage, it has sometimes proven convenient to refer to these signals as bits, values, elements, symbols, letters, terms, numbers, etc.

However, it should be borne in mind that all of these and similar terms are associated with appropriate physical quantities and are merely convenient labels applied to such quantities. As will be clear from the discussion, and unless specifically stated otherwise, throughout this application any discussion utilizing terms such as “processing,” “computing,” “calculation,” “determination,” “display,” etc. refers to physical ( It is recognized that the term refers to the operations and processes of a computer system or similar processing device (e.g., an electrical, optical, or quantum computing device) that manipulates and transforms data expressed in quantities (e.g., electronically). This term refers to a processing device that manipulates or converts physical quantities within a component of a computer system (e.g., registers, memory, or other such information storage, transmission, or display devices, etc.) into other data similarly expressed by physical quantities within other components. Refers to actions and processes.

It is recognized that embodiments of the invention may be compatible with and implemented using a variety of different types of tangible memory or storage (e.g., RAM, DRAM, flash, hard drive, CD, DVD, etc.). Memory or storage can be considered a storage medium that can be changed or rewritten but is non-transitory. By referring to a non-transitory storage medium, we do not intend to limit the nature of the medium, and may include a variety of storage media (e.g., programmable, erasable, non-programmable, read/write, read-only, etc.), and may include "non-transitory" computer media. Readable media may include any computer-readable media except only transient propagated signals.

It is recognized that the description includes example concepts or embodiments associated with the new approach. It is also recognized that the list is not exhaustive and does not necessarily include all possible implementations. Concepts and embodiments may be implemented in hardware, firmware, software, etc. In one embodiment, a method or process describes operations performed by various processing components or units. In one example implementation, instructions or instructions associated with a method, process, operation, etc. may be stored in memory and cause a processor to implement the operation, function, action, etc.

The foregoing description of specific embodiments of the invention has been presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the invention to the precise form disclosed, and obviously many modifications and variations are possible in light of the foregoing teachings. The examples have been selected and described in order to best illustrate the principles of the invention and its practical application, thereby enabling those skilled in the art to best utilize the invention and its various embodiments with various modifications as appropriate to the particular application contemplated. The scope of the present invention is intended to be defined by the claims appended hereto and their equivalents. Listings of steps in a method claim do not imply any particular order in which those steps are performed, unless explicitly recited in the claim.

Claims (19)

As a test system,
a user interface configured to enable user interaction with the system;
A test board configured to be communicatively coupled to a plurality of devices under test (DUT), the DUTs being compatible with the CXL (compute express link) protocol,
A tester configured to direct testing of the plurality of DUTs, wherein the tester manages testing of the plurality of DUTs, including managing flexible and independent parallel testing across the plurality of DUTs.
test system.
According to paragraph 1,
The tester independently creates and manages workloads for the DUTs included in the plurality of DUTs,
test system.
According to paragraph 1,
The DUTs are memory devices, and the tester is configured to test different memory spaces in parallel, wherein the different memory spaces are included in the plurality of DUTs.
test system.
According to paragraph 1,
the DUTs are memory devices, and the tester is configured to test different memory spaces in parallel, the different memory spaces being within one of the DUTs included in the plurality of DUTs,
test system.
According to paragraph 1,
A workload is created and individually managed based on the individual characteristics of the DUTs,
test system.
According to paragraph 1,
The tests include performance tests,
test system.
According to paragraph 1,
The test includes a bandwidth test,
test system.
According to paragraph 1,
The test includes a latency test,
test system
According to paragraph 1,
The test includes error testing,
test system.
As a test method,
Instructing parallel testing of a device under test (DUT), wherein the test includes compute express link (CXL) protocol communication with the DUT, and the DUT is included in a plurality of DUTs,
Preventing test operations of the plurality of DUTs from detrimentally interfering with each other,
How to test.
According to clause 10,
The DUT is one of a plurality of DUTs, and directing testing of the DUT includes independently managing a test workload,
How to test.
According to clause 10,
The tests include performance tests,
How to test.
According to clause 10,
The test is performed according to the ratio of reading and recording indicated by the user,
How to test.
According to clause 10,
The test is performed at the system level rather than component level testing,
How to test
According to clause 10,
The DUT is a CXL type 3 memory expansion device,
How to test.
According to clause 10,
The instructions include testing of randomly generated address locations within the DUT,
How to test.
As a test system,
a user interface configured to enable user interaction with the system;
a test board configured to be communicatively coupled to a device under test (DUT), wherein the DUT is compatible with the compute express link (CXL) protocol;
A tester configured to direct testing of the DUT, wherein the tester manages testing of the DUT, wherein the tester is configured to initiate a plurality of test threads, the plurality of test threads corresponding to each of a plurality of ranges within the DUT. and wherein the tester is configured to manage testing for each of the plurality of ranges within the DUT in a flexible and independent parallel manner.
test system.
As a test system,
Where testing is based on generating independent workloads for each of multiple test threads,
test system.
As a test system,
If the test includes performance testing,
test system.

Images