WO2017053961A1 - Universal device testing system - Google Patents

Universal device testing system Download PDF

Info

Publication number
WO2017053961A1
WO2017053961A1 PCT/US2016/053768 US2016053768W WO2017053961A1 WO 2017053961 A1 WO2017053961 A1 WO 2017053961A1 US 2016053768 W US2016053768 W US 2016053768W WO 2017053961 A1 WO2017053961 A1 WO 2017053961A1
Authority
WO
WIPO (PCT)
Prior art keywords
testing
devices
test
interface
slot
Prior art date
Application number
PCT/US2016/053768
Other languages
French (fr)
Inventor
Samant Kumar
Dinesh Kumar
Shivashankar Diddimani
Gunjan Samaiya
Ina HUH
Jin Ryu
Rajeev Tiwari
Original Assignee
Contec, Llc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US14/866,780 external-priority patent/US9491454B1/en
Priority claimed from US14/866,630 external-priority patent/US9960989B2/en
Priority claimed from US14/866,720 external-priority patent/US9810735B2/en
Priority claimed from US14/866,752 external-priority patent/US10122611B2/en
Priority claimed from US14/948,143 external-priority patent/US9992084B2/en
Priority claimed from US14/948,925 external-priority patent/US9838295B2/en
Priority claimed from US14/987,538 external-priority patent/US9900116B2/en
Application filed by Contec, Llc filed Critical Contec, Llc
Publication of WO2017053961A1 publication Critical patent/WO2017053961A1/en
Priority to US15/642,915 priority Critical patent/US10291959B2/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/50—Testing arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/12—Network monitoring probes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N17/00—Diagnosis, testing or measuring for television systems or their details
    • H04N17/04—Diagnosis, testing or measuring for television systems or their details for receivers

Definitions

  • the present invention is directed to a system for testing devices.
  • FIG. 1 illustrates a high-level system architecture for testing devices, according to certain embodiments.
  • FIG. 2 illustrates some of the testing components and the interaction between the testing components, according to certain embodiments.
  • FIG. 3 illustrates a sample architecture that includes the testing components, according to certain embodiments.
  • FIG. 4 illustrates a set top box under test, according to certain embodiments.
  • FIG. 5 illustrates a wireless router under test, according to certain
  • FIG. 6 illustrates a cable modem/eMTA device under test, according to certain embodiments.
  • FIG. 7 illustrates a high-level operator dashboard for interacting with the core testing execution environment, according to certain embodiments.
  • FIG. 8 illustrates some components of a sample l-Frame, according to certain embodiments.
  • FIG. 9 illustrates a sample architecture showing bi-directional asynchronous communication between the operator dashboard, web-socket layer and core test execution machine, according to certain embodiments.
  • FIG. 10 illustrates a sample server side Node.JS layer, according to certain embodiments.
  • FIG. 1 1 is a high-level flow chart that illustrates some steps performed by a core testing executor/processor for testing devices, according to certain
  • FIG. 12 is a high-level schematic that illustrates device under test (DUT) probes through the use of virtualization containers, according to certain
  • FIG. 13 illustrates a high-level schematic of a test station for testing devices such as wireless devices, according to certain embodiments.
  • FIG. 14 illustrates requests for WiFi resource locks, according to certain embodiments.
  • FIG. 15 illustrates the response by the central resource control server to the requests from the slots in the test station, according to certain embodiments.
  • FIG. 16 illustrates the release of locks on frequency Channels for WiFi tests, according to certain embodiments.
  • FIG. 17 illustrates the granting of a lock on a frequency Channel to a given slot for performing a WiFi L3 test, according to certain embodiments.
  • FIG. 18 illustrates the release of a lock on a frequency Channel for a WiFi L3 test, according to certain embodiments.
  • FIG. 19 illustrates the requests from multiple slots for locks on frequency Channels in order to perform respective WiFi L3 tests, according to certain embodiments.
  • FIG. 20 illustrates that central resource control server will grant only one lock per frequency Channel for WiFi L3 tests, according to certain embodiments.
  • FIG. 21 illustrates the release of locks on a frequency Channels after completion of WiFi L3 tests, according to certain embodiments.
  • FIG. 22 illustrates the grant of a recently released lock on a frequency Channel to the next slot request in the queue, according to certain embodiments.
  • an innovative system can test a set of devices simultaneously. Further, such a testing system is capable of testing disparate devices simultaneously.
  • DUTs devices under test
  • Non-limiting examples of devices under test (DUTs) include set top boxes, cable modems, embedded multimedia terminal adapters, and wireless routers including broadband wireless routers for the home or for commercial networks.
  • such a testing system provides a separate set of interfaces to be tested for each device of the set of devices that is under testing. Further, such a system is designed to be adaptive by being extendable for testing new devices with corresponding new testing interfaces without fundamentally changing the core architecture of the testing system.
  • the testing system includes a core testing subsystem with a user interface and asynchronous communication among the system components such that new types of devices and new tests can be added and executed in a seamless fashion.
  • the user interface can communicate through web sockets with a universal tester. Such communication is in real-time, bidirectional and asynchronous so that the user can control and monitor the testing of multiple devices simultaneously and independently of each other using the same universal tester, and in certain embodiments, its associated test bench.
  • the testing system is capable of testing a set of similar types of devices or a set of disparate devices, wherein the plurality of devices are tested simultaneously by the testing system.
  • a testing solution system can be a three layer implementation. The number of layers may vary from implementation to implementation.
  • FIG. 1 illustrates a high-level system architecture for testing devices, according to certain embodiments.
  • FIG. 1 shows a test bench browser interface 102 that is in communication with a web-socket 104, that is, in turn, in communication with a core testing processor 106.
  • the test bench browser interface 102 that is in communication with a web-socket 104, that is, in turn, in communication with a core testing processor 106.
  • communication between the test bench browser 102, web-socket 104 and core testing processor 106 can be a TCP/IP communication.
  • the web browser is used as a user interface that communicates through web-sockets with the core testing processor.
  • communication may be in the form of JSON messages using TCP/IP protocol, according to certain
  • JSON Java script object notation for transmitting data between the server and web applications.
  • FIG. 2 illustrates some of the testing components and the interaction between the testing components, according to certain embodiments.
  • FIG. 2 shows a user interface 202, web-sockets 204, a core testing processor 206, database 208, test configuration modules 210, testing environment modules 212, a plurality of probes (214, 216, 218) to connect the devices under test (DUTs) to the core testing processor 206, and a speed test module 220, according to certain embodiments.
  • Speed testing is used for evaluating the performance of the WiFi and other media network connection and accessibility of the device under test.
  • FIG. 2 shows as non- limiting examples, a WiFi probe 214, an Ethernet local area network (LAN) probe 216 and a MoCA probe 218.
  • LAN local area network
  • various probes can be included such as a wireless local area network (WLAN) probe, an Ethernet wide area network (WAN) probe, a multimedia over coax alliance (MoCA) WAN probe, a MoCA LAN probe and a wireless probe via antenna.
  • WLAN wireless local area network
  • WAN Ethernet wide area network
  • MoCA multimedia over coax alliance
  • servers and other components in the testing system may be distributed over a plurality of computers.
  • core testing processor 206 loads and reads files from test configuration modules 210 and test environment modules 212 to initialize various components of the testing system.
  • the system notifies a user that is using the testing system to test one or more DUTs of the readiness of the testing system.
  • the user installs each DUT of the set of DUTs that are to be tested on the test bench and the serial number of each DUT is scanned.
  • the user installs each DUT in a separate Faraday cage (slot) in the test bench.
  • the core testing processor 206 receives the serial number information of each DUT and using the serial number, retrieves further information associated with each DUT based on the serial number from database 208, according to certain embodiments.
  • the core testing processor 206 dynamically loads test configuration information 210 and test environment information 212 based on device information such as make, model etc. of a given DUT. After the test configuration and test environment information are loaded, the core testing
  • processor 206 begins executing the various tests corresponding to each DUT so that the set of DUTs can be tested simultaneously.
  • Each test may correspond to underlying testing modules associated with WiFi, LAN, WAN or MoCA etc., interfaces of the DUT and such modules can be executed locally, remotely or at the device.
  • the test configuration information identifies the test modules and corresponding testing scripts that are to be executed by the core testing processor 206 at run time.
  • the core testing processor 206 also provides the test results and other feedback information to the user via the browser user interface 202 and web sockets 204. Further, the user can send user input and requests to the system through the browser user interface 202 and web sockets 204.
  • core testing processor 206 determines the success or failure of a given test based on the test configuration parameters and output results of the testing. Further, upon failure of a given test, core testing processor 206 may continue further testing or halt test execution based on test configuration parameters, according to certain embodiments.
  • a success message can be sent to the user via the browser user interface 202 and web sockets 204.
  • the user does not have to wait until the testing of all the DUTs in the set has been completed to begin installing other devices that need testing. Further, the testing of the devices need not be started at the same time. Soon after the testing is completed for a given DUT, the tested DUT may be uninstalled from its slot in the test bench and a new DUT can be installed in its slot so that testing can begin for the newly installed device.
  • test results can be stored locally and/or pushed to the cloud so that the results can be viewed remotely from any location. Further, the test results can be aggregated. According to certain embodiments,
  • aggregated data includes data combined from several data
  • Summary reports can be generated from such aggregated data.
  • Non-limiting examples of summary reports include charts and graphs that display information on all the DUTs or at least a subset of the DUTs.
  • the summary reports generated from the aggregated data can provide an overview of the testing information and characteristics of the DUTs.
  • the aggregated data can reveal trends and other related information associated with the DUTs.
  • the aggregated data can include user-level data, access account activity, etc.
  • the testing system includes a billing system to charge for the testing services for each device.
  • FIG. 3 illustrates a sample architecture that includes the testing components of a universal tester, according to certain embodiments.
  • FIG. 3 shows a browser user interface or operator dashboard 302, a test controller 304, a universal tester 306 and a DUT 308. There may be multiple devices under testing simultaneously but only one device under test is shown for convenience in FIG. 3.
  • browser user interface or operator dashboard 302 may include information 310 associated with each device under test.
  • the information 310 can include DUT serial number 31 1 , and testing progress information 312.
  • Browser user interface or operator dashboard 302 may also include user command function buttons 314 and drop down menus (not shown in FIG. 3).
  • the user can configure slot details (e.g., port numbers, IP address for the slot, etc.), configure testing preferences such as push to cloud, export to billing, etc.
  • test controller 304 may include a universal tester webserver 316 that is in communication (e.g., TCP/IP) with a universal tester database 318.
  • a billing process within the controller (not shown in FIG. 3) may be in communication with a billing service or application (not shown in FIG. 3).
  • database 318 can be a SQL database.
  • Database 318 can store information associated with each slot in the test bench.
  • database 318 can store for each slot, test details, test history, test logs, DUT information (e.g., DUT serial number, model name, etc.), testing
  • preferences/configuration user interface details/preferences/configuration, billing information, cloud push information, MSO/customer information (media subscriber organization), OEM (original equipment manufacturer) information, slot information, user information, and any persistent data needed by the universal device testing system for running tests.
  • MSO/customer information media subscriber organization
  • OEM original equipment manufacturer
  • universal tester 306 may include web sockets 320 that are in communication (e.g., TCP/IP) with browser user interface or operator dashboard 302 and core testing processor 324.
  • core testing processor 324 is in communication with test controller 304 (e.g., TCP/IP) and in communication (e.g., Telnet/SSH secure shell) with probes/containers (328, 330, 332, 334).
  • Core testing processor 324 is also in communication with configuration modules 322 (e.g., testing and environment configuration).
  • Non-limiting examples of probes include WiFi probe 328, LAN probe 330, MoCA probe 332 and WAN probe 334. There may be other types of probes including MoCA WAN probe, MoCA LAN probe and different types of wireless probes besides WiFi probes depending on the characteristics of the device being tested.
  • WiFi probe 328, LAN probe 330, MoCA probe 332 and WAN probe 334 communicate with the respective device under test through the relevant ports on the device such as WiFi port 336, LAN port 338, MoCA port 340 and WAN port 342.
  • Core testing processor 324 executes the relevant configured tests for the respective DUT. Status and test results can be sent to the user's dashboard (using JSON format messages as a non-limiting example) via the web-sockets.
  • the core testing executor/processor when executing a specific test for a given DUT, loads and reads test configuration information (for example from an XML structure) and identifies the relevant test script that needs to be executed. Inputs that are needed for executing the relevant test script are retrieved and supplied as inputs to the relevant test script.
  • test configuration information for example from an XML structure
  • Reset Password Operator scans password which is stored temporarily for use in the remainder of test until finished
  • Bandwidth must be greater than 60 Mbps on TCP (Reverse) or 70 Mbps on UDP (Forward)
  • Bandwidth must be greater than 700 Mbps
  • Bandwidth must be greater than 60 Mbps
  • Bandwidth must be greater than 60 Mbps
  • Bandwidth must be greater than 700 Mbps
  • the core testing executor/processor uses a reflection and command design pattern to invoke the relevant configured script(s) corresponding to each DUT being tested.
  • the command design pattern one or more of the following are encapsulated in an object: an object, method name, arguments.
  • the core testing processor uses the Python "reflection" capability to execute the relevant test scripts for a given DUT. The core testing processor is agnostic of the inner workings of the relevant test scripts for a given DUT.
  • lightweight software containers are provided.
  • virtualization containers are used to abstract the connection of probes to the different DUT interfaces in order to avoid conflicts.
  • virtualization containers are Linux containers.
  • Linux container LXC
  • LXC Linux container
  • lightweight virtualization containers are used to ensure isolation across servers.
  • virtualization containers resources can be isolated, services restricted, and processes provisioned to have an almost completely private view of the operating system with their own process ID space, file system structure, and network interfaces.
  • Multiple virtualization containers share the same kernel, but each virtualization container can be constrained to only use a defined amount of resources such as CPU, memory, network resources and I/O.
  • the relevant test script connects to the DUT interfaces directly or through the virtualization containers to execute the tests.
  • the core testing executor/processor receives the test results from running the relevant test scripts.
  • the core testing executor/processor can further process and interpret such results and can also send the results to the user's browser via web sockets.
  • the respective core testing executors/processors are in communication (e.g., Telnet/SSH secure shell) with the virtualization containers (there may be multiple virtualization containers).
  • the virtualization containers (probes) are in communication with corresponding DUT interfaces using Telnet/SSH/TCP/UDP/HTTP/HTTPS etc., as non-limiting examples.
  • a system for testing a plurality of devices comprises: a universal tester, at least one test controller, a plurality of sets of testing probes, and a plurality of web sockets, wherein: the universal tester is enabled for communication with a platform independent user interface through the plurality of web sockets; each respective set of testing probes of at least a subset of the plurality of sets of testing probes are for communicating with respective ports of a respective device under test of the plurality of devices installed in the universal tester for purposes of testing; and the plurality of web sockets enable real-time bi-directional and asynchronous communication between the platform independent user interface and the universal tester for simultaneously testing the plurality of devices under test by the universal tester.
  • the system for testing a plurality of devices further comprises a MoCA LAN bridge.
  • the system for testing a plurality of devices further comprises a MoCA WAN bridge.
  • the system for testing a plurality of devices further comprises a splitter.
  • the system for testing a plurality of devices further comprises a cable modem terminal system (CMTS).
  • CMTS cable modem terminal system
  • FIG. 4 illustrates a testing architecture for a set top box under test, according to certain embodiments.
  • multiple similar or disparate devices can be tested simultaneously and independently of each other using the same universal tester.
  • multiple set top boxes can be tested simultaneously and independently of each other using the same universal tester, along with other types of devices using the same universal tester.
  • FIG. 4 shows a universal tester 404 and set top box 402, which is the device under test for this specific case.
  • Universal tester 404 includes a plurality of virtualization containers (probes) for communicating with corresponding interfaces of set top box 402.
  • the core testing processor of the universal tester uses the HDMI (high definition multimedia interface) probe/container 406b to test the HDMI interface 406a of set top box 402.
  • audio/video probe/container 408b can be used to test the audio/video interface 408a of set top box 402.
  • Another audio/video probe/container 410b can be used to test the Coax TV Output interface 410a of set top box 402.
  • IR (infrared) probe/container 412b can be used to test the IR interface 412a of set top box 402.
  • CATV coax probe/container 416b can be used to test the Coax interface 416a of set top box 402.
  • the associated core testing processor executes the relevant configured tests for the set top box 402. Status and test results can be sent to the user's dashboard (using JSON format messages as a non-limiting example) via the web-sockets.
  • a system for testing a plurality of devices comprises: a universal tester, at least one test controller, a plurality of sets of testing probes, and a plurality of web sockets, wherein: the plurality of devices includes a plurality of set top boxes; the universal tester is enabled for communication with a platform independent user interface through the plurality of web sockets; the plurality of sets of testing probes comprising: at least one HDMI probe for testing a
  • the plurality of web sockets enable real-time bi-directional and asynchronous communication between the platform independent user interface and the universal tester for simultaneously testing the plurality of devices under test by the universal tester.
  • FIG. 5 illustrates a testing architecture 500 for a wireless router under test, according to certain embodiments.
  • multiple similar or disparate devices can be tested simultaneously and independently of each other using the same universal tester.
  • multiple wireless routers can be tested simultaneously and independently of each other using the same universal tester, along with other types of devices using the same universal tester.
  • FIG. 5 shows a universal tester 504 and a wireless router 502, which is the device under test for this specific case.
  • Non-limiting examples of wireless routers include broadband wireless routers for the home or for commercial networks.
  • Universal tester 504 includes a plurality of virtualization containers (probes) for communicating with corresponding interfaces of wireless router 502.
  • the core testing processor of the universal tester uses the LAN probes/containers 506b, 508b, 510b, 512b to test corresponding LAN interfaces 506a, 508a, 510a, 512a of wireless router 502.
  • WLAN (wireless LAN) probe/container 516b can be used to test WLAN interface 516a of wireless router 502.
  • Ethernet WAN probe/container 528b can be used to test WAN interface 528a of wireless router 502.
  • MoCA LAN probe/container 518b can be used to test Coax interface 532 of wireless router 502 via MoCA LAN bridge 518a and splitter 520.
  • MoCA WAN probe/container 530b can be used to test Coax interface 532 of wireless router 502 via MoCA WAN bridge 530a and splitter 520.
  • the associated core testing processor executes the relevant configured tests for the wireless router 502. Status and test results can be sent to the user's dashboard (using JSON format messages as a non-limiting example) via the web-sockets.
  • the core testing executor/processor when executing a specific test for a given DUT, loads and reads test configuration information (for example from an XML structure) and identifies the relevant test script that needs to be executed. Inputs that are needed for executing the relevant test script are retrieved and supplied as inputs to the relevant test script.
  • test configuration information for example from an XML structure
  • Bandwidth must be greater than 60 Mbps on TCP (Reverse) or 70 Mbps on UDP (Forward)
  • threshold e.g., -50dB
  • Bandwidth must be greater than threshold (e.g., 700 Mbps)
  • All Rates must be greater than threshold (e.g., 180 Mbps)
  • Bandwidth must be greater than threshold (e.g., 60 Mbps)
  • Bandwidth must be greater than threshold (e.g., 60 Mbps)
  • Bandwidth must be greater than threshold (e.g., 60 Mbps)
  • Iperf3 Speed Test 5 seconds Reverse and Forward, (e.g., Sending 1200 Mbps Bandwidth)
  • Bandwidth must be greater than threshold (e.g., 700 Mbps)
  • a system for testing a plurality of devices comprises: a universal tester, at least one test controller, a plurality of sets of testing probes, and a plurality of web sockets, wherein: the plurality of devices includes a plurality of wireless routers; the universal tester is enabled for communication with a platform independent user interface through the plurality of web sockets; the plurality of sets of testing probes comprising: a plurality of LAN probes for testing
  • corresponding LAN interfaces of a wireless router of the plurality of wireless routers at least one WLAN probe for testing a corresponding WLAN interface of the wireless router of the plurality of wireless routers, at least one Ethernet WAN probe for testing a corresponding WAN interface of the wireless router of the plurality of wireless routers, at least one MoCA LAN probe for testing a corresponding coax interface of the wireless router of the plurality of wireless routers, at least one MoCA WAN probe for testing a corresponding coax interface of the wireless router of the plurality of wireless routers; and the plurality of web sockets enable real-time bi-directional and asynchronous communication between the platform independent user interface and the universal tester for simultaneously testing the plurality of devices under test by the universal tester.
  • FIG. 6 illustrates a testing architecture 600 for a cable modem/eMTA under test, according to certain embodiments.
  • multiple similar or disparate devices can be tested simultaneously and independently of each other using the same universal tester.
  • multiple cable modem s/eMTAs can be tested simultaneously and independently of each other using the same universal tester, along with other types of devices using the same universal tester.
  • FIG. 6 shows a universal tester 604 and cable modem/ eMTA (embedded multimedia terminal adapter) 602, which is the device under test for this specific case.
  • Universal tester 604 includes a plurality of virtualization containers (probes) for communicating with corresponding interfaces of cable modem/eMTA 602.
  • the core testing processor of the universal tester uses the LAN probes/containers 606b, 608b, 610b, 612b to test corresponding LAN interfaces 606a, 608a, 610a, 612a of cable modem/eMTA 602.
  • FXS (foreign exchange station) probe/container 614b can be used to test the FXS interface 614a of cable modem/ eMTA 602.
  • WLAN (wireless LAN) probe/container 616b can be used to test WLAN interface 616a of cable modem/eMTA 602.
  • MoCA LAN probe/container 618b can be used to test MOCA LAN interface of cable modem/ eMTA 602 via MoCA LAN bridge 618a and splitter 620.
  • NCS network based call signaling protocol specification
  • IMS IP multimedia subsystem
  • Provision probe/container 626 can be used to test DOCSIS WAN/MOCA LAN interface 630 of cable modem/eMTA 602 via CMTS 622 and splitter 620.
  • WAN probe/container 628 can be used to test DOCSIS WAN/MOCA LAN interface 630 of cable modem/eMTA 602 via CMTS 622 and splitter 620.
  • the associated core testing processor executes the relevant configured tests for the cable modem/eMTA 602. Status and test results can be sent to the user's dashboard (using JSON format messages as a non-limiting example) via the web-sockets.
  • the core testing executor/processor when executing a specific test for a given DUT, loads and reads test configuration information (for example from an XML structure) and identifies the relevant test script that needs to be executed. Inputs that are needed for executing the relevant test script are retrieved and supplied as inputs to the relevant test script.
  • test configuration information for example from an XML structure
  • Lock waiting timeout 1800 sec L3 Locks must be run one at a time and when no L2 Lock is running
  • Bandwidth must be greater than 60 Mbps on TCP (Reverse) or 70 Mbps on UDP (Forward)
  • Bandwidth must be greater than threshold (e.g., 700 Mbps) Threshold may vary depending on the model
  • Iperf3 Speed Test 5 seconds Reverse and Forward, (e.g., Sending 240
  • Bandwidth must be greater than threshold (e.g., 60 Mbps) Threshold values may vary depending on the model
  • Iperf3 Speed Test 5 seconds Reverse and Forward, (e.g., Sending 1200 Mbps Bandwidth)
  • Bandwidth must be greater than threshold (e.g., 700 Mbps) Threshold values may vary depending on the model
  • a system for testing a plurality of devices comprises: a universal tester, at least one test controller, a plurality of sets of testing probes, and a plurality of web sockets, wherein: the plurality of devices includes a plurality of cable modems/eMTA devices; the universal tester is enabled for communication with a platform independent user interface through the plurality of web sockets; the plurality of sets of testing probes comprising: a plurality of LAN probes for testing corresponding LAN interfaces of a cable modem/eMTA device of the plurality of cable modem/eMTA devices, at least one FXS probe for testing a corresponding FXS interface of the cable modem/eMTA device of the plurality of cable modem/eMTA devices, at least one WLAN probe for testing a corresponding WLAN interface of the cable modem/eMTA device of the plurality of cable
  • modem/eMTA devices at least one MoCA LAN probe for testing a corresponding MoCA LAN interface of the cable modem/eMTA device of the plurality of cable modem/eMTA devices, at least one NCS/IMS probe for testing a corresponding DOCSIS WAN interface of the cable modem/eMTA device of the plurality of cable modem/eMTA devices, at least one provision probe for testing the corresponding DOCSIS WAN interface of the cable modem/eMTA device of the plurality of cable modem/eMTA devices, at least one WAN probe for testing the corresponding
  • DOCSIS WAN interface of the cable modem/eMTA device of the plurality of cable modem/eMTA devices; and the plurality of web sockets enable real-time bidirectional and asynchronous communication between the platform independent user interface and the universal tester for simultaneously testing the plurality of devices under test by the universal tester.
  • the testing system provides a separate set of interfaces to be tested for each device that is under testing of the set of devices.
  • FIG. 7 illustrates a high-level operator dashboard for interacting with the core testing execution environment, according to certain embodiments.
  • FIG. 7 shows an operator dashboard 702 (user interface) that is capable of asynchronous
  • Operator dashboard 702 includes a plurality of l-Frames 704a-p in HTML (inline frames).
  • the HTML inline frame element represents a nested browsing context, effectively embedding another HTML page into the current page.
  • Each I- Frame corresponds to a slot in the test bench for testing the devices.
  • a device that is to be tested (device under test or DUT) is installed in a slot in the test bench.
  • different types of devices can be installed in the slots in the test bench for simultaneous testing.
  • the slots in the test bench are not restricted to testing same types of devices.
  • Disparate devices can be tested simultaneously in the test bench. The respective tests associated with each slot do not interfere with tests running in other slots in the test bench.
  • Non-limiting examples of devices under test (DUTs) include set top boxes, cable modems, embedded multimedia terminal adapters, and wireless routers including broadband wireless routers for the home or for commercial networks.
  • operator dashboard 702 may be implemented as a neutral platform such as a web-based browser. Such a web-based browser type of operator dashboard can offer flexible access to a user that is at the same location as the test bench or from a laptop, mobile phone, tablet, etc., that is remote from the test bench.
  • the HTML based l-Frames in operator dashboard 702 allow a user to send commands and interact with the core testing execution machine with respect to each DUT and independently of other DUTs installed in the test bench such that the user can run tests for all the installed DUTs simultaneously. Further, the user can control and monitor the tests for all the installed DUTs simultaneously using the l-Frames of operator dashboard 702.
  • the user can configure slot details (e.g., port numbers, IP address for the slot, etc.), configure testing preferences such as push to cloud, export to billing, etc.
  • slot details e.g., port numbers, IP address for the slot, etc.
  • testing preferences such as push to cloud, export to billing, etc.
  • the l-Frames provide the requisite isolation for executing the tests of each of the DUTs in the test bench simultaneously but independently of each other. In other words, the DUTS installed in the test bench can all be tested in parallel without conflicting with each other.
  • FIG. 8 illustrates some components of a sample l-Frame, according to certain embodiments.
  • FIG. 8 shows an l-Frame 802 that includes HTML 804, Java script 806 and a client side web-Socket.10 808.
  • each l-Frame in the operator dashboard (user interface) is mapped to one of the slots in the test bench which is completely different from the run-of-the-mill client-server (web) architecture.
  • the run-of-the-mill client-server (web) architecture the user makes a request and a corresponding HTML output is served up to the user's browser.
  • the operator dashboard with a plurality of l-Frames can provide real-time continuous feedback to the user for each DUT once the user initiates test execution for the DUTs.
  • the user can use a respective l-Frame to receive feedback such as testing progress and testing results associated with a specific DUT of the plurality of DUTs undergoing parallel testing on the test bench.
  • the user can also interact with the core testing execution machine using the operator dashboard that includes a plurality of l-Frames.
  • the user/test operator might need to provide feedback to the core testing execution machine such as scanning in passwords, providing feedback on certain conditions associated with the test bench and/or core testing machine.
  • the feedback can include information needed for the testing procedure such as factory reset information, cage closed confirmation, Wi-Fi Protected Setup (WPS) LED confirmation, USB LED confirmation, LAN Coax LED confirmation, MoCA WAN LED confirmation, etc.
  • WPS Wi-Fi Protected Setup
  • USB LED confirmation USB LED confirmation
  • LAN Coax LED confirmation LAN Coax LED confirmation
  • MoCA WAN LED confirmation etc.
  • the core testing machine comprises multiple slots (at the test bench) for installing a DUT in each slot.
  • each DUT in a respective slot is associated with its respective lightweight virtualization container (probes abstraction) and core testing executor/processor.
  • the core testing machine may comprise N core testing servers and each of the N core testing servers may be associated with M core testing
  • the core testing machine need not have every slot installed with a DUT in order to begin running the tests.
  • the slots are used as needed.
  • the testing of a given DUT can start and finish independently of the other DUTs installed in the test bench of the core testing machine.
  • the use of DUT testing interfaces (probes) through software containers (virtualization containers) can avoid network conflicts while testing multiple DUTs simultaneously by the core testing machine.
  • FIG. 9 illustrates a sample architecture showing bi-directional asynchronous communication between the operator dashboard, web-socket layer and core test execution machine, according to certain embodiments.
  • FIG. 9 shows operator dashboard 902 (user interface) including a plurality of l-Frames (904a-d, 910a-d, 916a-d, 922a-d), a plurality of web-sockets (906a-d, 912a-d, 918a-d, 924a-d), and a plurality of test execution environments (908a-d, 914a-d, 920a-d, 926a-d).
  • each l-Frame can communicate asynchronously with a corresponding test testing executor environment.
  • the asynchronous communication can be achieved because the JavaScript Socket. IO on the client side browser dashboard communicates bi-directionally with corresponding web socket (Socket. IO) server-side implementation in Node.JS.
  • each l-Frame (904a-d, 910a- d, 916a-d, 922a-d) can bi-directionally interact with its corresponding web socket (906a-d, 912a-d, 918a-d, 924a-d) server-side implementation.
  • Each web socket (906a-d, 912a-d, 918a-d, 924a-d) can in turn interact bi-directionally with its corresponding test execution environment (908a-d, 914a-d, 920a-d, 926a-d).
  • the communication between the l-Frames and the web socket (Socket. IO) server-side uses TCP/IP protocol.
  • the communication between the web sockets (Socket. IO) server-side implementation in Node.JS and the corresponding testing executor environments uses TCP/IP protocol.
  • the core test execution environment maintains the current state of the device testing execution and upon communication reconnection, pushes the state information to the browser implemented l-Frames of the operator dashboard.
  • the foregoing feature allows users to refresh or restart their browser at any time without resulting in loss-of-state. Such a feature also allows the user to stop or abort a given test for a corresponding DUT in the test bench. Such a feature further allows a user to monitor the progress of one or more tests simultaneously using different browser sessions. Such browser sessions can be opened on the same device or on different devices.
  • a browser that supports CSS (cascade style sheets), JavaScript, JQuery (or other suitable cross-platform JavaScript library designed to simplify the client-side scripting of HTML), and client side Socket. IO is used.
  • FIG. 10 illustrates a sample server side Node.JS layer, according to certain embodiments.
  • FIG. 4 shows user interface middleware 400 that includes a server side Node.JS layer 1002 and a Socket. IO 1004 (web socket layer). Socket. IO 404 is one implementation of the web socket protocol. As previously described,
  • such a web-socket layer can be implemented as a Socket. IO server hosted in Node.JS environment.
  • Socket. IO server is an event-driven server.
  • the web socket layer can perform the following: • Enables real-time asynchronous bi-directional communication.
  • Probes test the following interfaces on the DUT (when such interfaces are available on the DUT):
  • Ethernet Local Area Network assigned probe runs Ethernet-based connection tests
  • Ethernet Wide Area Network assigned probe runs Ethernet-based connection tests
  • MoCA connection establishes connection, and runs MoCA-related connection tests
  • MoCA WAN assigned probe sets up MoCA connection, establishes
  • Wireless 2.4 GHz assigned probe sets up wireless connection
  • Wireless 5.0 GHz assigned probe sets up wireless connection
  • a user interface for a testing machine comprises: a plurality of l-Frames, wherein each l-Frame of at least a subset of the plurality of l-Frames is associated with a respective slot of a plurality of slots on the testing machine for installing, in the respective slot, a respective device under test (DUT) of a plurality of DUTs; and a plurality of client side web sockets associated with the plurality of l-Frames, wherein each client side web socket of at least a subset of the plurality of client side web sockets communicates with a corresponding web socket in a middleware web socket layer for achieving isolation and
  • DUT device under test
  • the middleware web socket layer enables real-time asynchronous communication between the user interface and a core testing environment of the testing machine.
  • the middleware web socket layer enables real-time bi-directional communication between the user interface and a core testing environment of the testing machine.
  • the client side web sockets communicate with the middleware web socket layer using TCP/IP communication.
  • the middleware web socket layer communicates with a core testing environment of the testing machine using TCP/IP communication.
  • the middleware web socket layer can be a cloud based implementation.
  • a core testing machine comprises multiple slots for installing a DUT in each slot.
  • each DUT in a respective slot is associated with its respective lightweight virtualization containers (probes abstraction) and core testing executor/processor.
  • the core testing machine may comprise N core testing servers and each of the N core testing servers may be associated with M core testing executors/processors.
  • the core testing machine need not have every slot installed with a DUT in order to begin running the tests. The slots are used as needed. Further, the testing of a given DUT can start and finish independently of the other DUTs installed in the test bench of the core testing machine.
  • FIG. 1 1 a high-level flow chart that illustrates some steps performed by a core testing executor/processor for testing devices, according to certain embodiments.
  • the core testing executor/processor is associated with a server.
  • the core testing executor is special processor.
  • a user installs one or more devices to be tested into test bench of the core testing machine for testing the devices.
  • the user scans the barcode (or other identification) of each device to be tested.
  • a device that is to be tested using the core testing executor/processor is also referred to as "DUT" herein. Each DUT is then submitted for testing at block 1 104.
  • FIG. 1 1 will be described with respect to a single DUT even though the core testing machine is capable of testing multiple DUTs
  • the respective core testing executor/processor receives a corresponding serial number information and validates the corresponding DUT at block 1 106.
  • the core testing executor/processor retrieves device information such as make, model, customer, etc. of a given DUT based on the serial number information from a database or web service, for example.
  • the core testing executor/processor loads the specific test configuration information corresponding to the given DUT.
  • Each DUT type (based on make, model, etc.) may be associated with different test configuration information.
  • Each test configuration information includes a set of testing steps.
  • the core testing executor/processor begins to read a testing step of the test configuration information for a given DUT.
  • the core testing executor/processor executes the test step that was read at block 1 1 12.
  • the core testing executor/processor records the result of the executed test step.
  • the core testing executor/processor determines whether the DUT passed or failed the executed test step. If the DUT failed the executed test step at block 1 1 18, then the core testing executor/processor determines whether to abort the testing procedure (based on the test configuration) at block 1 120. If the core testing executor/processor determines to abort the testing procedure at block 1 120, then the DUT is deemed to have failed the test step at block 1 122.
  • a message of the completion and/or the results of the test step is sent to the user's browser via web-sockets in real-time, for example.
  • the user interface can show test progress in addition to testing information such as port numbers, IP address for each DUT slot, DUT serial number, and testing preferences related to billing and pushing to the cloud, etc.
  • the use can also provide input associated with a given test (e.g., password).
  • the user via user interface) can communicate with the core testing executor/processor using asynchronous feedback and interaction.
  • communication may be in the form of JSON messages using TCP/IP protocol, according to certain embodiments.
  • JSON is Java script object notation for transmitting data between the server and web applications.
  • the core testing machine can test a set of similar types of devices or disparate types of devices simultaneously using a separate set of interfaces for each device that is under test testing.
  • Each N core testing server may comprise M number of core testing executors/processors.
  • a total of N multiplied by M number of DUTS can be tested simultaneously (one DUT is each of the N X M slots, for example).
  • the use of DUT testing interfaces (probes) through virtualization containers can avoid network conflicts while testing multiple DUTs simultaneously by the core testing machine.
  • the core testing servers and core testing executors/processors (and other components) in the testing system may be distributed over a plurality of computers.
  • FIG. 12 is a high-level schematic that illustrates DUT probes through the use of software containers (virtualization containers), according to certain embodiments.
  • FIG. 12 shows core test executor/processor 1202, slot 1204, and DUTs 1206 and 1208. There may be more than two DUTs but only two of them are shown in FIG. 12 for convenience.
  • Slot 1204 includes as non-limiting examples, Ethernet wide area network (WAN) probes 1210a, 1216a, Ethernet local area network (LAN) probes 1212a, 1218a and a multimedia over coax alliance (MoCA) probes 1214a, 1220a (MoCA probes can be WAN or LAN, for example).
  • WAN wide area network
  • LAN Ethernet local area network
  • MoCA multimedia over coax alliance
  • Slot 1204 are connected to the interfaces of DUT 1206 and DUT 1208 respectively.
  • WAN probe 1210a is connected to WAN port 1210b of DUT 1206.
  • LAN probe 1212a is connected to LAN port 1212b of DUT 1206.
  • MoCA probe 1214a is connected to MoCA port 1214b of DUT 1206.
  • WAN probe 1216a is connected to WAN port 1216b of DUT 1208.
  • LAN probe 1218a is connected to LAN port 1218b of DUT 1208.
  • MoCA probe 1220a is connected to MoCA port 1220b of DUT 1208.
  • Probes test the following interfaces on the DUT (when such interfaces are available on the DUT):
  • Ethernet Local Area Network assigned probe runs Ethernet-based connection and speed tests
  • Ethernet Wide Area Network assigned probe runs Ethernet-based connection and speed tests
  • MoCA connection establishes connection, and runs MoCA-related connection and speed tests
  • MoCA WAN assigned probe sets up MoCA connection, establishes
  • Wireless 2.4 GHz assigned probe sets up wireless connection
  • Wireless 5.0 GHz assigned probe sets up wireless connection
  • a system for testing device comprises: a testing machine with a plurality of slots, wherein each slot of the plurality of slots is for installing a device-under-test (DUT) of a plurality of DUTs; a plurality of core testing processors, wherein each core testing processor of the plurality of core testing processors is associated with a respective slot of the plurality of slots; a plurality of lightweight virtualization containers, where a respective lightweight virtualization container of the plurality of lightweight virtualization containers is associated with one of the slots that might have DUT installed, wherein the plurality of lightweight virtualization containers enable isolation of respective testing processes and testing resources associated with each respective device-under-test.
  • DUT device-under-test
  • the plurality of lightweight virtualization containers comprise testing probes for testing a respective DUT of the plurality of DUTs.
  • Virtualization containers can also be referred to as probes herein.
  • the plurality of lightweight virtualization containers are used for testing one or more DUT interfaces at the DUT comprising: Ethernet Local Area Network (LAN) interface; Ethernet Wide Area Network (WAN) interface; Multimedia over Coax Alliance (MoCA) LAN interface; Multimedia over Coax Alliance (MoCA) WAN interface; Wireless 2.4 GHz interface; Wireless 5.0 GHz interface; Phone ports (FXS) interface; USB interface; video interface; and audio interface.
  • LAN Local Area Network
  • WAN Wide Area Network
  • MoCA Multimedia over Coax Alliance
  • MoCA Multimedia over Coax Alliance
  • FXS Phone ports
  • each core testing processor of at least a subset of the plurality of core testing processors is associated with a respective web socket for communication that is isolated and independent of communication associated with other core testing processors of the plurality of core testing processors.
  • a respective core testing processor of the plurality of core testing processors communicates with a user interface.
  • a respective core testing processor of the plurality of core testing processors communicates using asynchronous feedback and interaction.
  • a respective core testing processor of the plurality of core testing processors communicates using JSON messages.
  • the respective core testing processor of the plurality of core testing processors communicates using TCP/IP protocol.
  • the respective core testing processor of the plurality of core testing processors retrieves at run time a respective test configuration corresponding to the DUT installed in the respective slot associated with respective core testing processor; loads the set of tests associated with the DUT installed in the respective slot associated with respective core testing processor; and executes the loaded set of tests.
  • a central resource control server controls resources such as locks on WiFi resources (herein also referred to as "WiFi resource lock").
  • WiFi resource locks include locks on resources for WiFi L2 (Layer 2) tests, WiFi L3 (Layer 3) tests and DOCSIS (Data Over Cable Service Interface Specification) tests.
  • a WiFi L2 resource lock involves a lock on resources for the Address Resolution Protocol (ARP) test.
  • ARP Address Resolution Protocol
  • Another example of a WiFi L2 resource lock involves a lock on resources for a WiFi connecting card test.
  • a non-limiting example of a WiFi L3 resource lock involves a lock on resources for a WiFi speed test.
  • a non-limiting example of a DOCSIS resource lock involves a lock on resources for a DOCSIS speed test.
  • Layer 2 and Layer 3 refer to the layers in the OSI model (Open System Interconnection model). Layer 2 is the data link layer of the OSI model.
  • Layer 3 is the network layer of the OSI model.
  • a testing system that provides a separate set of interfaces for each device (of a plurality of devices) that is under testing can perform WiFi L2 and WiFi L3 tests in a manner that minimizes or avoids wireless interference.
  • the testing system minimizes or avoids wireless interference by controlling locks on WiFi frequency channels.
  • the testing system comprises at least one test station.
  • each test station includes a plurality of physical slots for testing devices.
  • a subset of the plurality of physical slots is associated with a corresponding test server.
  • a test station may have four test servers, each of which is associated with a set of four physical slots of the plurality of physical slots.
  • the embodiments are not restricted to four test servers and four physical slots per test server.
  • the number of test servers and physical slots may vary from implementation to implementation.
  • each test server includes virtuahzation containers that act as probes for testing devices installed in the physical slots in the test station.
  • the testing system includes a central resource control server running on at least one test control computer.
  • each physical slot on the test station is assigned a specific frequency Channel on the WiFi frequency band.
  • a given physical slot can be assigned any of the frequency Channels 1 , 6, 1 1 (these channels are non-overlapping), according to certain embodiments.
  • a given physical slot can be assigned any of the frequency Channels 36, 40, 44, 48, 149, 153, 157, 161 , according to certain embodiments.
  • Such resources are shared across the test servers and associated slots of the test station.
  • each slot of the test station is assigned a frequency Channel in a manner to minimize wireless interference between the slots.
  • only one WiFi L3 test can be performed per Channel in the test station.
  • the central resource control server determines whether to grant WiFi resources to a given slot on a test station when the given slot requests a WiFi resource.
  • a given slot on the test station may request a lock on frequency Channel for performing a WiFi L2 test or for performing a WiFi L3 test.
  • the central resource control server determines whether to grant locks (on a given Channel), release locks and block slots from running specific WiFi tests based on certain criteria.
  • the WiFi frequency band may vary from implementation to implementation.
  • FIG. 13 to FIG. 22 are described with reference to the WiFi 2.4 GHz band.
  • the embodiments are not restricted to the WiFi 2.4 GHz band.
  • FIG. 13 to FIG. 22 are described using example test scenarios comprising sample requests from a sample subset of slots for WiFi resource locks. Such test scenarios are merely examples that help illustrate features of the
  • Such examples are to be regarded in an illustrative rather than a restrictive sense.
  • Other examples may include different requests from a different sample subset of slots for WiFi resource locks.
  • FIG. 13 illustrates a high-level schematic of a test station for testing devices such as wireless devices, according to certain embodiments.
  • FIG. 13 shows a test station 1300.
  • test station 1300 comprises a central resource control server 1302 that runs on a test control computer 1304.
  • Test station 1300 further comprises a plurality of test servers 1306a, 1306b, 1306c, 1306d.
  • Test station 1300 comprises a plurality of physical slots. According to certain embodiments, each test sever is associated with four physical slots but the
  • test server 1306a is associated with physical slots such as Slot 1 (1308a), Slot 2 (1308b), Slot 3 (1308c), Slot 4 (1308d).
  • Test server 1306b is associated with physical slots such as Slot 5 (1310a), Slot 6 (1310b), Slot 7 (1310c), Slot 8 (131 Od).
  • Test server 1306c is associated with physical slots such as Slot 9 (1312a), Slot 10 (1312b), Slot 1 1 (1312c), Slot 12 (1312d).
  • Test server 1306d is associated with physical slots such as Slot 13 (1313a), Slot 14 (1313b), Slot 15 (1313c), Slot 16 (1313d).
  • central resource control server 1302 controls WiFi resources such as WiFi L2 Channel 1 lock (1314), WiFi L3 Channel 1 lock (1316), WiFi L2 Channel 6 lock (1318), and WiFi L3 Channel 6 lock (1320),
  • each physical slot is assigned a Channel.
  • Slot 1 (1308a) is assigned Channel 1
  • Slot 2 (1308b) is assigned Channel 6
  • Slot 3 (1308c) is assigned Channel 1
  • Slot 4 (1308d) is assigned Channel 6
  • Slot 5 (1310a) is assigned Channel 6
  • Slot 6 (131 Ob) is assigned Channel 1
  • Slot 7 (1310c) is assigned Channel 1 1
  • Slot 8 (131 Od) is assigned Channel 6
  • the given slot When a given slot has a WiFi device installed in the given slot for testing, the given slot requests for a WiFi resource lock from central resource control server 1302 in order to perform a relevant wireless test on the installed device.
  • central resource control server 1302 grants WiFi resource locks in manner that minimizes interference between the various wireless tests that are running in the slots in test station 1300.
  • FIG. 14 illustrates requests for WiFi resource locks, according to certain embodiments.
  • Slot 1 (1308a) requests (1322) a lock on frequency Channel 1 in order to perform a WiFi L2 test on a device installed in Slot 1 for testing.
  • Slot 2 (1308b) requests (1324) a lock on frequency Channel 6 in order to perform a WiFi L2 test on a device installed in Slot 2 for testing.
  • Slot 3 (1308c) requests (1326) a lock on frequency Channel 1 in order to perform a WiFi L2 test on a device installed in Slot 3 for testing.
  • Slot 4 (1308d) requests (1328) a lock on frequency Channel 6 in order to perform a WiFi L2 test on a device installed in Slot 4 for testing.
  • Slot 5 (1310a) requests (1332) a lock on frequency Channel 6 in order to perform a WiFi L3 test on a device installed in Slot 5 for testing.
  • Slot 6 (1310b) requests (1330) a lock on frequency Channel 1 in order to perform a WiFi L2 test on a device installed in Slot 6 for testing.
  • the above requests from the slots for locks on frequency Channels are sent to central resource control server 1302.
  • FIG. 15 illustrates the response by central resource control server 1302 to the requests from the slots in the test station, according to certain embodiments.
  • central resource control server 1302 grants all the requests for locks for a given frequency Channel with respect to WiFi L2 tests as long as there is no ongoing WiFi L3 test for the given frequency Channel, as illustrated by FIG. 15.
  • FIG. 15 shows that Slot 1 (1308a) is granted (1322a) a lock (1322aa) on frequency Channel 1 by central resource control server 1302 in order to perform a WiFi L2 test on a device installed in Slot 1 for testing.
  • Slot 2 (1308b) is granted (1324a) a lock (1324aa) on frequency Channel 6 in order to perform a WiFi L2 test on a device installed in Slot 2 for testing.
  • Slot 3 (1308c) is granted (1326a) a lock (1326aa) on frequency Channel 1 in order to perform a WiFi L2 test on a device installed in Slot 3 for testing.
  • Slot 4 (1308d) is granted (1328a) a lock (1328aa) on frequency Channel 6 in order to perform a WiFi L2 test on a device installed in Slot 4 for testing.
  • Slot 6 (131 Ob) is granted (1330a) a lock (1330aa) on frequency Channel 1 in order to perform a WiFi L2 test on a device installed in Slot 6 for testing.
  • Slot 5 (1310a) is not granted (1332a) a lock (blocked 132aa) on frequency Channel 6 in order to perform a WiFi L3 test on a device installed in Slot 5 for testing.
  • Slot 5 (1310a) is not granted (1332a) a lock on frequency Channel 6 because a WiFi L3 test can only be run in a given Channel if there are no other tests running in the same Channel at a given time.
  • Slot 5 (1310a) cannot be granted (1332a) a lock on frequency Channel 6 for a WiFi L3 test because Slot 2 (1308b), and Slot 4 (1308d) have already been granted locks on frequency Channel 6 for performing their respective WiFi L2 tests.
  • Slot 5 (1310a) needs to request a lock on frequency Channel 6 to perform a WiFi L3 test at a later time, according to certain embodiments. According to certain other embodiments, Slot 5 (1310a) can keep its request for the lock pending in a request queue. Central resource control server 1302 will grant locks based on the next request in the queue as long the lock grant does not cause more than one WiFi L3 test to be run per frequency Channel.
  • FIG. 16 illustrates the release of locks on frequency Channels for WiFi tests, according to certain embodiments.
  • FIG. 16 shows that once a given slot completes its respective WiFi test, the given slot releases its lock on the frequency Channel that it was granted to perform the WiFi test or the given slot sends information to central resource control server 1300 that the given lock can be released.
  • FIG. 16 shows that Slot 1 (1308a) releases (1322b) its lock on frequency Channel 1 because it has completed (1322bb) the WiFi L2 test on a device installed in Slot 1 for testing.
  • Slot 2 (1308b) releases (1324b) its lock on frequency Channel 6 because it has completed (1324bb) the WiFi L2 test on a device installed in Slot 2 for testing.
  • Slot 3 (1308c) releases (1326b) its lock on frequency Channel 1 because it has completed
  • FIG. 17 illustrates the granting of a lock on a frequency Channel to a given slot for performing a WiFi L3 test, according to certain embodiments.
  • FIG. 17 shows that central resource control server 1302 grants (1332b) the request from Slot 5 (1310a) for a lock (1332bb) on Channel 6 to perform a WiFi L3 test now that the other locks on Channel 6 are released, according to certain embodiments.
  • FIG. 18 illustrates the release of a lock on a frequency Channel for a WiFi L3 test, according to certain embodiments.
  • FIG. 18 shows that once Slot 5 (1310a) has successfully completed (1332cc) its WiFi L3 test using its lock on frequency Channel 6, Slot 5 (1310a) releases (1332c) its lock on frequency Channel 6 and sends information on test completion to central resource control server 1302.
  • central resource control server 1302 is free to grant multiple locks on a given Channel for WiFi L2 tests or a single lock for a WiFi L3 test per channel (since WiFi L3 test must run alone per Channel) in response to requests for locks from slots on test station 1300.
  • FIG. 19 illustrates the requests from multiple slots for locks on Channels in order to perform respective WiFi L3 tests, according to certain embodiments.
  • FIG. 19 shows that after performing one or more WiFi L2 tests successfully (a slot may need to perform more than one WiFi L2 test, for example), a given slot can proceed to perform a WiFi L3 test, if desired.
  • Slot 1 requests (1336) a lock on frequency Channel 1 in order to perform a WiFi L3 test on a device installed in Slot 1 for testing.
  • Slot 2 requests (1308b) a lock on frequency Channel 6 in order to perform a WiFi L3 test on a device installed in Slot 2 for testing.
  • Slot 3 (1308c) requests (1340) a lock on frequency Channel 1 in order to perform a WiFi L3 test on a device installed in Slot 3 for testing.
  • Slot 4 requests (1342) a lock on frequency Channel 6 in order to perform a WiFi L3 test on a device installed in Slot 4 for testing.
  • Slot 6 (1310b) requests (1344) a lock on frequency Channel 1 in order to perform a WiFi L3 test on a device installed in Slot 6 for testing.
  • the above requests from the slots for locks on frequency Channels are sent to central resource control server 1302.
  • central resource control server 1302 will grant only one lock per frequency Channel for WiFi L3 tests in order to ensure minimum wireless interference per frequency Channel when several WiFi tests are run at the same time using various frequency Channels, as described further herein with reference to FIG. 20.
  • FIG. 20 illustrates that the central resource control server will grant only one lock per frequency Channel for WiFi L3 tests, according to certain embodiments.
  • FIG. 20 shows that central resource control server 1302 grants (1336a), to Slot 1 (1308a), a lock (1336aa) on frequency Channel 1 so that Slot 1 (1308a) can perform a WiFi L3 test on a device installed in Slot 1 for testing.
  • FIG. 20 also shows that central resource control server 1302 grants (1342a), to Slot 4 (1308d), a lock
  • central resource control server 1302 rejects (1338a, 1340a, 1344a) the respective requests of Slot 2 (1308b), Slot 3 (1308c) and Slot 6 (1310b). Thus, such requested locks are blocked (1338aa, 1340aa, 1344aa). According to certain embodiments, central resource control server 1302 grants requests for locks on frequency Channels on a first-come- first-served basis.
  • central resource control server 1302 grants requests for locks on frequency Channels based on business rules. Since Slot 2 (1308b), Slot 3 (1308c) and Slot 6 (1310b) were denied locks for their respective WiFi L3 tests, Slot 2 (1308b), Slot 3 (1308c) and Slot 6 (1310b) need to request again for locks for their respective WiFi L3 tests from central resource control server 1302 at a later time as described with reference to FIG. 21 herein. [001 19] FIG. 21 illustrates the release of locks on a frequency Channels after completion of WiFi L3 tests, according to certain embodiments. FIG.
  • central resource control server 1302 can grant multiple locks on the frequency Channels for WiFi L2 tests (if needed) in response to requests for locks from slots on test station 1300. Since Slot 2 (1308b), Slot 3 (1308c) and Slot 6 (1310b) were previously denied locks for their respective WiFi L3 tests, Slot 2 (1308b), Slot 3 (1308c) and Slot 6 (1310b) can now request (1338, 1340, 1344) for locks for their respective WiFi L3 tests from central resource control server 1302, according to certain embodiments.
  • FIG. 22 illustrates the grant of a recently released lock on a frequency
  • central resource control server 1302 can grant L3 locks to the next request for an L3 lock per frequency Channel.
  • the requests for L3 locks from Slot 2 (1308b), Slot 3 (1308c) and Slot 6 (1310b) were denied. Assume that the request for an L3 lock on Channel 1 from Slot 6 (1310b) precedes the request for an L3 lock on Channel 1 from Slot 3 (1308c).
  • central resource control server 1302 grants (1344b) Slot 6 (1310b) a lock (1344bb) on Channel 1 for its WiFi L3 test while denying (1340a) the request from Slot 3 (1308c) for an L3 lock (blocked 1340aa) on Channel 1 . Further, central resource control server 1302 grants (1338b) to Slot 2 (1310b) a lock (1338bb) on Channel 6 for its WiFi L3 test since there is no other pending L3 lock or pending L2 lock on Channel 6, according to certain embodiments. [00121] In the foregoing specification, embodiments of the invention have been described with reference to numerous specific details that may vary from

Abstract

A system for testing multiple disparate or similar devices independently and simultaneously using different types of device probes is disclosed. The devices under test (DUTs) can include set top boxes, wireless routers and modem/eMTA devices. The system includes real-time, bi-directional/asynchronous communication and interaction between the system components. Also, an operator dashboard (user interface) used for testing disparate devices simultaneously and independently and further capable of asynchronous communication is disclosed. In addition, a core testing executor/processor for testing a plurality of devices simultaneously using virtualization containers to connect to interfaces of corresponding devices under test is disclosed. Further, a testing system that provides a separate set of virtualization container probes for each of at least a subset of devices that is under testing can perform WiFi Layer 2 and WiFi Layer 3 tests in a manner that minimizes or avoids wireless interference is disclosed.

Description

UNIVERSAL DEVICE TESTING SYSTEM
TECHNICAL FIELD
[0001] The present invention is directed to a system for testing devices.
BRIEF DESCRIPTION OF THE DRAWINGS
[0002] For a better understanding of the aforementioned aspects of the invention as well as additional aspects and embodiments thereof, reference should be made to the description of embodiments below, in conjunction with the following drawings in which like reference numerals refer to corresponding parts throughout the figures.
[0003] FIG. 1 illustrates a high-level system architecture for testing devices, according to certain embodiments.
[0004] FIG. 2 illustrates some of the testing components and the interaction between the testing components, according to certain embodiments.
[0005] FIG. 3 illustrates a sample architecture that includes the testing components, according to certain embodiments.
[0006] FIG. 4 illustrates a set top box under test, according to certain embodiments.
[0007] FIG. 5 illustrates a wireless router under test, according to certain
embodiments.
[0008] FIG. 6 illustrates a cable modem/eMTA device under test, according to certain embodiments.
[0009] FIG. 7 illustrates a high-level operator dashboard for interacting with the core testing execution environment, according to certain embodiments.
[0010] FIG. 8 illustrates some components of a sample l-Frame, according to certain embodiments.
[001 1] FIG. 9 illustrates a sample architecture showing bi-directional asynchronous communication between the operator dashboard, web-socket layer and core test execution machine, according to certain embodiments.
[0012] FIG. 10 illustrates a sample server side Node.JS layer, according to certain embodiments. [0013] FIG. 1 1 is a high-level flow chart that illustrates some steps performed by a core testing executor/processor for testing devices, according to certain
embodiments.
[0014] FIG. 12 is a high-level schematic that illustrates device under test (DUT) probes through the use of virtualization containers, according to certain
embodiments.
[0015] FIG. 13 illustrates a high-level schematic of a test station for testing devices such as wireless devices, according to certain embodiments.
[0016] FIG. 14 illustrates requests for WiFi resource locks, according to certain embodiments.
[0017] FIG. 15 illustrates the response by the central resource control server to the requests from the slots in the test station, according to certain embodiments.
[0018] FIG. 16 illustrates the release of locks on frequency Channels for WiFi tests, according to certain embodiments.
[0019] FIG. 17 illustrates the granting of a lock on a frequency Channel to a given slot for performing a WiFi L3 test, according to certain embodiments.
[0020] FIG. 18 illustrates the release of a lock on a frequency Channel for a WiFi L3 test, according to certain embodiments.
[0021] FIG. 19 illustrates the requests from multiple slots for locks on frequency Channels in order to perform respective WiFi L3 tests, according to certain embodiments.
[0022] FIG. 20 illustrates that central resource control server will grant only one lock per frequency Channel for WiFi L3 tests, according to certain embodiments.
[0023] FIG. 21 illustrates the release of locks on a frequency Channels after completion of WiFi L3 tests, according to certain embodiments.
[0024] FIG. 22 illustrates the grant of a recently released lock on a frequency Channel to the next slot request in the queue, according to certain embodiments. DETAILED DESCRIPTION
[0025] Methods, systems, user interfaces, and other aspects of the invention are described. Reference will be made to certain embodiments of the invention, examples of which are illustrated in the accompanying drawings. While the invention will be described in conjunction with the embodiments, it will be understood that it is not intended to limit the invention to these particular embodiments alone. On the contrary, the invention is intended to cover alternatives, modifications and
equivalents that are within the spirit and scope of the invention. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.
[0026] Moreover, in the following description, numerous specific details are set forth to provide a thorough understanding of the present invention. However, it will be apparent to one of ordinary skill in the art that the invention may be practiced without these particular details. In other instances, methods, procedures, components, and networks that are well known to those of ordinary skill in the art are not described in detail to avoid obscuring aspects of the present invention.
[0027] Testing System Overview
[0028] According to certain embodiments, an innovative system can test a set of devices simultaneously. Further, such a testing system is capable of testing disparate devices simultaneously. Non-limiting examples of devices under test (DUTs) include set top boxes, cable modems, embedded multimedia terminal adapters, and wireless routers including broadband wireless routers for the home or for commercial networks.
[0029] According to certain embodiments, such a testing system provides a separate set of interfaces to be tested for each device of the set of devices that is under testing. Further, such a system is designed to be adaptive by being extendable for testing new devices with corresponding new testing interfaces without fundamentally changing the core architecture of the testing system. As a non-limiting example, the testing system includes a core testing subsystem with a user interface and asynchronous communication among the system components such that new types of devices and new tests can be added and executed in a seamless fashion. [0030] According to certain embodiments, the user interface can communicate through web sockets with a universal tester. Such communication is in real-time, bidirectional and asynchronous so that the user can control and monitor the testing of multiple devices simultaneously and independently of each other using the same universal tester, and in certain embodiments, its associated test bench.
[0031 ] According to certain embodiments, the testing system is capable of testing a set of similar types of devices or a set of disparate devices, wherein the plurality of devices are tested simultaneously by the testing system.
[0032] According to certain embodiments, a testing solution system can be a three layer implementation. The number of layers may vary from implementation to implementation. FIG. 1 illustrates a high-level system architecture for testing devices, according to certain embodiments. FIG. 1 shows a test bench browser interface 102 that is in communication with a web-socket 104, that is, in turn, in communication with a core testing processor 106. According to certain embodiments, the
communication between the test bench browser 102, web-socket 104 and core testing processor 106 can be a TCP/IP communication. As a non-limiting example, the web browser is used as a user interface that communicates through web-sockets with the core testing processor. As a non-limiting example, communication may be in the form of JSON messages using TCP/IP protocol, according to certain
embodiments. JSON is Java script object notation for transmitting data between the server and web applications.
[0033] FIG. 2 illustrates some of the testing components and the interaction between the testing components, according to certain embodiments. FIG. 2 shows a user interface 202, web-sockets 204, a core testing processor 206, database 208, test configuration modules 210, testing environment modules 212, a plurality of probes (214, 216, 218) to connect the devices under test (DUTs) to the core testing processor 206, and a speed test module 220, according to certain embodiments. Speed testing is used for evaluating the performance of the WiFi and other media network connection and accessibility of the device under test. FIG. 2 shows as non- limiting examples, a WiFi probe 214, an Ethernet local area network (LAN) probe 216 and a MoCA probe 218. In other words, according to certain embodiments, various probes can be included such as a wireless local area network (WLAN) probe, an Ethernet wide area network (WAN) probe, a multimedia over coax alliance (MoCA) WAN probe, a MoCA LAN probe and a wireless probe via antenna.
According to certain embodiments, servers and other components in the testing system may be distributed over a plurality of computers.
[0034] According to certain embodiments, core testing processor 206 loads and reads files from test configuration modules 210 and test environment modules 212 to initialize various components of the testing system. When the system is ready to begin testing after the initialization process, the system notifies a user that is using the testing system to test one or more DUTs of the readiness of the testing system. The user installs each DUT of the set of DUTs that are to be tested on the test bench and the serial number of each DUT is scanned. According to certain embodiments, the user installs each DUT in a separate Faraday cage (slot) in the test bench.
According to certain embodiments, there are several Faraday cages in a given test bench so that a plurality of DUTs can be tested simultaneously using the same test bench and same universal tester. The core testing processor 206 receives the serial number information of each DUT and using the serial number, retrieves further information associated with each DUT based on the serial number from database 208, according to certain embodiments. The core testing processor 206 dynamically loads test configuration information 210 and test environment information 212 based on device information such as make, model etc. of a given DUT. After the test configuration and test environment information are loaded, the core testing
processor 206 begins executing the various tests corresponding to each DUT so that the set of DUTs can be tested simultaneously. Each test may correspond to underlying testing modules associated with WiFi, LAN, WAN or MoCA etc., interfaces of the DUT and such modules can be executed locally, remotely or at the device.
[0035] According to certain embodiments, the test configuration information identifies the test modules and corresponding testing scripts that are to be executed by the core testing processor 206 at run time. The core testing processor 206 also provides the test results and other feedback information to the user via the browser user interface 202 and web sockets 204. Further, the user can send user input and requests to the system through the browser user interface 202 and web sockets 204.
[0036] According to certain embodiments, core testing processor 206 determines the success or failure of a given test based on the test configuration parameters and output results of the testing. Further, upon failure of a given test, core testing processor 206 may continue further testing or halt test execution based on test configuration parameters, according to certain embodiments.
[0037] Upon completion of the relevant tests, a success message can be sent to the user via the browser user interface 202 and web sockets 204. Even though the DUTs in the set of DUTs are tested simultaneously, the user does not have to wait until the testing of all the DUTs in the set has been completed to begin installing other devices that need testing. Further, the testing of the devices need not be started at the same time. Soon after the testing is completed for a given DUT, the tested DUT may be uninstalled from its slot in the test bench and a new DUT can be installed in its slot so that testing can begin for the newly installed device.
[0038] According to certain embodiments, the test results can be stored locally and/or pushed to the cloud so that the results can be viewed remotely from any location. Further, the test results can be aggregated. According to certain
embodiments, aggregated data includes data combined from several data
measurements. Summary reports can be generated from such aggregated data. Non-limiting examples of summary reports include charts and graphs that display information on all the DUTs or at least a subset of the DUTs. Thus, the summary reports generated from the aggregated data can provide an overview of the testing information and characteristics of the DUTs. The aggregated data can reveal trends and other related information associated with the DUTs. Further, the aggregated data can include user-level data, access account activity, etc. According to certain embodiments, the testing system includes a billing system to charge for the testing services for each device.
[0039] FIG. 3 illustrates a sample architecture that includes the testing components of a universal tester, according to certain embodiments. FIG. 3 shows a browser user interface or operator dashboard 302, a test controller 304, a universal tester 306 and a DUT 308. There may be multiple devices under testing simultaneously but only one device under test is shown for convenience in FIG. 3.
[0040] According to certain embodiments, browser user interface or operator dashboard 302 may include information 310 associated with each device under test. The information 310 can include DUT serial number 31 1 , and testing progress information 312. Browser user interface or operator dashboard 302 may also include user command function buttons 314 and drop down menus (not shown in FIG. 3). According to certain embodiments, the user can configure slot details (e.g., port numbers, IP address for the slot, etc.), configure testing preferences such as push to cloud, export to billing, etc.
[0041 ] According to certain embodiments, test controller 304 may include a universal tester webserver 316 that is in communication (e.g., TCP/IP) with a universal tester database 318. A billing process within the controller (not shown in FIG. 3) may be in communication with a billing service or application (not shown in FIG. 3). As a non- limiting example, database 318 can be a SQL database. Database 318 can store information associated with each slot in the test bench. As non-limiting examples, database 318 can store for each slot, test details, test history, test logs, DUT information (e.g., DUT serial number, model name, etc.), testing
preferences/configuration, user interface details/preferences/configuration, billing information, cloud push information, MSO/customer information (media subscriber organization), OEM (original equipment manufacturer) information, slot information, user information, and any persistent data needed by the universal device testing system for running tests.
[0042] According to certain embodiments, universal tester 306 may include web sockets 320 that are in communication (e.g., TCP/IP) with browser user interface or operator dashboard 302 and core testing processor 324. According to certain embodiments, core testing processor 324 is in communication with test controller 304 (e.g., TCP/IP) and in communication (e.g., Telnet/SSH secure shell) with probes/containers (328, 330, 332, 334). Core testing processor 324 is also in communication with configuration modules 322 (e.g., testing and environment configuration). Non-limiting examples of probes include WiFi probe 328, LAN probe 330, MoCA probe 332 and WAN probe 334. There may be other types of probes including MoCA WAN probe, MoCA LAN probe and different types of wireless probes besides WiFi probes depending on the characteristics of the device being tested.
[0043] According to certain embodiments, WiFi probe 328, LAN probe 330, MoCA probe 332 and WAN probe 334 communicate with the respective device under test through the relevant ports on the device such as WiFi port 336, LAN port 338, MoCA port 340 and WAN port 342. Core testing processor 324 executes the relevant configured tests for the respective DUT. Status and test results can be sent to the user's dashboard (using JSON format messages as a non-limiting example) via the web-sockets.
[0044] According to certain embodiments, when executing a specific test for a given DUT, the core testing executor/processor loads and reads test configuration information (for example from an XML structure) and identifies the relevant test script that needs to be executed. Inputs that are needed for executing the relevant test script are retrieved and supplied as inputs to the relevant test script. The following is a non-limiting sample script.
Create DUT object & Environment Object
Verify Serial Number
Verify Warranty
Check Report Server
Check DUT Staging
Checks for DUT Serial number in Database or Web-service Get DUT Readiness Information
Checks Web-service for test readiness status of DUT in the test
process
Configure Container Environment
Clear Environment Temp Files
Analyze DUT for Factory Reset
Checks ability to login to DUT
Asks operator to manually Factory Reset if unable to login Confirm Factory Reset (if needed)
Waits for operator to confirm that DUT was factory reset and booted up properly
Check Ethernet LAN connections to DUT
Ping connections: Eth LAN 1 , 2, 3, 4
Fails if any ping to these connections fail
Detect DUT
Checks connection to DUT through socket connection
Reset Password Operator scans password which is stored temporarily for use in the remainder of test until finished
Login to GUI
Done through web-scraping
Get DUT Information and compare values
Information retrieved through web-scraping
Enable Telnet
Enables telnet on DUT through web-scraping
Factory Reset
Factory resets DUT through telnet command
Enable Telnet after Factory Reset
Enables telnet on DUT through web-scraping
Confirm Power, WAN Ethernet, and Internet LEDs
Confirm all LAN Ethernet LEDs
Confirm WiFi LED
Configure Wireless Network
Through telnet commands
Sets N Mode
Enables Privacy
Sets WPA (Wi-Fi Protected Access)
Removes WEP (Wired Equivalent Privacy)
Assigns WiFi Channel to DUT (channel different by slot)
[Channel 1 : slots 1 , 4, 7, 10, 13, 16]
[Channel 6: slots 2, 5, 8, 1 1 , 14]
[Channel 1 1 : slots 3, 6, 9, 12, 15]
Verifies changes through GUI
Disables WiFi once done through telnet
Check Firmware Version and Upgrade Firmware (if needed)
Firmware version: 40.21 .18
Cage Closed Confirmation Check
Asks Operator to Close Door on Cage
Connect Wireless Card
Waits on shared Resource Server (located on TC) for Resource L2 (Layer 2) Lock Lock waiting timeout: 600 sec
All L2 Locks are able to run in parallel but not when any L3 (Layer 3) Lock is running
Obtains Lock
Enables WiFi through telnet
Set WiFi Card
Total Retries allowed: 6 (2 sets of 3 retries) Ping WiFi from DUT
L2 ARP Test on WiFi: must receive 10/10 ARP packets
Total Retries allowed: 6 (2 sets of 3 retries) If either Set WiFi Card or L2 ARP Test Fail after its 3 retries, Ask
Operator to Check Antennas
Performs one more retry in full (set of 3 retries each for Set WiFi Card and L2 ARP WiFi Test) after Check Antennas
Disables WiFi through telnet
Releases Lock
Wireless to LAN Ethernet Speed Test
Waits on shared Resource Server (located on TC) for Resource L3
Lock
Lock waiting timeout: 1800 sec
L3 Locks must be run one at a time and when no L2 Lock is running
Obtains Lock
Enables WiFi through telnet
Connects WiFi Card
Iperf3 Speed Test, 5 seconds for UDP Speed Test, 7 seconds for TCP
Speed Test, Sending 200 Mbps Bandwidth
Bandwidth must be greater than 60 Mbps on TCP (Reverse) or 70 Mbps on UDP (Forward)
If Fail after 2 retries, ask operator to Check Antennas Retries up to 2 times more if still Fail
Therefore, Total Retries allowed: 4 (2 sets of 2 retries) Runs sudo iwlist wlanO scan and returns all Wireless Signals seen Results parsed to print all visible SSIDs and its matching Signal level
Disables WiFi through telnet
Releases Lock
Confirm WPS LED
Confirm LAN Coax LED
Confirm USB 1 +2 LEDs
Configure WAN MoCA
Confirm WAN Coax LED
Ping WAN MoCA
L2 Test on LAN Ethernet
Arp Test from Eth LAN 1 to Eth LAN 2, 3, 4
Must receive 10/10 on all LAN connections
LAN Ethernet to LAN Ethernet Speed Test
From Eth LAN 1 to Eth LAN 2, 3, 4
Iperf3 Speed Test, 5 seconds Reverse and Forward, Sending 1200
Mbps Bandwidth
Bandwidth must be greater than 700 Mbps
Total Retries allowed: 2
Check WAN and LAN MoCA Data Rates
Rx and Tx Data rates for both WAN and LAN MoCA retrieved through telnet
All Rates must be greater than 180 Mbps
LAN Ethernet to WAN MoCA FTP Speed Test
From Eth LAN 1 to WAN MoCA
Iperf3 Speed Test, 5 seconds Reverse and Forward, Sending 1200
Mbps Bandwidth
Bandwidth must be greater than 60 Mbps
Total Retries allowed: 2
LAN MoCA to LAN Ethernet FTP Speed Test
From Eth LAN 1 to LAN MoCA
Iperf3 Speed Test, 5 seconds Reverse and Forward, Sending 240
Mbps Bandwidth
Bandwidth must be greater than 60 Mbps Total Retries allowed: 2
LAN MoCA to WAN MoCA FTP Speed Test
From LAN MoCA to WAN MoCA
Iperf3 Speed Test, 5 seconds Reverse and Forward, Sending 240
Mbps Bandwidth
Bandwidth must be greater than 60 Mbps
Total Retries allowed: 2
Enable WAN Ethernet
Through telnet command
LAN Ethernet to WAN Ethernet FTP Speed Test
From Eth LAN 1 to Eth WAN
Iperf3 Speed Test, 5 seconds Reverse and Forward, Sending 1200 Mbps Bandwidth
Bandwidth must be greater than 700 Mbps
Total Retries allowed: 2
Clear Persistent Logs
Final Factory Restore
[0045] According to certain embodiments, the core testing executor/processor uses a reflection and command design pattern to invoke the relevant configured script(s) corresponding to each DUT being tested. For example, in the command design pattern one or more of the following are encapsulated in an object: an object, method name, arguments. According to certain embodiments, the core testing processor uses the Python "reflection" capability to execute the relevant test scripts for a given DUT. The core testing processor is agnostic of the inner workings of the relevant test scripts for a given DUT.
[0046] According to certain embodiments, lightweight software containers
(virtualization containers) are used to abstract the connection of probes to the different DUT interfaces in order to avoid conflicts. Non-limiting examples of virtualization containers are Linux containers. As a non-limiting example, Linux container (LXC) is an operating-system-level virtualization environment for running multiple isolated Linux systems (virtualization containers) on a single Linux control host. In other words, lightweight virtualization containers are used to ensure isolation across servers. By using virtualization containers, resources can be isolated, services restricted, and processes provisioned to have an almost completely private view of the operating system with their own process ID space, file system structure, and network interfaces. Multiple virtualization containers share the same kernel, but each virtualization container can be constrained to only use a defined amount of resources such as CPU, memory, network resources and I/O. The relevant test script connects to the DUT interfaces directly or through the virtualization containers to execute the tests. The core testing executor/processor receives the test results from running the relevant test scripts. The core testing executor/processor can further process and interpret such results and can also send the results to the user's browser via web sockets. According to certain embodiments, the respective core testing executors/processors are in communication (e.g., Telnet/SSH secure shell) with the virtualization containers (there may be multiple virtualization containers). The virtualization containers (probes) are in communication with corresponding DUT interfaces using Telnet/SSH/TCP/UDP/HTTP/HTTPS etc., as non-limiting examples.
[0047] According to certain embodiments, a system for testing a plurality of devices comprises: a universal tester, at least one test controller, a plurality of sets of testing probes, and a plurality of web sockets, wherein: the universal tester is enabled for communication with a platform independent user interface through the plurality of web sockets; each respective set of testing probes of at least a subset of the plurality of sets of testing probes are for communicating with respective ports of a respective device under test of the plurality of devices installed in the universal tester for purposes of testing; and the plurality of web sockets enable real-time bi-directional and asynchronous communication between the platform independent user interface and the universal tester for simultaneously testing the plurality of devices under test by the universal tester.
[0048] According to certain embodiments, the system for testing a plurality of devices further comprises a MoCA LAN bridge.
[0049] According to certain embodiments, the system for testing a plurality of devices further comprises a MoCA WAN bridge.
[0050] According to certain embodiments, the system for testing a plurality of devices further comprises a splitter. [0051 ] According to certain embodiments, the system for testing a plurality of devices further comprises a cable modem terminal system (CMTS).
[0052] FIG. 4 illustrates a testing architecture for a set top box under test, according to certain embodiments. As previously explained, multiple similar or disparate devices can be tested simultaneously and independently of each other using the same universal tester. Thus, multiple set top boxes can be tested simultaneously and independently of each other using the same universal tester, along with other types of devices using the same universal tester. For purposes of simplicity only one set top box is shown in FIG. 4. FIG. 4 shows a universal tester 404 and set top box 402, which is the device under test for this specific case. Universal tester 404 includes a plurality of virtualization containers (probes) for communicating with corresponding interfaces of set top box 402. For example, the core testing processor of the universal tester (as described herein) uses the HDMI (high definition multimedia interface) probe/container 406b to test the HDMI interface 406a of set top box 402. Similarly, audio/video probe/container 408b can be used to test the audio/video interface 408a of set top box 402. Another audio/video probe/container 410b can be used to test the Coax TV Output interface 410a of set top box 402. IR (infrared) probe/container 412b can be used to test the IR interface 412a of set top box 402. CATV coax probe/container 416b can be used to test the Coax interface 416a of set top box 402. The associated core testing processor executes the relevant configured tests for the set top box 402. Status and test results can be sent to the user's dashboard (using JSON format messages as a non-limiting example) via the web-sockets.
[0053] According to certain embodiments, a system for testing a plurality of devices comprises: a universal tester, at least one test controller, a plurality of sets of testing probes, and a plurality of web sockets, wherein: the plurality of devices includes a plurality of set top boxes; the universal tester is enabled for communication with a platform independent user interface through the plurality of web sockets; the plurality of sets of testing probes comprising: at least one HDMI probe for testing a
corresponding HDMI interface of a set top box of the plurality of set top boxes, at least one audio video probe for testing a corresponding audio video interface of the set top box of the plurality of set top boxes, at least one audio video probe for testing a corresponding coax TV output interface of the set top box of the plurality of set top boxes, at least one IR probe for testing a corresponding IR interface of the set top box of the plurality of set top boxes, at least one CATV coax probe for testing a corresponding coax interface of the set top box of the plurality of set top boxes; and the plurality of web sockets enable real-time bi-directional and asynchronous communication between the platform independent user interface and the universal tester for simultaneously testing the plurality of devices under test by the universal tester.
[0054] FIG. 5 illustrates a testing architecture 500 for a wireless router under test, according to certain embodiments. As previously explained, multiple similar or disparate devices can be tested simultaneously and independently of each other using the same universal tester. Thus, multiple wireless routers can be tested simultaneously and independently of each other using the same universal tester, along with other types of devices using the same universal tester. For purposes of simplicity only one wireless router is shown in FIG. 5. FIG. 5 shows a universal tester 504 and a wireless router 502, which is the device under test for this specific case. Non-limiting examples of wireless routers include broadband wireless routers for the home or for commercial networks. Universal tester 504 includes a plurality of virtualization containers (probes) for communicating with corresponding interfaces of wireless router 502. For example, the core testing processor of the universal tester (as described herein) uses the LAN probes/containers 506b, 508b, 510b, 512b to test corresponding LAN interfaces 506a, 508a, 510a, 512a of wireless router 502. WLAN (wireless LAN) probe/container 516b can be used to test WLAN interface 516a of wireless router 502. Ethernet WAN probe/container 528b can be used to test WAN interface 528a of wireless router 502. MoCA LAN probe/container 518b can be used to test Coax interface 532 of wireless router 502 via MoCA LAN bridge 518a and splitter 520. MoCA WAN probe/container 530b can be used to test Coax interface 532 of wireless router 502 via MoCA WAN bridge 530a and splitter 520. The associated core testing processor executes the relevant configured tests for the wireless router 502. Status and test results can be sent to the user's dashboard (using JSON format messages as a non-limiting example) via the web-sockets.
[0055] According to certain embodiments, when executing a specific test for a given DUT, the core testing executor/processor loads and reads test configuration information (for example from an XML structure) and identifies the relevant test script that needs to be executed. Inputs that are needed for executing the relevant test script are retrieved and supplied as inputs to the relevant test script. The following a non-limiting sample script.
Create DUT object & Environment Object
Verify Serial Number
Verify Warranty
Check Report Server
Check DUT Staging
Checks for DUT Serial number in Database or Web-service
Get DUT Readiness Information
Checks Web-service for test readiness status of DUT in the test process
Configure Container Environment
Clear Environment Temp Files
Analyze DUT for Factory Reset
Checks ability to login to DUT
Asks operator to manually Factory Reset if unable to login
Confirm Factory Reset (if needed)
Waits for operator to confirm that DUT was factory reset and booted properly
Check Ethernet LAN connections to DUT
Ping connections: Eth LAN 1 , 2, 3, 4
Fails if any ping to these connections fail
Detect DUT
Checks connection to DUT through socket connection Reset Password
Operator scans password which is stored temporarily for use in the remainder of test until finished
Login to GUI
Done through web-scraping
Get DUT Information and compare values
Information retrieved through web-scraping
Enable Telnet Enables telnet on DUT through web-scraping
Factory Reset
Factory resets DUT through telnet command
Enable Telnet after Factory Reset
Enables telnet on DUT through web-scraping
Confirm Power, WAN Ethernet, and Internet LEDs
Confirm all LAN Ethernet LEDs
Confirm WiFi LED
Configure Wireless Network
Through telnet commands
Sets N Mode
Enables Privacy
Sets WPA (Wi-Fi Protected Access)
Removes WEP (Wired Equivalent Privacy)
Assigns WiFi Channel to DUT (channel different by slot)
[Channel 1 : slots 1 , 4, 7, 10, 13, 16]
[Channel 6: slots 2, 5, 8, 1 1 , 14]
[Channel 1 1 : slots 3, 6, 9, 12, 15]
Verifies changes through GUI
Disables WiFi once done through telnet
Check Firmware Version and Upgrade Firmware (if needed)
Firmware version varies by models
Cage Closed Confirmation Check
Asks Operator to Close Door on Cage
Connect Wireless Card
Waits on shared Resource Server (located on TC) for Resource L2 (Layer 2) Lock
Lock waiting timeout: 600 sec
All L2 Locks are able to run in parallel but not when any L3 (Layer 3) Lock is running
Obtains Lock
Enables WiFi through telnet
Set WiFi Card
Total Retries allowed: 6 (2 sets of 3 retries) Ping WiFi from DUT
L2 ARP Test on WiFi: must receive 10/10 ARP packets
Total Retries allowed: 6 (2 sets of 3 retries) If either Set WiFi Card or L2 ARP Test Fail after its 3 retries, Ask
Operator to Check Antennas
Performs one more retry in full (set of 3 retries each for Set WiFi Card and L2 ARP WiFi Test) after Check Antennas
Disables WiFi through telnet
Releases Lock
Wireless to LAN Ethernet Speed Test
Waits on shared Resource Server (located on TC) for Resource L3
Lock
Lock waiting timeout: 1800 sec
L3 Locks must be run one at a time and when no L2 Lock is running
Obtains Lock
Enables WiFi through telnet
Connects WiFi Card
Iperf3 Speed Test, 5 seconds for UDP Speed Test, 7 seconds for TCP
Speed Test, Sending 200 Mbps Bandwidth
Bandwidth must be greater than 60 Mbps on TCP (Reverse) or 70 Mbps on UDP (Forward)
If Fail after 2 retries, ask operator to Check Antennas Retries up to 2 times more if still Fail
Therefore, Total Retries allowed: 4 (2 sets of 2 retries) Runs sudo iwlist wlanO scan and returns all Wireless Signals seen
Results parsed to print all visible SSIDs and its matching Signal level
Disables WiFi through telnet
Releases Lock
Check wireless signal strength
Confirm if signal strength for each antenna is greater than threshold (e.g., -50dB)
Confirm WPS LED Confirm LAN Coax LED
Confirm USB 1 +2 LEDs
Configure WAN MoCA
Confirm WAN Coax LED
Ping WAN MoCA
Verify WPS trigger
Confirm if WPS state toggles on the DUT after instruction
L2 Test on LAN Ethernet
Arp Test from Eth LAN 1 to Eth LAN 2, 3, 4
Must receive 10/10 on all LAN connections
LAN Ethernet to LAN Ethernet Speed Test
From Eth LAN 1 to Eth LAN 2, 3, 4
Iperf3 Speed Test, 5 seconds Reverse and Forward, (e.g., Sending
1200 Mbps Bandwidth)
Bandwidth must be greater than threshold (e.g., 700 Mbps)
Total Retries allowed: 2
Check WAN and LAN MoCA Data Rates
Rx and Tx Data rates for both WAN and LAN MoCA retrieved through telnet
All Rates must be greater than threshold (e.g., 180 Mbps)
LAN Ethernet to WAN MoCA FTP Speed Test
From Eth LAN 1 to WAN MoCA
Iperf3 Speed Test, 5 seconds Reverse and Forward (e.g., Sending
1200 Mbps Bandwidth)
Bandwidth must be greater than threshold (e.g., 60 Mbps)
Total Retries allowed: 2
LAN MoCA to LAN Ethernet FTP Speed Test
From Eth LAN 1 to LAN MoCA
Iperf3 Speed Test, 5 seconds Reverse and Forward, (e.g., Sending
240 Mbps Bandwidth)
Bandwidth must be greater than threshold (e.g., 60 Mbps)
Total Retries allowed: 2
LAN MoCA to WAN MoCA FTP Speed Test
From LAN MoCA to WAN MoCA Iperf3 Speed Test, 5 seconds Reverse and Forward, (e.g., Sending 240 Mbps Bandwidth)
Bandwidth must be greater than threshold (e.g., 60 Mbps)
Total Retries allowed: 2
Enable WAN Ethernet
Through telnet command
LAN Ethernet to WAN Ethernet FTP Speed Test
From Eth LAN 1 to Eth WAN
Iperf3 Speed Test, 5 seconds Reverse and Forward, (e.g., Sending 1200 Mbps Bandwidth)
Bandwidth must be greater than threshold (e.g., 700 Mbps)
Total Retries allowed: 2
Clear Persistent Logs
Final Factory Restore
[0056] According to certain embodiments, a system for testing a plurality of devices comprises: a universal tester, at least one test controller, a plurality of sets of testing probes, and a plurality of web sockets, wherein: the plurality of devices includes a plurality of wireless routers; the universal tester is enabled for communication with a platform independent user interface through the plurality of web sockets; the plurality of sets of testing probes comprising: a plurality of LAN probes for testing
corresponding LAN interfaces of a wireless router of the plurality of wireless routers, at least one WLAN probe for testing a corresponding WLAN interface of the wireless router of the plurality of wireless routers, at least one Ethernet WAN probe for testing a corresponding WAN interface of the wireless router of the plurality of wireless routers, at least one MoCA LAN probe for testing a corresponding coax interface of the wireless router of the plurality of wireless routers, at least one MoCA WAN probe for testing a corresponding coax interface of the wireless router of the plurality of wireless routers; and the plurality of web sockets enable real-time bi-directional and asynchronous communication between the platform independent user interface and the universal tester for simultaneously testing the plurality of devices under test by the universal tester.
[0057] FIG. 6 illustrates a testing architecture 600 for a cable modem/eMTA under test, according to certain embodiments. As previously explained, multiple similar or disparate devices can be tested simultaneously and independently of each other using the same universal tester. Thus, multiple cable modem s/eMTAs can be tested simultaneously and independently of each other using the same universal tester, along with other types of devices using the same universal tester. For purposes of simplicity only one cable modem/ eMTA is shown in FIG. 6. FIG. 6 shows a universal tester 604 and cable modem/ eMTA (embedded multimedia terminal adapter) 602, which is the device under test for this specific case. Universal tester 604 includes a plurality of virtualization containers (probes) for communicating with corresponding interfaces of cable modem/eMTA 602. For example, the core testing processor of the universal tester (as described herein) uses the LAN probes/containers 606b, 608b, 610b, 612b to test corresponding LAN interfaces 606a, 608a, 610a, 612a of cable modem/eMTA 602. Similarly, FXS (foreign exchange station) probe/container 614b can be used to test the FXS interface 614a of cable modem/ eMTA 602. WLAN (wireless LAN) probe/container 616b can be used to test WLAN interface 616a of cable modem/eMTA 602. MoCA LAN probe/container 618b can be used to test MOCA LAN interface of cable modem/ eMTA 602 via MoCA LAN bridge 618a and splitter 620. NCS (network based call signaling protocol specification)/IMS (IP multimedia subsystem) probe/container 624 can be used to test DOCSIS
WAN/MOCA LAN interface 630 of cable modem/ eMTA 602 via CMTS (cable modem termination system) 622 and splitter 620. Provision probe/container 626 can be used to test DOCSIS WAN/MOCA LAN interface 630 of cable modem/eMTA 602 via CMTS 622 and splitter 620. WAN probe/container 628 can be used to test DOCSIS WAN/MOCA LAN interface 630 of cable modem/eMTA 602 via CMTS 622 and splitter 620. The associated core testing processor executes the relevant configured tests for the cable modem/eMTA 602. Status and test results can be sent to the user's dashboard (using JSON format messages as a non-limiting example) via the web-sockets.
[0058] According to certain embodiments, when executing a specific test for a given DUT, the core testing executor/processor loads and reads test configuration information (for example from an XML structure) and identifies the relevant test script that needs to be executed. Inputs that are needed for executing the relevant test script are retrieved and supplied as inputs to the relevant test script. The following is a non-limiting sample script. Create DUT object & Environment Object
Verify Serial Number
Verify Warranty
Check Report Server
Check DUT Staging
Checks for DUT Serial number in Database or Web-service
Get DUT Readiness Information
Checks Web-service for test readiness status of DUT in the test
process
Configure Container Environment
Clear Environment Temp Files
Confirm Factory Reset
Waits for operator to confirm that DUT was factory reset and booted up properly
Check Ethernet LAN connections to DUT
Ping connections: Eth LAN 1 , 2, 3, 4
Fails if any ping to these connections fail
Detect DUT
Checks connection to DUT through socket connection
Reset Password
Operator scans password which is stored temporarily for use in the remainder of test until finished
Login to GUI
Done through web-scraping
Get DUT Information and compare values
Information retrieved through web-scraping
Confirm Power
Confirm all LAN Ethernet LEDs
Confirm WiFi LED
Configure Wireless Network
Through telnet commands
Sets N Mode
Enables Privacy
Sets WPA (Wi-Fi Protected Access) Removes WEP (Wired Equivalent Privacy)
Assigns WiFi Channel to DUT (channel different by slot)
[Channel 1 : slots 1 , 4, 7, 10, 13, 16]
[Channel 6: slots 2, 5, 8, 1 1 , 14]
[Channel 1 1 : slots 3, 6, 9, 12, 15]
Verifies changes through GUI
Disables WiFi once done through telnet
Check Firmware Version and Upgrade Firmware (if needed)
Firmware version varies by model
Cage Closed Confirmation Check
Asks Operator to Close Door on Cage
Connect Wireless Card
Waits on shared Resource Server (located on TC) for Resource L2 (Layer 2) Lock
Lock waiting timeout: 600 sec
All L2 Locks are able to run in parallel but not when any L3 (Layer 3) Lock is running
Obtains Lock
Enables WiFi through telnet
Set WiFi Card
Total Retries allowed: 6 (2 sets of 3 retries) Ping WiFi from DUT
L2 ARP Test on WiFi: must receive 10/10 ARP packets
Total Retries allowed: 6 (2 sets of 3 retries) If either Set WiFi Card or L2 ARP Test Fail after its 3 retries, Ask
Operator to Check Antennas
Performs one more retry in full (set of 3 retries each for Set WiFi Card and L2 ARP WiFi Test) after Check Antennas
Disables WiFi through telnet
Releases Lock
Wireless to LAN Ethernet Speed Test
Waits on shared Resource Server (located on TC) for Resource L3
Lock
Lock waiting timeout: 1800 sec L3 Locks must be run one at a time and when no L2 Lock is running
Obtains Lock
Enables WiFi through telnet
Connects WiFi Card
Iperf3 Speed Test, 5 seconds for UDP Speed Test, 7 seconds for TCP
Speed Test, Sending 200 Mbps Bandwidth
Bandwidth must be greater than 60 Mbps on TCP (Reverse) or 70 Mbps on UDP (Forward)
If Fail after 2 retries, ask operator to Check Antennas Retries up to 2 times more if still Fail
Therefore, Total Retries allowed: 4 (2 sets of 2 retries) Runs sudo iwlist wlanO scan and returns all Wireless Signals seen
Results parsed to print all visible SSIDs and its matching Signal level
Disables WiFi through telnet
Releases Lock
Confirm WPS LED
Confirm LAN Coax LED
Confirm USB 1 +2 LEDs
Confirm US/DS LEDs (upstream/downstream LEDs)
Confirm Online LED
Confirm Telephone LEDs
L2 Test on LAN Ethernet
Arp Test from Eth LAN 1 to Eth LAN 2, 3, 4
Must receive 10/10 on all LAN connections
LAN Ethernet to LAN Ethernet Speed Test
From Eth LAN 1 to Eth LAN 2, 3, 4
Iperf3 Speed Test, 5 seconds Reverse and Forward, (e.g., Sending
1200 Mbps Bandwidth)
Bandwidth must be greater than threshold (e.g., 700 Mbps) Threshold may vary depending on the model
Total Retries allowed: 2
LAN MoCA to LAN Ethernet FTP Speed Test From Eth LAN 1 to LAN MoCA
Iperf3 Speed Test, 5 seconds Reverse and Forward, (e.g., Sending 240
Mbps Bandwidth)
Bandwidth must be greater than threshold (e.g., 60 Mbps) Threshold values may vary depending on the model
Total Retries allowed: 2
LAN Ethernet to WAN DOCSIS FTP Speed Test
From Eth LAN 1 to DOCSIS WAN
Iperf3 Speed Test, 5 seconds Reverse and Forward, (e.g., Sending 1200 Mbps Bandwidth)
Bandwidth must be greater than threshold (e.g., 700 Mbps) Threshold values may vary depending on the model
Total Retries allowed: 2
Voice Testing
Test capability to support incoming and outgoing calls
Test capability to support 2-port, 4-port, and 8-port
Test capability to support NCS and DIP protocols
Clear Persistent Logs
Final Factory Restore
[0059] According to certain embodiments, a system for testing a plurality of devices comprises: a universal tester, at least one test controller, a plurality of sets of testing probes, and a plurality of web sockets, wherein: the plurality of devices includes a plurality of cable modems/eMTA devices; the universal tester is enabled for communication with a platform independent user interface through the plurality of web sockets; the plurality of sets of testing probes comprising: a plurality of LAN probes for testing corresponding LAN interfaces of a cable modem/eMTA device of the plurality of cable modem/eMTA devices, at least one FXS probe for testing a corresponding FXS interface of the cable modem/eMTA device of the plurality of cable modem/eMTA devices, at least one WLAN probe for testing a corresponding WLAN interface of the cable modem/eMTA device of the plurality of cable
modem/eMTA devices, at least one MoCA LAN probe for testing a corresponding MoCA LAN interface of the cable modem/eMTA device of the plurality of cable modem/eMTA devices, at least one NCS/IMS probe for testing a corresponding DOCSIS WAN interface of the cable modem/eMTA device of the plurality of cable modem/eMTA devices, at least one provision probe for testing the corresponding DOCSIS WAN interface of the cable modem/eMTA device of the plurality of cable modem/eMTA devices, at least one WAN probe for testing the corresponding
DOCSIS WAN interface of the cable modem/eMTA device of the plurality of cable modem/eMTA devices; and the plurality of web sockets enable real-time bidirectional and asynchronous communication between the platform independent user interface and the universal tester for simultaneously testing the plurality of devices under test by the universal tester.
[0060] Testing Interface
[0061 ] According to certain embodiments, the testing system provides a separate set of interfaces to be tested for each device that is under testing of the set of devices.
[0062] FIG. 7 illustrates a high-level operator dashboard for interacting with the core testing execution environment, according to certain embodiments. FIG. 7 shows an operator dashboard 702 (user interface) that is capable of asynchronous
communication with a core testing subsystem associated with testing a plurality of devices simultaneously. Operator dashboard 702 includes a plurality of l-Frames 704a-p in HTML (inline frames). The HTML inline frame element represents a nested browsing context, effectively embedding another HTML page into the current page.
[0063] The embodiments are not restricted to the number of l-Frames shown in FIG. 7. The number of l-Frames may vary from implementation to implementation. Each I- Frame corresponds to a slot in the test bench for testing the devices. A device that is to be tested (device under test or DUT) is installed in a slot in the test bench.
According to certain embodiments, different types of devices can be installed in the slots in the test bench for simultaneous testing. In other words, the slots in the test bench are not restricted to testing same types of devices. Disparate devices can be tested simultaneously in the test bench. The respective tests associated with each slot do not interfere with tests running in other slots in the test bench. Non-limiting examples of devices under test (DUTs) include set top boxes, cable modems, embedded multimedia terminal adapters, and wireless routers including broadband wireless routers for the home or for commercial networks. [0064] According to certain embodiments, operator dashboard 702 may be implemented as a neutral platform such as a web-based browser. Such a web-based browser type of operator dashboard can offer flexible access to a user that is at the same location as the test bench or from a laptop, mobile phone, tablet, etc., that is remote from the test bench.
[0065] According to certain embodiments, the HTML based l-Frames in operator dashboard 702 allow a user to send commands and interact with the core testing execution machine with respect to each DUT and independently of other DUTs installed in the test bench such that the user can run tests for all the installed DUTs simultaneously. Further, the user can control and monitor the tests for all the installed DUTs simultaneously using the l-Frames of operator dashboard 702.
According to certain embodiments, the user can configure slot details (e.g., port numbers, IP address for the slot, etc.), configure testing preferences such as push to cloud, export to billing, etc. The l-Frames provide the requisite isolation for executing the tests of each of the DUTs in the test bench simultaneously but independently of each other. In other words, the DUTS installed in the test bench can all be tested in parallel without conflicting with each other.
[0066] FIG. 8 illustrates some components of a sample l-Frame, according to certain embodiments. FIG. 8 shows an l-Frame 802 that includes HTML 804, Java script 806 and a client side web-Socket.10 808. As described herein, each l-Frame in the operator dashboard (user interface) is mapped to one of the slots in the test bench which is completely different from the run-of-the-mill client-server (web) architecture. In the run-of-the-mill client-server (web) architecture, the user makes a request and a corresponding HTML output is served up to the user's browser. In contrast, the operator dashboard with a plurality of l-Frames, each of which is mapped to a DUT in the test bench, can provide real-time continuous feedback to the user for each DUT once the user initiates test execution for the DUTs. For example, the user can use a respective l-Frame to receive feedback such as testing progress and testing results associated with a specific DUT of the plurality of DUTs undergoing parallel testing on the test bench. The user can also interact with the core testing execution machine using the operator dashboard that includes a plurality of l-Frames. For example, the user/test operator might need to provide feedback to the core testing execution machine such as scanning in passwords, providing feedback on certain conditions associated with the test bench and/or core testing machine. As non- limiting examples, the feedback can include information needed for the testing procedure such as factory reset information, cage closed confirmation, Wi-Fi Protected Setup (WPS) LED confirmation, USB LED confirmation, LAN Coax LED confirmation, MoCA WAN LED confirmation, etc. Thus, the user needs to be able to communicate asynchronously with various components of the device testing system. Such asynchronous communication is enabled by the operator dashboard with the plurality of l-Frames and associated web-sockets described in greater detail herein with respect to FIG. 9.
[0067] According to certain embodiments, the core testing machine comprises multiple slots (at the test bench) for installing a DUT in each slot. As a non-limiting example, each DUT in a respective slot is associated with its respective lightweight virtualization container (probes abstraction) and core testing executor/processor. For example, the core testing machine may comprise N core testing servers and each of the N core testing servers may be associated with M core testing
executors/processors. According to certain embodiments, the core testing machine need not have every slot installed with a DUT in order to begin running the tests. The slots are used as needed. Further, the testing of a given DUT can start and finish independently of the other DUTs installed in the test bench of the core testing machine. According to certain embodiments, the use of DUT testing interfaces (probes) through software containers (virtualization containers) can avoid network conflicts while testing multiple DUTs simultaneously by the core testing machine.
[0068] FIG. 9 illustrates a sample architecture showing bi-directional asynchronous communication between the operator dashboard, web-socket layer and core test execution machine, according to certain embodiments. FIG. 9 shows operator dashboard 902 (user interface) including a plurality of l-Frames (904a-d, 910a-d, 916a-d, 922a-d), a plurality of web-sockets (906a-d, 912a-d, 918a-d, 924a-d), and a plurality of test execution environments (908a-d, 914a-d, 920a-d, 926a-d). According to certain embodiments, each l-Frame can communicate asynchronously with a corresponding test testing executor environment. The asynchronous communication can be achieved because the JavaScript Socket. IO on the client side browser dashboard communicates bi-directionally with corresponding web socket (Socket. IO) server-side implementation in Node.JS. In other words, each l-Frame (904a-d, 910a- d, 916a-d, 922a-d) can bi-directionally interact with its corresponding web socket (906a-d, 912a-d, 918a-d, 924a-d) server-side implementation. Each web socket (906a-d, 912a-d, 918a-d, 924a-d) can in turn interact bi-directionally with its corresponding test execution environment (908a-d, 914a-d, 920a-d, 926a-d).
According to certain embodiments, the communication between the l-Frames and the web socket (Socket. IO) server-side uses TCP/IP protocol. According to certain embodiments, the communication between the web sockets (Socket. IO) server-side implementation in Node.JS and the corresponding testing executor environments uses TCP/IP protocol. In the event that the TCI/IP connection is lost, the l-Frame Socket. IO on the client side attempts to reconnect to the web socket server side and displays the status of the connection to the user, accordingly. According to certain embodiments, the core test execution environment maintains the current state of the device testing execution and upon communication reconnection, pushes the state information to the browser implemented l-Frames of the operator dashboard. The foregoing feature allows users to refresh or restart their browser at any time without resulting in loss-of-state. Such a feature also allows the user to stop or abort a given test for a corresponding DUT in the test bench. Such a feature further allows a user to monitor the progress of one or more tests simultaneously using different browser sessions. Such browser sessions can be opened on the same device or on different devices. According to certain embodiments, a browser that supports CSS (cascade style sheets), JavaScript, JQuery (or other suitable cross-platform JavaScript library designed to simplify the client-side scripting of HTML), and client side Socket. IO is used.
[0069] FIG. 10 illustrates a sample server side Node.JS layer, according to certain embodiments. FIG. 4 shows user interface middleware 400 that includes a server side Node.JS layer 1002 and a Socket. IO 1004 (web socket layer). Socket. IO 404 is one implementation of the web socket protocol. As previously described,
communication between the user interface and the core test execution environment is enabled through the web-socket layer. According to certain embodiments, such a web-socket layer can be implemented as a Socket. IO server hosted in Node.JS environment. Such a Socket. IO server is an event-driven server.
[0070] According to certain embodiments, the web socket layer can perform the following: • Enables real-time asynchronous bi-directional communication.
• Provides real-time feedback to users on test execution results, etc., and prompts user for input required for the test execution.
• Hides the core test execution environment from test clients.
• Helps maintain the test execution state with the help of the core testing executor.
[0071 ] According to certain embodiments, it is possible to keep the web socket layer in the cloud such that the device testing can be executed remotely from anywhere. Probes test the following interfaces on the DUT (when such interfaces are available on the DUT):
• Ethernet Local Area Network (LAN): assigned probe runs Ethernet-based connection tests
• Ethernet Wide Area Network (WAN): assigned probe runs Ethernet-based connection tests
• Multimedia over Coax Alliance (MoCA) LAN: assigned probe sets up
MoCA connection, establishes connection, and runs MoCA-related connection tests
• MoCA WAN: assigned probe sets up MoCA connection, establishes
connection, and runs MoCA-related connection tests
• Wireless 2.4 GHz: assigned probe sets up wireless connection,
establishes connection, and runs WiFi-related connection tests on 2.4 GHz frequency
• Wireless 5.0 GHz: assigned probe sets up wireless connection,
establishes connection, and runs WiFi-related connection tests on 5.0 GHz frequency
• Phone ports (FXS): assigned probe sets up phone service simulation, establishes connection, and runs phone-based connection tests
• USB: assigned probe runs USB-functionality tests
• Video: assigned probe runs video-related tests
• Audio: assigned probe runs audio-related tests
[0072] According to certain embodiments, a user interface for a testing machine comprises: a plurality of l-Frames, wherein each l-Frame of at least a subset of the plurality of l-Frames is associated with a respective slot of a plurality of slots on the testing machine for installing, in the respective slot, a respective device under test (DUT) of a plurality of DUTs; and a plurality of client side web sockets associated with the plurality of l-Frames, wherein each client side web socket of at least a subset of the plurality of client side web sockets communicates with a corresponding web socket in a middleware web socket layer for achieving isolation and
independent testing of each respective DUT from other respective DUTs of the plurality of DUTs.
[0073] According to certain embodiments, the middleware web socket layer enables real-time asynchronous communication between the user interface and a core testing environment of the testing machine.
[0074] According to certain embodiments, the middleware web socket layer enables real-time bi-directional communication between the user interface and a core testing environment of the testing machine.
[0075] According to certain embodiments, the client side web sockets communicate with the middleware web socket layer using TCP/IP communication.
[0076] According to certain embodiments, the middleware web socket layer communicates with a core testing environment of the testing machine using TCP/IP communication.
[0077] According to certain embodiments, the middleware web socket layer can be a cloud based implementation.
[0078] Core Testing Machine
[0079] According to certain embodiments, a core testing machine comprises multiple slots for installing a DUT in each slot. As a non-limiting example, each DUT in a respective slot is associated with its respective lightweight virtualization containers (probes abstraction) and core testing executor/processor. For example, the core testing machine may comprise N core testing servers and each of the N core testing servers may be associated with M core testing executors/processors. According to certain embodiments, the core testing machine need not have every slot installed with a DUT in order to begin running the tests. The slots are used as needed. Further, the testing of a given DUT can start and finish independently of the other DUTs installed in the test bench of the core testing machine.
[0080] According to certain embodiments, FIG. 1 1 a high-level flow chart that illustrates some steps performed by a core testing executor/processor for testing devices, according to certain embodiments. According to certain embodiments, the core testing executor/processor is associated with a server. According to certain other embodiments, the core testing executor is special processor. At block 1 100, a user installs one or more devices to be tested into test bench of the core testing machine for testing the devices. According to certain embodiments, at block 1 102, the user scans the barcode (or other identification) of each device to be tested. A device that is to be tested using the core testing executor/processor is also referred to as "DUT" herein. Each DUT is then submitted for testing at block 1 104. For purposes of convenience, FIG. 1 1 will be described with respect to a single DUT even though the core testing machine is capable of testing multiple DUTs
simultaneously. For a given DUT, the respective core testing executor/processor receives a corresponding serial number information and validates the corresponding DUT at block 1 106. At block 1 108, the core testing executor/processor retrieves device information such as make, model, customer, etc. of a given DUT based on the serial number information from a database or web service, for example. At block 1 1 10, the core testing executor/processor loads the specific test configuration information corresponding to the given DUT. Each DUT type (based on make, model, etc.) may be associated with different test configuration information. Each test configuration information includes a set of testing steps. At block 1 1 12, the core testing executor/processor begins to read a testing step of the test configuration information for a given DUT. At block 1 1 14, the core testing executor/processor executes the test step that was read at block 1 1 12. At block 1 1 16, the core testing executor/processor records the result of the executed test step. At block 1 1 18, the core testing executor/processor determines whether the DUT passed or failed the executed test step. If the DUT failed the executed test step at block 1 1 18, then the core testing executor/processor determines whether to abort the testing procedure (based on the test configuration) at block 1 120. If the core testing executor/processor determines to abort the testing procedure at block 1 120, then the DUT is deemed to have failed the test step at block 1 122. If the core testing executor/processor determines not to abort the testing procedure at block 1 120, or if the DUT passed the executed test step at block 1 1 18, then control passes to block 1 124 where the core testing executor/processor determines whether there is another test step to be executed from the set of testing steps for the given DUT. If it is determined that there is another test step to be executed, then control passes back to block 1 1 12. If it is determined that there are no more test steps in the set of test steps to be executed for the given DUT, then the DUT is deemed to have passed the test procedure at block 1 126, according to certain embodiments of the invention. Upon completion of each test step for a given DUT, a message of the completion and/or the results of the test step is sent to the user's browser via web-sockets in real-time, for example. Thus, the user interface can show test progress in addition to testing information such as port numbers, IP address for each DUT slot, DUT serial number, and testing preferences related to billing and pushing to the cloud, etc. The use can also provide input associated with a given test (e.g., password). The user (via user interface) can communicate with the core testing executor/processor using asynchronous feedback and interaction. As a non-limiting example, communication may be in the form of JSON messages using TCP/IP protocol, according to certain embodiments. JSON is Java script object notation for transmitting data between the server and web applications.
[0081 ] According to certain embodiments, the core testing machine can test a set of similar types of devices or disparate types of devices simultaneously using a separate set of interfaces for each device that is under test testing. As a non-limiting example, there may be N core testing servers. Each N core testing server may comprise M number of core testing executors/processors. Thus, a total of N multiplied by M number of DUTS can be tested simultaneously (one DUT is each of the N X M slots, for example). According to certain embodiments, the use of DUT testing interfaces (probes) through virtualization containers can avoid network conflicts while testing multiple DUTs simultaneously by the core testing machine. According to certain embodiments, the core testing servers and core testing executors/processors (and other components) in the testing system may be distributed over a plurality of computers.
[0082] FIG. 12 is a high-level schematic that illustrates DUT probes through the use of software containers (virtualization containers), according to certain embodiments. FIG. 12 shows core test executor/processor 1202, slot 1204, and DUTs 1206 and 1208. There may be more than two DUTs but only two of them are shown in FIG. 12 for convenience. Slot 1204 includes as non-limiting examples, Ethernet wide area network (WAN) probes 1210a, 1216a, Ethernet local area network (LAN) probes 1212a, 1218a and a multimedia over coax alliance (MoCA) probes 1214a, 1220a (MoCA probes can be WAN or LAN, for example). Depending on the nature of the DUT and the DUT's corresponding test configuration, there may also be wireless probes via antenna (WiFi probes, for example). Slot 1204 are connected to the interfaces of DUT 1206 and DUT 1208 respectively. For example, WAN probe 1210a is connected to WAN port 1210b of DUT 1206. LAN probe 1212a is connected to LAN port 1212b of DUT 1206. MoCA probe 1214a is connected to MoCA port 1214b of DUT 1206. Similarly, WAN probe 1216a is connected to WAN port 1216b of DUT 1208. LAN probe 1218a is connected to LAN port 1218b of DUT 1208. MoCA probe 1220a is connected to MoCA port 1220b of DUT 1208.
[0083] Probes test the following interfaces on the DUT (when such interfaces are available on the DUT):
• Ethernet Local Area Network (LAN): assigned probe runs Ethernet-based connection and speed tests
• Ethernet Wide Area Network (WAN): assigned probe runs Ethernet-based connection and speed tests
• Multimedia over Coax Alliance (MoCA) LAN: assigned probe sets up
MoCA connection, establishes connection, and runs MoCA-related connection and speed tests
• MoCA WAN: assigned probe sets up MoCA connection, establishes
connection, and runs MoCA-related connection and speed tests
• Wireless 2.4 GHz: assigned probe sets up wireless connection,
establishes connection, and runs WiFi-related connection tests on 2.4 GHz frequency
• Wireless 5.0 GHz: assigned probe sets up wireless connection,
establishes connection, and runs WiFi-related connection tests on 5.0 GHz frequency
• Phone ports (FXS): assigned probe sets up phone service simulation, establishes connection, and runs phone-based connection tests • USB: assigned probe runs USB-functionality tests
• Video: assigned probe runs video-related tests
• Audio: assigned probe runs audio-related tests
[0084] According to certain embodiments, a system for testing device comprises: a testing machine with a plurality of slots, wherein each slot of the plurality of slots is for installing a device-under-test (DUT) of a plurality of DUTs; a plurality of core testing processors, wherein each core testing processor of the plurality of core testing processors is associated with a respective slot of the plurality of slots; a plurality of lightweight virtualization containers, where a respective lightweight virtualization container of the plurality of lightweight virtualization containers is associated with one of the slots that might have DUT installed, wherein the plurality of lightweight virtualization containers enable isolation of respective testing processes and testing resources associated with each respective device-under-test.
[0085] According to certain embodiments, the plurality of lightweight virtualization containers comprise testing probes for testing a respective DUT of the plurality of DUTs. Virtualization containers can also be referred to as probes herein.
[0086] According to certain embodiments, the plurality of lightweight virtualization containers are used for testing one or more DUT interfaces at the DUT comprising: Ethernet Local Area Network (LAN) interface; Ethernet Wide Area Network (WAN) interface; Multimedia over Coax Alliance (MoCA) LAN interface; Multimedia over Coax Alliance (MoCA) WAN interface; Wireless 2.4 GHz interface; Wireless 5.0 GHz interface; Phone ports (FXS) interface; USB interface; video interface; and audio interface.
[0087] According to certain embodiments, each core testing processor of at least a subset of the plurality of core testing processors is associated with a respective web socket for communication that is isolated and independent of communication associated with other core testing processors of the plurality of core testing processors.
[0088] According to certain embodiments, a respective core testing processor of the plurality of core testing processors communicates with a user interface. [0089] According to certain embodiments, a respective core testing processor of the plurality of core testing processors communicates using asynchronous feedback and interaction.
[0090] According to certain embodiments, a respective core testing processor of the plurality of core testing processors communicates using JSON messages.
[0091 ] According to certain embodiments, the respective core testing processor of the plurality of core testing processors communicates using TCP/IP protocol.
[0092] According to certain embodiments, the respective core testing processor of the plurality of core testing processors: retrieves at run time a respective test configuration corresponding to the DUT installed in the respective slot associated with respective core testing processor; loads the set of tests associated with the DUT installed in the respective slot associated with respective core testing processor; and executes the loaded set of tests.
[0093] Test Sequences
[0094] According to certain embodiments, a central resource control server controls resources such as locks on WiFi resources (herein also referred to as "WiFi resource lock"). Non-limiting examples of WiFi resource locks include locks on resources for WiFi L2 (Layer 2) tests, WiFi L3 (Layer 3) tests and DOCSIS (Data Over Cable Service Interface Specification) tests. For example, a WiFi L2 resource lock involves a lock on resources for the Address Resolution Protocol (ARP) test. Another example of a WiFi L2 resource lock involves a lock on resources for a WiFi connecting card test. A non-limiting example of a WiFi L3 resource lock involves a lock on resources for a WiFi speed test. A non-limiting example of a DOCSIS resource lock involves a lock on resources for a DOCSIS speed test. Layer 2 and Layer 3 refer to the layers in the OSI model (Open System Interconnection model). Layer 2 is the data link layer of the OSI model. Layer 3 is the network layer of the OSI model.
[0095] According to certain embodiments, a testing system that provides a separate set of interfaces for each device (of a plurality of devices) that is under testing can perform WiFi L2 and WiFi L3 tests in a manner that minimizes or avoids wireless interference. [0096] According to certain embodiments, the testing system minimizes or avoids wireless interference by controlling locks on WiFi frequency channels.
[0097] According to certain embodiments, the testing system comprises at least one test station. According to certain embodiments, each test station includes a plurality of physical slots for testing devices. As a non-limiting example, a subset of the plurality of physical slots is associated with a corresponding test server. As a non- limiting example, a test station may have four test servers, each of which is associated with a set of four physical slots of the plurality of physical slots. The embodiments are not restricted to four test servers and four physical slots per test server. The number of test servers and physical slots may vary from implementation to implementation. According to certain embodiments, each test server includes virtuahzation containers that act as probes for testing devices installed in the physical slots in the test station.
[0098] According to certain embodiments, the testing system includes a central resource control server running on at least one test control computer.
[0099] According to certain embodiments, each physical slot on the test station is assigned a specific frequency Channel on the WiFi frequency band. For the WiFi 2.4 GHz band, a given physical slot can be assigned any of the frequency Channels 1 , 6, 1 1 (these channels are non-overlapping), according to certain embodiments. For the WiFi 5.0 GHz band, a given physical slot can be assigned any of the frequency Channels 36, 40, 44, 48, 149, 153, 157, 161 , according to certain embodiments. Such resources (WiFi frequency Channels) are shared across the test servers and associated slots of the test station.
[00100] According to certain embodiments, each slot of the test station is assigned a frequency Channel in a manner to minimize wireless interference between the slots.
[00101 ] According to certain embodiments, only one WiFi L3 test can be performed per Channel in the test station.
[00102] According to certain embodiments, the central resource control server determines whether to grant WiFi resources to a given slot on a test station when the given slot requests a WiFi resource. As a non-limiting example, a given slot on the test station may request a lock on frequency Channel for performing a WiFi L2 test or for performing a WiFi L3 test. [00103] According to certain embodiments, the central resource control server determines whether to grant locks (on a given Channel), release locks and block slots from running specific WiFi tests based on certain criteria.
[00104] The WiFi frequency band may vary from implementation to implementation.
[00105] For ease of explanation, FIG. 13 to FIG. 22 are described with reference to the WiFi 2.4 GHz band. The embodiments are not restricted to the WiFi 2.4 GHz band. Further, FIG. 13 to FIG. 22 are described using example test scenarios comprising sample requests from a sample subset of slots for WiFi resource locks. Such test scenarios are merely examples that help illustrate features of the
embodiments. Such examples are to be regarded in an illustrative rather than a restrictive sense. Other examples may include different requests from a different sample subset of slots for WiFi resource locks.
[00106] FIG. 13 illustrates a high-level schematic of a test station for testing devices such as wireless devices, according to certain embodiments. FIG. 13 shows a test station 1300. According to certain embodiments, test station 1300 comprises a central resource control server 1302 that runs on a test control computer 1304. Test station 1300 further comprises a plurality of test servers 1306a, 1306b, 1306c, 1306d. Test station 1300 comprises a plurality of physical slots. According to certain embodiments, each test sever is associated with four physical slots but the
embodiments are not restricted to four slots per test server. For example, test server 1306a is associated with physical slots such as Slot 1 (1308a), Slot 2 (1308b), Slot 3 (1308c), Slot 4 (1308d). Test server 1306b is associated with physical slots such as Slot 5 (1310a), Slot 6 (1310b), Slot 7 (1310c), Slot 8 (131 Od). Test server 1306c is associated with physical slots such as Slot 9 (1312a), Slot 10 (1312b), Slot 1 1 (1312c), Slot 12 (1312d). Test server 1306d is associated with physical slots such as Slot 13 (1313a), Slot 14 (1313b), Slot 15 (1313c), Slot 16 (1313d). According to certain embodiments, central resource control server 1302 controls WiFi resources such as WiFi L2 Channel 1 lock (1314), WiFi L3 Channel 1 lock (1316), WiFi L2 Channel 6 lock (1318), and WiFi L3 Channel 6 lock (1320),
[00107] According to certain embodiments, each physical slot is assigned a Channel. For example, Slot 1 (1308a) is assigned Channel 1 , Slot 2 (1308b) is assigned Channel 6, Slot 3 (1308c) is assigned Channel 1 , Slot 4 (1308d) is assigned Channel 6, Slot 5 (1310a) is assigned Channel 6, Slot 6 (131 Ob) is assigned Channel 1 , Slot 7 (1310c) is assigned Channel 1 1 , Slot 8 (131 Od) is assigned Channel 6, Slot 9
(1312a) is assigned Channel 1 , Slot 10 (1312b) is assigned Channel 1 1 , Slot 1 1 (1312c) is assigned Channel 1 , Slot 12 (1312d) is assigned Channel 6, Slot 13 (1313a) is assigned Channel 1 1 , Slot 14 (1313b) is assigned Channel 6, Slot 15 (1313c) is assigned Channel 1 , Slot 16 (1313d) is assigned Channel 6, according to certain embodiments.
[00108] When a given slot has a WiFi device installed in the given slot for testing, the given slot requests for a WiFi resource lock from central resource control server 1302 in order to perform a relevant wireless test on the installed device. According to certain embodiments, central resource control server 1302 grants WiFi resource locks in manner that minimizes interference between the various wireless tests that are running in the slots in test station 1300.
[00109] FIG. 14 illustrates requests for WiFi resource locks, according to certain embodiments. Slot 1 (1308a) requests (1322) a lock on frequency Channel 1 in order to perform a WiFi L2 test on a device installed in Slot 1 for testing. Slot 2 (1308b) requests (1324) a lock on frequency Channel 6 in order to perform a WiFi L2 test on a device installed in Slot 2 for testing. Slot 3 (1308c) requests (1326) a lock on frequency Channel 1 in order to perform a WiFi L2 test on a device installed in Slot 3 for testing. Slot 4 (1308d) requests (1328) a lock on frequency Channel 6 in order to perform a WiFi L2 test on a device installed in Slot 4 for testing. Slot 5 (1310a) requests (1332) a lock on frequency Channel 6 in order to perform a WiFi L3 test on a device installed in Slot 5 for testing. Slot 6 (1310b) requests (1330) a lock on frequency Channel 1 in order to perform a WiFi L2 test on a device installed in Slot 6 for testing. The above requests from the slots for locks on frequency Channels are sent to central resource control server 1302.
[001 10] FIG. 15 illustrates the response by central resource control server 1302 to the requests from the slots in the test station, according to certain embodiments.
According to certain embodiments, central resource control server 1302 grants all the requests for locks for a given frequency Channel with respect to WiFi L2 tests as long as there is no ongoing WiFi L3 test for the given frequency Channel, as illustrated by FIG. 15. FIG. 15 shows that Slot 1 (1308a) is granted (1322a) a lock (1322aa) on frequency Channel 1 by central resource control server 1302 in order to perform a WiFi L2 test on a device installed in Slot 1 for testing. Similarly, Slot 2 (1308b) is granted (1324a) a lock (1324aa) on frequency Channel 6 in order to perform a WiFi L2 test on a device installed in Slot 2 for testing. Slot 3 (1308c) is granted (1326a) a lock (1326aa) on frequency Channel 1 in order to perform a WiFi L2 test on a device installed in Slot 3 for testing. Slot 4 (1308d) is granted (1328a) a lock (1328aa) on frequency Channel 6 in order to perform a WiFi L2 test on a device installed in Slot 4 for testing. Slot 6 (131 Ob) is granted (1330a) a lock (1330aa) on frequency Channel 1 in order to perform a WiFi L2 test on a device installed in Slot 6 for testing.
[001 1 1 ] However, Slot 5 (1310a) is not granted (1332a) a lock (blocked 132aa) on frequency Channel 6 in order to perform a WiFi L3 test on a device installed in Slot 5 for testing. Slot 5 (1310a) is not granted (1332a) a lock on frequency Channel 6 because a WiFi L3 test can only be run in a given Channel if there are no other tests running in the same Channel at a given time. In this example, Slot 5 (1310a) cannot be granted (1332a) a lock on frequency Channel 6 for a WiFi L3 test because Slot 2 (1308b), and Slot 4 (1308d) have already been granted locks on frequency Channel 6 for performing their respective WiFi L2 tests. Thus, Slot 5 (1310a) needs to request a lock on frequency Channel 6 to perform a WiFi L3 test at a later time, according to certain embodiments. According to certain other embodiments, Slot 5 (1310a) can keep its request for the lock pending in a request queue. Central resource control server 1302 will grant locks based on the next request in the queue as long the lock grant does not cause more than one WiFi L3 test to be run per frequency Channel.
[001 12] FIG. 16 illustrates the release of locks on frequency Channels for WiFi tests, according to certain embodiments. FIG. 16 shows that once a given slot completes its respective WiFi test, the given slot releases its lock on the frequency Channel that it was granted to perform the WiFi test or the given slot sends information to central resource control server 1300 that the given lock can be released. FIG. 16 shows that Slot 1 (1308a) releases (1322b) its lock on frequency Channel 1 because it has completed (1322bb) the WiFi L2 test on a device installed in Slot 1 for testing. Slot 2 (1308b) releases (1324b) its lock on frequency Channel 6 because it has completed (1324bb) the WiFi L2 test on a device installed in Slot 2 for testing. Slot 3 (1308c) releases (1326b) its lock on frequency Channel 1 because it has completed
(1326bb) the WiFi L2 test on a device installed in Slot 3 for testing. Slot 4 (1308d) releases (1328b) its lock on frequency Channel 6 because it has completed
(1328bb) the WiFi L2 test on a device installed in Slot 4 for testing. Slot 6 (131 Ob) releases (1330b) its lock on frequency Channel 1 because it has completed
(1330bb) the WiFi L2 test on a device installed in Slot 6 for testing. Each of such slots sends information on test completion to central resource control server 1302, according to certain embodiments.
[001 13] The request from Slot 5 (1310a) for a lock on frequency Channel 6 was previously blocked (1332aa). However, Slot 5 (1310a) can again request (1332) for a lock on Channel 6 to perform a WiFi L3 test now that the other locks on Channel 6 are released, according to certain embodiments.
[001 14] FIG. 17 illustrates the granting of a lock on a frequency Channel to a given slot for performing a WiFi L3 test, according to certain embodiments. FIG. 17 shows that central resource control server 1302 grants (1332b) the request from Slot 5 (1310a) for a lock (1332bb) on Channel 6 to perform a WiFi L3 test now that the other locks on Channel 6 are released, according to certain embodiments.
[001 15] FIG. 18 illustrates the release of a lock on a frequency Channel for a WiFi L3 test, according to certain embodiments. FIG. 18 shows that once Slot 5 (1310a) has successfully completed (1332cc) its WiFi L3 test using its lock on frequency Channel 6, Slot 5 (1310a) releases (1332c) its lock on frequency Channel 6 and sends information on test completion to central resource control server 1302. Now, central resource control server 1302 is free to grant multiple locks on a given Channel for WiFi L2 tests or a single lock for a WiFi L3 test per channel (since WiFi L3 test must run alone per Channel) in response to requests for locks from slots on test station 1300.
[001 16] FIG. 19 illustrates the requests from multiple slots for locks on Channels in order to perform respective WiFi L3 tests, according to certain embodiments. FIG. 19 shows that after performing one or more WiFi L2 tests successfully (a slot may need to perform more than one WiFi L2 test, for example), a given slot can proceed to perform a WiFi L3 test, if desired.
[001 17] In FIG. 19, Slot 1 (1308a) requests (1336) a lock on frequency Channel 1 in order to perform a WiFi L3 test on a device installed in Slot 1 for testing. Slot 2 (1308b) requests (1338) a lock on frequency Channel 6 in order to perform a WiFi L3 test on a device installed in Slot 2 for testing. Slot 3 (1308c) requests (1340) a lock on frequency Channel 1 in order to perform a WiFi L3 test on a device installed in Slot 3 for testing. Slot 4 (1308d) requests (1342) a lock on frequency Channel 6 in order to perform a WiFi L3 test on a device installed in Slot 4 for testing. Slot 6 (1310b) requests (1344) a lock on frequency Channel 1 in order to perform a WiFi L3 test on a device installed in Slot 6 for testing. The above requests from the slots for locks on frequency Channels are sent to central resource control server 1302.
According to certain embodiments, central resource control server 1302 will grant only one lock per frequency Channel for WiFi L3 tests in order to ensure minimum wireless interference per frequency Channel when several WiFi tests are run at the same time using various frequency Channels, as described further herein with reference to FIG. 20.
[001 18] FIG. 20 illustrates that the central resource control server will grant only one lock per frequency Channel for WiFi L3 tests, according to certain embodiments. FIG. 20 shows that central resource control server 1302 grants (1336a), to Slot 1 (1308a), a lock (1336aa) on frequency Channel 1 so that Slot 1 (1308a) can perform a WiFi L3 test on a device installed in Slot 1 for testing. FIG. 20 also shows that central resource control server 1302 grants (1342a), to Slot 4 (1308d), a lock
(1342aa) on frequency Channel 6 so that Slot 4 (1308d) can perform a WiFi L3 test on a device installed in Slot 4 for testing. Since only one WiFi L3 test can be run per frequency Channel and central resource control server 1302 has already granted one lock on Channel 1 and one lock on Channel 6 for WiFi L3 tests, central resource control server 1302 rejects (1338a, 1340a, 1344a) the respective requests of Slot 2 (1308b), Slot 3 (1308c) and Slot 6 (1310b). Thus, such requested locks are blocked (1338aa, 1340aa, 1344aa). According to certain embodiments, central resource control server 1302 grants requests for locks on frequency Channels on a first-come- first-served basis. According to certain other embodiments, central resource control server 1302 grants requests for locks on frequency Channels based on business rules. Since Slot 2 (1308b), Slot 3 (1308c) and Slot 6 (1310b) were denied locks for their respective WiFi L3 tests, Slot 2 (1308b), Slot 3 (1308c) and Slot 6 (1310b) need to request again for locks for their respective WiFi L3 tests from central resource control server 1302 at a later time as described with reference to FIG. 21 herein. [001 19] FIG. 21 illustrates the release of locks on a frequency Channels after completion of WiFi L3 tests, according to certain embodiments. FIG. 21 shows that once Slot 1 (1308a) has successfully completed (1336bb) its WiFi L3 test using its lock on frequency Channel 1 , Slot 1 (1308a) releases (1336b) its lock on frequency Channel 1 and sends information on test completion to central resource control server 1302. Similarly, when Slot 4 (1308c) has successfully completed (1342bb) its WiFi L3 test using its lock on frequency Channel 6, Slot 4 (1308d) releases (1342b) its lock on frequency Channel 6 and sends information on test completion to central resource control server 1302. Now, central resource control server 1302 can grant locks on Channel 1 and Channel 6 respectively for WiFi L3 tests to slots that request such locks. Also, since no WiFi L3 tests are running, central resource control server 1302 can grant multiple locks on the frequency Channels for WiFi L2 tests (if needed) in response to requests for locks from slots on test station 1300. Since Slot 2 (1308b), Slot 3 (1308c) and Slot 6 (1310b) were previously denied locks for their respective WiFi L3 tests, Slot 2 (1308b), Slot 3 (1308c) and Slot 6 (1310b) can now request (1338, 1340, 1344) for locks for their respective WiFi L3 tests from central resource control server 1302, according to certain embodiments.
[00120] FIG. 22 illustrates the grant of a recently released lock on a frequency
Channel to the next slot request in the queue, according to certain embodiments. Continuing with the above example of FIG. 20 and FIG. 21. As explained with reference to FIG. 20, when Slot 1 (1308a) and Slot 4 (1308d) release their respective locks on frequency Channel 1 and Channel 6 respectively, central resource control server 1302 can grant L3 locks to the next request for an L3 lock per frequency Channel. As previously described with reference to FIG. 20, the requests for L3 locks from Slot 2 (1308b), Slot 3 (1308c) and Slot 6 (1310b) were denied. Assume that the request for an L3 lock on Channel 1 from Slot 6 (1310b) precedes the request for an L3 lock on Channel 1 from Slot 3 (1308c). Thus, central resource control server 1302 grants (1344b) Slot 6 (1310b) a lock (1344bb) on Channel 1 for its WiFi L3 test while denying (1340a) the request from Slot 3 (1308c) for an L3 lock (blocked 1340aa) on Channel 1 . Further, central resource control server 1302 grants (1338b) to Slot 2 (1310b) a lock (1338bb) on Channel 6 for its WiFi L3 test since there is no other pending L3 lock or pending L2 lock on Channel 6, according to certain embodiments. [00121] In the foregoing specification, embodiments of the invention have been described with reference to numerous specific details that may vary from
implementation to implementation. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.

Claims

1 . A system for testing a plurality of devices, the system comprising:
a universal tester;
at least one test controller;
a plurality of sets of testing probes;
a plurality of web sockets;
wherein,
the universal tester is enabled for communication with a platform
independent user interface through the plurality of web sockets; each respective set of testing probes of at least a subset of the plurality of sets of testing probes are for communicating with respective ports of a respective device under test of the plurality of devices installed in the universal tester for purposes of testing; and the plurality of web sockets enable real-time bi-directional and
asynchronous communication between the platform
independent user interface and the universal tester for simultaneously testing the plurality of devices under test by the universal tester.
2. The system of Claim 1 , wherein the real-time bi-directional and asynchronous communication of the plurality of web sockets enable a user to control the testing of the plurality of devices simultaneously but asynchronously and independently of each other using the universal tester.
3. The system of Claim 1 , wherein the plurality of devices installed in the universal tester for purposes of simultaneous testing comprise a set of disparate devices.
4. The system of Claim 1 , wherein the plurality of devices installed in the universal tester for purposes of simultaneous testing comprise a set of similar devices.
5. The system of Claim 1 , further being adaptable to augmenting the test controller, the plurality of web sockets, the user interface and the plurality of sets of testing probes to accommodate testing of new types of devices.
6. A system for testing a plurality of devices, the system comprising:
a universal tester;
at least one test controller;
a plurality of sets of testing probes;
a plurality of web sockets;
wherein,
the plurality of devices includes a plurality of set top boxes; the universal tester is enabled for communication with a platform
independent user interface through the plurality of web sockets; the plurality of sets of testing probes comprising:
at least one HDMI probe for testing a corresponding HDMI interface of a set top box of the plurality of set top boxes; at least one audio video probe for testing a corresponding audio video interface of the set top box of the plurality of set top boxes;
at least one audio video probe for testing a corresponding coax TV output interface of the set top box of the plurality of set top boxes;
at least one IR probe for testing a corresponding IR interface of the set top box of the plurality of set top boxes;
at least one CATV coax probe for testing a corresponding coax interface of the set top box of the plurality of set top boxes; and
the plurality of web sockets enable real-time bi-directional and
asynchronous communication between the platform
independent user interface and the universal tester for simultaneously testing the plurality of devices by the universal tester.
7. The system of Claim 6, wherein the real-time bi-directional and asynchronous communication of the plurality of web sockets enable a user to control the testing of the plurality of devices simultaneously but asynchronously and independently of each other using the universal tester.
8. The system of Claim 6, wherein the plurality of devices installed in the universal tester for purposes of simultaneous testing comprise a set of disparate devices.
9. The system of Claim 6, wherein the plurality of devices installed in the universal tester for purposes of simultaneous testing comprise a set of similar devices.
10. The system of Claim 6, further being adaptable to augmenting the test controller, the plurality of web sockets, the user interface and the plurality of sets of testing probes to accommodate testing of new types of devices.
1 1 . A system for testing a plurality of devices, the system comprising:
a universal tester;
at least one test controller;
a plurality of sets of testing probes;
a plurality of web sockets;
wherein,
the plurality of devices includes a plurality of wireless routers;
the universal tester is enabled for communication with a platform
independent user interface through the plurality of web sockets; the plurality of sets of testing probes comprising:
a plurality of LAN probes for testing corresponding LAN
interfaces of a wireless router of the plurality of wireless routers;
at least one WLAN probe for testing a corresponding WLAN interface of the wireless router of the plurality of wireless routers; at least one Ethernet WAN probe for testing a corresponding
WAN interface of the wireless router of the plurality of wireless routers;
at least one MoCA LAN probe for testing a corresponding coax interface of the wireless router of the plurality of wireless routers;
at least one MoCA WAN probe for testing a corresponding coax interface of the wireless router of the plurality of wireless routers; and
the plurality of web sockets enable real-time bi-directional and
asynchronous communication between the platform
independent user interface and the universal tester for simultaneously testing the plurality of devices by the universal tester.
12. The system of Claim 1 1 , further comprising a MoCA LAN bridge.
13. The system of Claim 1 1 , further comprising a MoCA WAN bridge.
14. The system of Claim 1 1 , further comprising a splitter.
15. The system of Claim 1 1 , wherein the real-time bi-directional and
asynchronous communication of the plurality of web sockets enables a user to control the testing of the plurality of devices simultaneously but asynchronously and independently of each other using the universal tester.
16. The system of Claim 1 1 , wherein the plurality of devices installed in the universal tester for purposes of simultaneous testing comprise a set of disparate devices.
17. The system of Claim 1 1 , wherein the plurality of devices installed in the universal tester for purposes of simultaneous testing comprise a set of similar devices.
18. The system of Claim 1 1 , further being adaptable to augmenting the test controller, the plurality of web sockets, the user interface and the plurality of sets of testing probes to accommodate testing of new types of devices.
19. A system for testing a plurality of devices, the system comprising:
a universal tester;
at least one test controller;
a plurality of sets of testing probes;
a plurality of web sockets;
wherein,
the plurality of devices includes a plurality of cable modem/eMTA
devices;
the universal tester is enabled for communication with a platform
independent user interface through the plurality of web sockets; the plurality of sets of testing probes comprising:
a plurality of LAN probes for testing corresponding LAN
interfaces of a cable modem/eMTA device of the plurality of cable modem/eMTA devices;
at least one FXS probe for testing a corresponding FXS
interface of the cable modem/eMTA device of the plurality of cable modem/eMTA devices;
at least one WLAN probe for testing a corresponding WLAN interface of the cable modem/eMTA device of the plurality of cable modem/eMTA devices;
at least one MoCA LAN probe for testing a corresponding MoCA
LAN interface of the cable modem/eMTA device of the plurality of cable modem/eMTA devices;
at least one NCS/IMS probe for testing a corresponding
DOCSIS WAN interface of the cable modem/eMTA device of the plurality of cable modem/eMTA devices; at least one provision probe for testing the corresponding
DOCSIS WAN interface of the cable modem/eMTA device of the plurality of cable modem/eMTA devices; at least one WAN probe for testing the corresponding DOCSIS
WAN interface of the cable modem/eMTA device of the plurality of cable modem/eMTA devices; and
the plurality of web sockets enable real-time bi-directional and
asynchronous communication between the platform
independent user interface and the universal tester for simultaneously testing the plurality of devices by the universal tester.
20. The system of Claim 19, further comprising a MoCA LAN bridge.
21 . The system of Claim 19, further comprising a cable modem termination system.
22. The system of Claim 19, further comprising a splitter.
23. The system of Claim 19, wherein the real-time bi-directional and
asynchronous communication of the plurality of web sockets enable a user to control the testing of the plurality of devices simultaneously but asynchronously and independently of each other using the universal tester.
24. The system of Claim 19, wherein the plurality of devices installed in the universal tester for purposes of simultaneous testing comprise a set of disparate devices.
25. The system of Claim 19, wherein the plurality of devices installed in the universal tester for purposes of simultaneous testing comprise a set of similar devices.
26. The system of Claim 19, further being adaptable to augmenting the test controller, the plurality of web sockets, the user interface and the plurality of sets of testing probes to accommodate testing of new types of devices.
27. A user interface for a testing machine, the user interface comprising:
a plurality of l-Frames, wherein:
each l-Frame of at least a subset of the plurality of l-Frames is
associated with a respective slot of a plurality of slots on the testing machine for installing, in the respective slot, a respective device under test (DUT) of a plurality of DUTs; and
a plurality of client side web sockets associated with the plurality of l-Frames, wherein:
each client side web socket of at least a subset of the plurality of client side web sockets communicates with a corresponding web socket in a middleware web socket layer for achieving isolation and independent testing of each respective DUT from other respective DUTs of the plurality of DUTs.
28. The user interface of Claim 27, wherein the middleware web socket layer enables real-time asynchronous communication between the user interface and a core testing environment of the testing machine.
29. The user interface of Claim 27, wherein the middleware web socket layer enables real-time bi-directional communication between the user interface and a core testing environment of the testing machine.
30. The user interface of Claim 27, wherein the client side web sockets communicate with the middleware web socket layer using TCP/IP communication.
31 . The user interface of Claim 27, wherein the middleware web socket layer communicates with a core testing environment of the testing machine using TCP/IP communication.
32. The user interface of Claim 27, wherein the middleware web socket layer can be a cloud based implementation.
33. A system for testing devices, the system comprising:
a testing machine with a plurality of slots, wherein each slot of the plurality of slots is for installing a device-under-test (DUT) of a plurality of DUTs;
a plurality of core testing processors, wherein each core testing processor of the plurality of core testing processors is associated with a respective slot of the plurality of slots;
a plurality of lightweight virtualization containers, where a respective lightweight virtualization container of the plurality of lightweight virtualization containers is associated with an interface of a DUT that is installed for testing, wherein the plurality of lightweight virtualization containers enable isolation of respective testing processes and testing resources associated with each respective device-under-test.
34. The system of Claim 33, wherein the plurality of lightweight virtualization containers comprise testing probes for testing corresponding interfaces of a respective DUT of the plurality of DUTs.
35. The system of Claim 33, wherein the plurality of lightweight virtualization containers are used for testing one or more DUT interfaces at the DUT comprising:
Ethernet Local Area Network (LAN) interface;
Ethernet Wide Area Network (WAN) interface;
Multimedia over Coax Alliance (MoCA) LAN interface;
Multimedia over Coax Alliance (MoCA) WAN interface;
Wireless 2.4 GHz interface;
Wireless 5.0 GHz interface;
Phone ports (FXS) interface;
USB interface;
video interface; and
audio interface.
36. The system of Claim 33, wherein each core testing processor of at least a subset of the plurality of core testing processors is associated with a respective web socket for communication that is isolated and independent of communication associated with other core testing processors of the plurality of core testing processors.
37. The system of Claim 33, wherein a respective core testing processor of the plurality of core testing processors communicates with a user interface.
38. The system of Claim 37, wherein the respective core testing processor of the plurality of core testing processors communicates using asynchronous feedback and interaction.
39. The system of Claim 37, wherein the respective core testing processor of the plurality of core testing processors communicates using JSON messages.
40. The system of Claim 37, wherein the respective core testing processor of the plurality of core testing processors communicates using TCP/IP protocol.
41 . The system of Claim 37, wherein the respective core testing processor of the plurality of core testing processors:
retrieves at run time a respective test configuration corresponding to the DUT installed in the respective slot associated with respective core testing processor; loads the set of tests associated with the DUT installed in the respective slot associated with respective core testing processor; and
executes the loaded set of tests.
42. A testing machine for testing a plurality of devices, the testing machine comprising:
a plurality of slots, wherein:
each slot of at least a subset of the plurality of slots is designed for installing a corresponding device of the plurality of devices to be tested;
each slot of the subset of the plurality of slots is assigned a respective wireless frequency channel for performing one or more wireless tests on each slot's corresponding installed device to be tested; at least one central resource control server, wherein:
the at least one central resource control server controls locking of a plurality of wireless frequency channels;
a respective slot of the subset of the plurality of slots sends a request, to the at least one central resource control server, for a lock on the respective slot's assigned frequency channel when the respective slot needs to perform a wireless test on the respective slot's corresponding installed device to be tested; the at least one central resource control server grants the request for the lock on the respective slot's assigned frequency channel based on a predetermined set of criteria.
43. The testing machine of Claim 42, wherein the predetermined set of criteria comprises: granting a lock on a respective frequency channel for performing Layer 2 wireless tests only if all previously granted locks on the respective frequency channel for performing Layer 3 wireless tests are released.
44. The testing machine of Claim 42, wherein the predetermined set of criteria comprises: granting a lock on a respective frequency channel for performing Layer 3 wireless tests only if all previously granted locks on the respective frequency channel for performing any wireless tests are released.
45. The testing machine of Claim 42, wherein the predetermined set of criteria comprises: granting a lock on a respective frequency channel for performing wireless tests based on a first-come-first-served basis.
46. The testing machine of Claim 42, wherein the predetermined set of criteria comprises: granting a lock on a respective frequency channel for performing wireless tests based on business rules.
47. The testing machine of Claim 42, further comprises a test control computer that is associated with the at least one central resource control server.
PCT/US2016/053768 2015-09-25 2016-09-26 Universal device testing system WO2017053961A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US15/642,915 US10291959B2 (en) 2015-09-25 2017-07-06 Set top boxes under test

Applications Claiming Priority (14)

Application Number Priority Date Filing Date Title
US14/866,752 2015-09-25
US14/866,780 US9491454B1 (en) 2015-09-25 2015-09-25 Set top boxes under test
US14/866,720 2015-09-25
US14/866,630 US9960989B2 (en) 2015-09-25 2015-09-25 Universal device testing system
US14/866,780 2015-09-25
US14/866,720 US9810735B2 (en) 2015-09-25 2015-09-25 Core testing machine
US14/866,630 2015-09-25
US14/866,752 US10122611B2 (en) 2015-09-25 2015-09-25 Universal device testing interface
US14/948,143 US9992084B2 (en) 2015-11-20 2015-11-20 Cable modems/eMTAs under test
US14/948,143 2015-11-20
US14/948,925 US9838295B2 (en) 2015-11-23 2015-11-23 Wireless routers under test
US14/948,925 2015-11-23
US14/987,538 2016-01-04
US14/987,538 US9900116B2 (en) 2016-01-04 2016-01-04 Test sequences using universal testing system

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US14/866,630 Continuation-In-Part US9960989B2 (en) 2015-09-25 2015-09-25 Universal device testing system

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/642,915 Continuation US10291959B2 (en) 2015-09-25 2017-07-06 Set top boxes under test

Publications (1)

Publication Number Publication Date
WO2017053961A1 true WO2017053961A1 (en) 2017-03-30

Family

ID=58387557

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2016/053768 WO2017053961A1 (en) 2015-09-25 2016-09-26 Universal device testing system

Country Status (1)

Country Link
WO (1) WO2017053961A1 (en)

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9810735B2 (en) 2015-09-25 2017-11-07 Contec, Llc Core testing machine
CN107426563A (en) * 2017-05-27 2017-12-01 四川长虹电器股份有限公司 The method for testing TV WIFI performances
US9838295B2 (en) 2015-11-23 2017-12-05 Contec, Llc Wireless routers under test
US9900116B2 (en) 2016-01-04 2018-02-20 Contec, Llc Test sequences using universal testing system
US9900113B2 (en) 2016-02-29 2018-02-20 Contec, Llc Universal tester hardware
US9960989B2 (en) 2015-09-25 2018-05-01 Contec, Llc Universal device testing system
CN108011883A (en) * 2017-12-05 2018-05-08 深圳市创维软件有限公司 A kind of remote debugging method, terminal device and server
US9992084B2 (en) 2015-11-20 2018-06-05 Contec, Llc Cable modems/eMTAs under test
CN108259895A (en) * 2017-12-18 2018-07-06 深圳市双翼科技股份有限公司 set-top box test method, system and terminal device
US10122611B2 (en) 2015-09-25 2018-11-06 Contec, Llc Universal device testing interface
US10158553B2 (en) 2015-09-25 2018-12-18 Contec, Llc Systems and methods for testing electronic devices using master-slave test architectures
US10291959B2 (en) 2015-09-25 2019-05-14 Contec, Llc Set top boxes under test
US10320651B2 (en) 2015-10-30 2019-06-11 Contec, Llc Hardware architecture for universal testing system: wireless router test
US20190182129A1 (en) * 2017-12-11 2019-06-13 Spirent Communications, Inc. Method and system for inducing pseudo https communications between one or more emulated servers and emulated clients to test a device therebetween
US10397304B2 (en) 2018-01-30 2019-08-27 Excentus Corporation System and method to standardize and improve implementation efficiency of user interface content
US10965578B2 (en) 2015-10-30 2021-03-30 Contec, Llc Hardware architecture for universal testing system: cable modem test
CN113064780A (en) * 2021-03-18 2021-07-02 深圳市吉祥腾达科技有限公司 Automatic test system and method based on router product
RU2751143C1 (en) * 2020-07-29 2021-07-08 федеральное государственное автономное образовательное учреждение высшего образования "Северо-Кавказский федеральный университет" Method for automation of sensor calibration of strapdown inertial system of unmanned aerial vehicle
US20220158912A1 (en) * 2020-11-16 2022-05-19 Juniper Networks, Inc. Active assurance of network slices
US11374973B2 (en) 2018-12-20 2022-06-28 Spirent Communications, Inc. Streamlining cryptographic processes in a test environment

Citations (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080315898A1 (en) * 2006-01-19 2008-12-25 International Business Machines Corporation Acquiring Test Data From An Electronic Circuit
US20090089854A1 (en) * 2007-09-27 2009-04-02 Contec Llc Arrangement and method for managing testing and repair of set-top boxes
US20110001833A1 (en) * 2009-07-01 2011-01-06 Spirent Communications, Inc. Computerized device and method for analyzing signals in a multimedia over coax alliance (moca) network and similar tdm / encrypted networks
US20110006794A1 (en) * 2008-02-27 2011-01-13 Scanimetrics Inc. Method and apparatus for interrogating electronic equipment components
US20110012632A1 (en) * 2009-07-15 2011-01-20 Merrow Brian S Conductive Heating
US20110035676A1 (en) * 2002-07-15 2011-02-10 Steven Tischer Apparatus and Method for Routing Communications Between Networks and Devices
US20110072306A1 (en) * 2009-09-24 2011-03-24 Contec Llc Method and System for Automated Test of End-User Devices
CN202261360U (en) * 2011-09-06 2012-05-30 汉柏科技有限公司 Device for testing robustness of router data path
US20120163227A1 (en) * 2003-08-01 2012-06-28 Opnet Technologies, Inc. Systems and methods for intelligent probe testing
US20120278826A1 (en) * 2011-04-27 2012-11-01 Echostar Technologies L.L.C. Systems and methods for highly scalable automated testing and monitoring of receiving devices
US20130104158A1 (en) * 2011-10-19 2013-04-25 Atc Logistics & Electronics, Inc. System and method for securing and testing set-top boxes
US8515015B2 (en) * 2008-05-09 2013-08-20 Verizon Patent And Licensing Inc. Method and system for test automation and dynamic test environment configuration
WO2013169728A2 (en) * 2012-05-07 2013-11-14 Flextronics Ap, Llc Universal device multi-function test apparatus
US8689071B2 (en) * 2010-08-30 2014-04-01 Contec Holdings, Ltd. Multimedia device test system
US20140156819A1 (en) * 2012-11-30 2014-06-05 Alexandros Cavgalar Communications modules for a gateway device, system and method
US20140207404A1 (en) * 2013-01-24 2014-07-24 Ltx-Credence Corporation Scalable test platform
US20140282783A1 (en) * 2013-03-15 2014-09-18 Certusview Technologies, Llc Hybrid fiber-coaxial (hfc) cable communication systems having well-aligned optical and radio-frequency links to facilitate upstream channel plans having high aggregate data capacity
US20140370821A1 (en) * 2013-06-12 2014-12-18 Apple Inc. Methods and Apparatus for Testing Electronic Devices with Antenna Arrays

Patent Citations (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110035676A1 (en) * 2002-07-15 2011-02-10 Steven Tischer Apparatus and Method for Routing Communications Between Networks and Devices
US20120163227A1 (en) * 2003-08-01 2012-06-28 Opnet Technologies, Inc. Systems and methods for intelligent probe testing
US20080315898A1 (en) * 2006-01-19 2008-12-25 International Business Machines Corporation Acquiring Test Data From An Electronic Circuit
US20090089854A1 (en) * 2007-09-27 2009-04-02 Contec Llc Arrangement and method for managing testing and repair of set-top boxes
US20110006794A1 (en) * 2008-02-27 2011-01-13 Scanimetrics Inc. Method and apparatus for interrogating electronic equipment components
US8515015B2 (en) * 2008-05-09 2013-08-20 Verizon Patent And Licensing Inc. Method and system for test automation and dynamic test environment configuration
US20110001833A1 (en) * 2009-07-01 2011-01-06 Spirent Communications, Inc. Computerized device and method for analyzing signals in a multimedia over coax alliance (moca) network and similar tdm / encrypted networks
US20110012632A1 (en) * 2009-07-15 2011-01-20 Merrow Brian S Conductive Heating
US20110072306A1 (en) * 2009-09-24 2011-03-24 Contec Llc Method and System for Automated Test of End-User Devices
US8689071B2 (en) * 2010-08-30 2014-04-01 Contec Holdings, Ltd. Multimedia device test system
US20120278826A1 (en) * 2011-04-27 2012-11-01 Echostar Technologies L.L.C. Systems and methods for highly scalable automated testing and monitoring of receiving devices
CN202261360U (en) * 2011-09-06 2012-05-30 汉柏科技有限公司 Device for testing robustness of router data path
US20130104158A1 (en) * 2011-10-19 2013-04-25 Atc Logistics & Electronics, Inc. System and method for securing and testing set-top boxes
WO2013169728A2 (en) * 2012-05-07 2013-11-14 Flextronics Ap, Llc Universal device multi-function test apparatus
US20140156819A1 (en) * 2012-11-30 2014-06-05 Alexandros Cavgalar Communications modules for a gateway device, system and method
US20140207404A1 (en) * 2013-01-24 2014-07-24 Ltx-Credence Corporation Scalable test platform
US20140282783A1 (en) * 2013-03-15 2014-09-18 Certusview Technologies, Llc Hybrid fiber-coaxial (hfc) cable communication systems having well-aligned optical and radio-frequency links to facilitate upstream channel plans having high aggregate data capacity
US20140370821A1 (en) * 2013-06-12 2014-12-18 Apple Inc. Methods and Apparatus for Testing Electronic Devices with Antenna Arrays

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
NORDMAN.: "Testing products with network connectivity", 21 June 2011 (2011-06-21), pages 1 - 20, XP055371699, Retrieved from the Internet <URL:http://citeseerx.ist.psu.edu/viewdoc/download?> [retrieved on 19012017] *

Cited By (36)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10291959B2 (en) 2015-09-25 2019-05-14 Contec, Llc Set top boxes under test
US11353507B2 (en) 2015-09-25 2022-06-07 Contec, Llc Core testing machine
US9810735B2 (en) 2015-09-25 2017-11-07 Contec, Llc Core testing machine
US10277497B2 (en) 2015-09-25 2019-04-30 Contec, Llc Systems and methods for testing electronic devices using master-slave test architectures
US10578670B2 (en) 2015-09-25 2020-03-03 Contec, Llc Core testing machine
US9960989B2 (en) 2015-09-25 2018-05-01 Contec, Llc Universal device testing system
US10158553B2 (en) 2015-09-25 2018-12-18 Contec, Llc Systems and methods for testing electronic devices using master-slave test architectures
US10122611B2 (en) 2015-09-25 2018-11-06 Contec, Llc Universal device testing interface
US10298483B2 (en) 2015-09-25 2019-05-21 Contec, Llc Universal device testing interface
US10581719B2 (en) 2015-10-30 2020-03-03 Contec, Llc Hardware architecture for universal testing system: wireless router test
US10965578B2 (en) 2015-10-30 2021-03-30 Contec, Llc Hardware architecture for universal testing system: cable modem test
US10320651B2 (en) 2015-10-30 2019-06-11 Contec, Llc Hardware architecture for universal testing system: wireless router test
US9992084B2 (en) 2015-11-20 2018-06-05 Contec, Llc Cable modems/eMTAs under test
US10230617B2 (en) 2015-11-23 2019-03-12 Contec, Llc Wireless routers under test
US10581718B2 (en) 2015-11-23 2020-03-03 Contec, Llc Wireless devices under test
US9838295B2 (en) 2015-11-23 2017-12-05 Contec, Llc Wireless routers under test
US10116397B2 (en) 2016-01-04 2018-10-30 Contec, Llc Test sequences using universal testing system
US9900116B2 (en) 2016-01-04 2018-02-20 Contec, Llc Test sequences using universal testing system
US9900113B2 (en) 2016-02-29 2018-02-20 Contec, Llc Universal tester hardware
CN107426563B (en) * 2017-05-27 2019-06-14 四川长虹电器股份有限公司 The method for testing TV WIFI performance
CN107426563A (en) * 2017-05-27 2017-12-01 四川长虹电器股份有限公司 The method for testing TV WIFI performances
CN108011883A (en) * 2017-12-05 2018-05-08 深圳市创维软件有限公司 A kind of remote debugging method, terminal device and server
US11070451B2 (en) * 2017-12-11 2021-07-20 Spirent Communications, Inc. Method and system for inducing pseudo HTTPS communications between one or more emulated servers and emulated clients to test a device therebetween
US20190182129A1 (en) * 2017-12-11 2019-06-13 Spirent Communications, Inc. Method and system for inducing pseudo https communications between one or more emulated servers and emulated clients to test a device therebetween
US11824740B2 (en) 2017-12-11 2023-11-21 Spirent Communications, Inc. Method and system for inducing secure communications between one or more emulated servers and emulated clients to test a device therebetween
CN108259895A (en) * 2017-12-18 2018-07-06 深圳市双翼科技股份有限公司 set-top box test method, system and terminal device
CN108259895B (en) * 2017-12-18 2020-05-19 深圳市双翼科技股份有限公司 Set top box testing method and system and terminal equipment
US10938880B2 (en) 2018-01-30 2021-03-02 Excentus Corporation System and method to standardize and improve implementation efficiency of user interface content
US11349902B2 (en) 2018-01-30 2022-05-31 Excentus Corporation System and method to standardize and improve implementation efficiency of user interface content
US11677807B2 (en) 2018-01-30 2023-06-13 Excentus Corporation System and method to standardize and improve implementation efficiency of user interface content
US10397304B2 (en) 2018-01-30 2019-08-27 Excentus Corporation System and method to standardize and improve implementation efficiency of user interface content
US11374973B2 (en) 2018-12-20 2022-06-28 Spirent Communications, Inc. Streamlining cryptographic processes in a test environment
RU2751143C1 (en) * 2020-07-29 2021-07-08 федеральное государственное автономное образовательное учреждение высшего образования "Северо-Кавказский федеральный университет" Method for automation of sensor calibration of strapdown inertial system of unmanned aerial vehicle
US20220158912A1 (en) * 2020-11-16 2022-05-19 Juniper Networks, Inc. Active assurance of network slices
US11936548B2 (en) 2020-11-16 2024-03-19 Juniper Networks, Inc. Active assurance for virtualized services
CN113064780A (en) * 2021-03-18 2021-07-02 深圳市吉祥腾达科技有限公司 Automatic test system and method based on router product

Similar Documents

Publication Publication Date Title
WO2017053961A1 (en) Universal device testing system
US10298483B2 (en) Universal device testing interface
US11353507B2 (en) Core testing machine
US10581718B2 (en) Wireless devices under test
US10291959B2 (en) Set top boxes under test
US9491454B1 (en) Set top boxes under test
US20190182134A1 (en) Cable modems/emtas under test
US9960989B2 (en) Universal device testing system
EP2972932B1 (en) Cloud based virtual mobile device
KR20140021677A (en) Method and apparatus for remote delivery of managed usb services via a mobile computing device
EP2866392B1 (en) Information processing system, information processing method, and communication device
EP3457657B1 (en) Access control method and system, and switch
US20170329739A1 (en) Methods and systems for loading a boot agent on a router network device
US20150222709A1 (en) Facilitating interactive support sessions for an embedded device using a portable device
EP2369893A1 (en) Access device and method of operating an access device
US20160094388A1 (en) Backup Wide Area Network Connection For Access Points And Routers
US20180184283A1 (en) Wireless configuration of wireless distribution system (wds) wi-fi range extenders using non-wi-fi wireless communication channels
US20180175913A1 (en) Provisioning devices using near-field communication
CN116915602A (en) Software upgrading method and system, electronic equipment and storage medium
CN115373927B (en) Product testing method, device, electronic equipment and computer readable medium
US20240080378A1 (en) System and method for expansion of open radio access network
O’Brien et al. Enabling communications for industry 4.0: Private 5g in smart manufacturing
Ramon et al. Using an LTE Modem
US20210168118A1 (en) Communication system and communication method
CN117459582A (en) Method and system for providing remote access for equipment not supporting remote access service

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16849863

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16849863

Country of ref document: EP

Kind code of ref document: A1