GB2433666A

GB2433666A - Set top box testing system

Info

Publication number: GB2433666A
Application number: GB0525977A
Authority: GB
Inventors: Andrew Buckley
Original assignee: NTL Group Ltd
Current assignee: NTL Group Ltd
Priority date: 2005-12-21
Filing date: 2005-12-21
Publication date: 2007-06-27
Also published as: GB0525977D0

Abstract

A set top box (STB) test system 100 comprises an STB command system for providing commands to at least one said STB 108, an STB output interface system 112 for receiving an output from said at least one STB 108, and a test control system 102 coupled to said STB command system and to said STB output interface system. The test control system 102 is configured to control said command system to issue commands to said at least one said STB and to read results 106 of said commands from said STB output interface system, to test said STB.

Description

Test Systems This invention is generally concerned with systems and

methods for testing set top boxes (STBs), and with STBs adapted for automatic testing.

Broadly speaking a set top box (STB) is a device which connects to a television and which receives an external signal and converts this into content for display on the television set. Often the external signal comprises a digital television signal provided via a cable connection (analogue or digital), although other signal sources include a satellite dish, an Ethernet cable, a telephone line, and a conventional VHF or UHF antenna. As well as video and audio data, content provided to the television increasingly includes Internet web pages, games and the like. Typically a set top box has an input to receive an RF (radio frequency) signal and an output to the television, generally a SCART (Syndicat des Constructeurs d'Appareils Radioréceptors et Televiseurs). The content is received in a portion of the RF spectrum on the input referred to as in-band; an out-of-band (OOB) portion of the spectrum may be used for a return path, i.e. for the set top box to send data back towards a system end.

Before being distributed to customers set top boxes are tested. It is desirable during such testing to use the set top box in an environment which realistically mimics that in a customer's home. Furthermore because it is difficult automatically to determine whether a television is displaying one or another TV programme, for example BBCI as opposed to say BBC2, to date testing has been carried out using people with infrared (IR) remote control who watch the TV output. It is desirable, however, to provide a more efficient means of testing set top boxes.

According to the present invention, there is therefore provided a set top box (STB) test system, the system comprising an STB command system for providing commands to at least one said STB, an STB output interface system for receiving an output from said at least one STB, and a test control system coupled to said STB command system and to said STB output interface system and configured to control said command system to issue commands to said at least one said STB and to read results of said commands from said STB output interface system, to test said STB.

In embodiments the STB test system is able to test a plurality of STBs simultaneously.

Preferably the output interface system, as described further below, is configured to receive multiple outputs from each STB, for example via a multiplexer. In embodiments the command system may comprise a system for providing a serial, wired interface such an RS232 interface to each STB, for example by means of a serial distribution unit (SDLJ). However, use of the serial port of an STB may require operation of the STB in a special mode (a so-called shell build) which consequently means that the tested system may not correspond exactly to that used by a customer.

More particularly overload of a serial port can affect the behaviour of the STB. It is therefore preferable that the STB command system includes an infrared (IR) command system for commanding an STB using an infrared remote control sensor of the STB. In other words, in embodiments, the system is equipped with a plurality of IR blasters, effectively one for each STB (a smaller number of blasters may be multiplexed).

Preferably the output interface system includes a composite video analysis (CVA) to receive a composite video output from the STB and to analyse the composite video output to provide video output analysis data for reading by the test control system. This video output analysis data may comprise one or more of text data, graphical data, and screen pattern identification data. In this way text output by the STB for display on the TV screen, such as "booting" or "this channel is not available" may be read to determine a corresponding condition, a combination of text and graphics may be analysed, for example an EPG (electronic programme guide) display such as a pop up now-and-next mini guide, icons such as mute may be identified and (predetermined) screen patterns may be identified. Examples of screen patterns include a "black screen", a splash screen which may be displayed at boot, and other patterns which may be provided for output by the set top box for example in a pre-recorded videostream.

Some information is particularly difficult to extract from a set top box, in particular information relating to the video and audio channel the set top box is outputting, as well as subtitle information, for example, is the STB outputting BBCI as requested?, is the audio also correct for the channel?, are the correct subtitles present?, and so forth.

Preferably this information should also be testable for video on demand.

The inventors have recognised that the (composite) video output from the STB can be used to provide additional information, encoded in the vertical blanking interval (VBI) of the output video. Thus it has been recognised that data can be extracted from an STB by having the STB include test data for the STB in the VBI of the video signal it outputs (as opposed, for example, to receiving a television signal with data such as teletext encoded in the VBI).

Thus in preferred embodiments the output interface system inchides a VBI data extractor to extract STB data from a VBI of the STB video output for reading by the test system.

Since the television may expect to receive data in the VBI, preferably the data is extracted from lines which are not already used by teletext and other similar information services. Thus preferably the VBI data extractor is configured to extract the STB data from one or both of lines 19 and 20 of the VBI. The skilled person will appreciate, however, that either or both of the VBIs of the first and second (odd and even) fields of an interlaced video signal may be employed. Thus, in general, in the first field lines 6 to 22 inclusive are available, and in the second field lines 319 to 335 are available. It will be appreciated that, if desired, a relatively large amount of data may be output from an STB in this way.

To explain further some examples of the data which may be output from an STB in the VBI, it is helpful to briefly review some aspects of typical digital television broadcast streams.

An example DVB (Digital Video Broadcast) multiplex comprises 12 video/audio channels within an 8MHz QAM (Quadrature Amplitude Modulation) band. As well as video and audio information, a channel may also include other information, for example EPG information. An example cable transmission comprises 27 multiplexes over the in-band (IB) range 200 MHz to 800 MHz. Embodiments of the test systems we describe may be used with a variety of DVB system including, but not limited to, DVB-C (cable), DVB-S and DVB-S2 (satellite) and DVB-T (terrestrial).

In general the video and audio information is transmitted in MPEG-2 transport streams, and there is a hierarchical structure of additional information including a network information table (NIT) for the multiplex, a program association table (PAT) which contains a list of programme map tables (PMTs), a service description table (SDT) and a video and audio PD (Program or Package Identifier) for each video and audio stream.

Subtitle text is also present. For a detailed description of the DV]3 transport streams, reference may be made to one or more of the DYB standards.

Thus, information which may be included in the STB data includes one or more of video PD data, audio PD data, and subtitle text data. Other information which it is useful to include comprises STB time, that is a clock time of the STB which can be used, for example, for British Summertime/Greenwich meantime testing; STB up-time data, for example, time since a last reboot, which can be used for example for stability testing; and STB heartbeat data, either an actual heartbeat or a signal indicating that the heartbeat is present, again useful for stability testing to determine, for example, whether the STB is still responding (the "heartbeat" typically comprising a regular, hardware interrupt). In some preferred embodiments the output interface system also includes a SCART interface to interface with a SCART connector of the STB and provide SCART-derived data to the test system. This SCART-derived data may comprise, for example, data indicating whether the feature is in a 4:3 or widescreen format.

Information on the video signal (composite, RGB, S-video or other) and left and right audio-derived information such as audio level, (total harmonic) distortion or other audio quality measures and the like.

The inventors have further recognised that there are certain situations in which one or more pins of the SCART connector may serve a dual purpose. In particular, pin 16 of the SCART connector, which usually provides RGB composite video, may be employed to indicate when the STB reboots, without interfering with a user's perception of the operation of the STB since, when the SIB reboots, the TV screen is generally black in any case.

More generally, STB data, which may include fault-related information, may be provided via the SCART connector by using a variable length pulse (pulse length encoding the data) on a pin of the SCART connector, preferably pin 16. Thus, for example, a pulse of a different length to that indicating a reboot may be employed to indicate a fault, although preferably in a debug mode, since pulsing pin 16 is generally not desired during normal use of the STB. It will be appreciated that quaIif'ing data could be sent using this variable pulse length technique, for example a plurality of bits, such as an ASCII string or Unicode code. Such an arrangement can be used to facilitate self-diagnosis of faults by an STB. For example, a front-end demultiplexer hardware reset may be signalled on the SCART connector and used, for example, to trigger recording of at least part of the transport stream so that an aspect of the stream which may be causing a problem can be identified.

Preferably the STB test system also includes an RF (Radio Frequency) feed system for the one or more STBs. This may comprise either a live or a recorded feed, optionally with a switch controlled by the test system to select between the two. Preferably the output interface system then also includes an out-of-band interface to receive a signal sent back down the RF feed by a set top box.

In embodiments the OOB data may be used to test the MIB (Manufacturers Information Database) in the set top box, broadly speaking data relating to a configuration of the STB such as one or more AGC (Automatic Gain Control) levels, an SNR (Signal to Noise Ratio) level, software version data and the like. This database may comprise a hierarchical arrangement of objects, and may include information that a network operation may wish to know, and it is therefore useful to be able to test the data in the MIB. There is generally a facility for obtaining M data from an out-of-band signal (which is received and interpreted by an Mifi manager software module). Mffi-derived data may also be displayed on the screea of the television to which the STB is connected, and thus a combination of composite video analysis and optical character recognition, as mentioned above, may be used to compare the MIB data derived from these two sources.

Preferably the RF feed system includes a channel multiplex storage device for providing one or more stored channel multiplexes to the one or more STBs. This facilitates a number of tests including, but not limited to: frequent NIT updates, for example every minutes rather than everyday, which "stresses" the STB processor; provision of audio test data, such as a sine wave test; provision of video test data such as a video pattern; and download testing for testing downloads of software to the STB. The timing and other parameters of these tests may be controlled by controlling the storage device.

As referred to above, in embodiments the control system may also include a trigger output for signalling an STB fault. The test system may then include a system for storing and/or identifying a portion of a channel multiplex provided on the RF feed associated with the STB fault. Thus a portion of a live feed may be recorded in response to the trigger (either after or before using a buffer) and/or pre-recorded test data may be indexed by the trigger output.

The RF feed may comprise one or more items of infonnation controllable by the test control system in addition to the STB video and audio data. Such information may include the following: NIT updates; AFDs (Aspect Format Descriptors); TDT (Time Definition Table) information defining DST/GMT; subtitle information; audio information; conditional access information (for example relating to subscription channels); and out-of-band (OOB) information. The OOB information may comprise SNMP (Simple Network Management Protocol) requests -which can be sent to an STB requesting, for example, M]B -derived information. IP requests, for example for downloading from walled garden sites; data relating to interactive services; data relating to billing; and the like.

In some preferred implementations of the test control system, a first computer system is employed for controlling the STB command system and a second computer system coupled to the STB output interface system is used for identifying an expected result of a command sent to an STB. The second computer system may report this identification to the first computer system. This facilitates implementation of the software. Thus, in particular, the second computer system may comprise a logging server agent (LSA) which logs all the commands issued by the first computer system, preferably with a time stamp, as well as outputs from the STBs. The responses are preferably passed back to the first computer system, described below as an ATE (Automated Test Equipment) host, which may then perform data analysis on the results. For example, a pass/fail matrix of the one or more tests conducted may be constructed, Preferably the test system is configured for testing a plurality of STBs simultaneously. In this case the second computer system may wait until the expected result has been received from a selected number of the STBs (for example selected according to a mask) before reporting the result.

Such an arrangement also facilitates the testing of different hardware and software versions of an STB side by side, The same set of test instructions or test script is used for testing two or more software or hardware versions side-by-side and the pass/fail matrices can then be compared. This is useful in identifying where a fault may be occurring, and can also be used when, say, testing a customer's faulty STB, for example to see whether a pass/fail matrix corresponds with, say, a known issue with a particular version of software.

In some preferred embodiments a script language is used to define one or more tests and the test control system includes an interface to receive and/or store such a script.

Optionally multi test scripts may be run, one after another. In preferred embodiments the instructions of a test script are of two types. Broadly speaking a first type (referred to later as a non-blocking instruction) for execution by the first (ATE) host computer system and a second type (referred to later as a "blocking" instruction) for at least partial execution by the second (LSA) computer system. The "non-blocking" instructions are, broadly speaking, executed sequentially one after another without waiting for a response from an STB (although one of the non-blocking instructions may be a delay instruction), whereas a "blocking" instruction waits for a particular response, generally defined by the instruction, from an STB, or for a time out condition. With multiple STBs the defined response may be required from some or all of the STBs for the expected response to be identified as having been successfully returned. In the script language preferably each command has a flag to indicate whether an instruction is of the first or second (blocking or non-blocking) type.

The invention also provides a method of testing one or more set top boxes (STBs), the method comprising issuing commands to said STBs, receiving responses from said STBs and comparing said responses with expected responses in a closed-loop automatic test system.

Preferably the responses are received from the STBs at least in part within a VBI interval.

The invention also provides means to implement the above-described methods.

In a further aspect the invention provides a set top box including the above-described capability for outputting STB data in a VBI interval; and an STB including the capability for signalling to provide test data using one of the pins of a SCART connector on the STB.

Thus in a further aspect the invention provides a set top box (STB) including one or more automatic test facilities selected from the group consisting of a system to include STB test data in a vertical blanking interval (VB1) of a video output from the STB and a system to control one or more pins of a SCART connector of the STB to output STB test data from said SCART connector.

The invention further provides computer program code, in particular on a carrier, for configuring a processor to implement the above-described test control system, and to implement a set-top box as described above. The software may be written in any conventional programming language, for example Visual Basic TM, C, or a lower level language, and may be provided on a carrier such as a magnetic and/or optical disk, on programmed memory or on a data carrier such as an optical or electrical signal carrier.

As the skilled person will appreciate the software may be distributed between a plurality of coupled components in communication with one another.

These and other aspects of the invention will now be further described, by way of example only, with reference to the accompanying figures in which: Figure 1 shows an outline block diagram of an embodiment of a system according to the invention; Figure 2 shows a detailed block diagram of an embodiment of a system according to the invention; Figure 3 shows a flow diagram of computer software for the system o Figure 2; and Figure 4 shows a block diagram of a set top box (STB) configured to implement an embodiment of an aspect of the present invention.

Referring to Figure 1, this shows an outline block diagram of an embodiment of an NTB test system 100. The system 100 comprises a test control ("STATE") system 102, which has an input comprising one or more text scripts and expected responses 104, and which provides a test results output 106. Racks are provided for mounting a plurality of STBs-undertest 108 in one embodiment 6 sets of 16 STBs. These receive input data including commands from an input multiplexer 110 and provide output data to an output multiplex (selector) 112. The test control system controls input multiplexer 110 and output multiplexer 112 and receives a plurality of output signals from the STBs under test. Broadly speaking the test control system issues commands and receives responses in accordance with a test script to perform STB testing.

The input multiplexer 110 receives a matrix of input data including IR blasting data according to one or more protocols (for example TWT V-two way TV, and RC5), RF feed data (as described above) shell commands, typically provided into a serial, for example RS232 port of an STB, and software downloads (for example an NIT update indicating to an STB that a new software version is available). The output data from the STBs preferably includes video data, for composite video analysis (CVA), audio data (from the STB SCART connector), VBI data extracted from data inserted into the VBI by an STB, serial response data from an STB serial port, Mffi data (received by an SNMP manager operating in the test control system 102), and SCART control signals, preferably received by an AC/DC coupled trigger for inputting both audio signals and DC voltages from an STB SCART connector.

Examples of STB functionality which the test control system is able to test include: correct channel tuning (for example that both video and audio of the correct channel, say BBCI, are present when the channel is selected); correct provisioning (that is a correct subscription level, for example basic or family pack, including functionality for adding/removing channels); stereo audio quality, audio levels and volume control range; electronic program guide population and functionality; interactive activity. The test strategies may include both unit (STB) tests and end-to-end tests typically including the transport stream and back end servers, tests of functions not explicitly provided to users such as software downloads, integration tests, performance tests, stability tests, bug fix tests and the like. As described further below the system preferably employs distributed processing and preferably has the capability for testing "black", "white" and "grey" STB builds. In this context "black" refers to black box testing of a normal build, i.e. the same build which a customer uses. White box testing refers to test of a software build which includes instrumentation; grey box testing refers to testing of a shell build, that is a build essentially purely for debug purposes. Typically scripts are written from functional requirement specifications and may include "look-for conditions (blocking commands") -that is commands looking for an expected result from an STB.

Conditions are detected by a logging server agent (as described further below). As mentioned. composite video analysis may be used preferably in conjunction with optical character recognition, in particular for analysing set-up and/or diagnostics screens which may display messages such as "service not running", and "credit left..".

The system of Figure 1 is a closed loop automatic test system and, in preferred embodiments provides automatic pass (or fail) evaluation of a plurality of STBs.

Referring now to Figure 2, this shows a detailed block diagram of the system 100 of Figure 1.

The test control system comprises ATE (Automatic Test Equipment) hosts controller and logging server 202 (preferably implemented using linux) which communicates with a logging server agent (LSA) database 204. The LSA 204 logs and time stamps script commands and STB outputs for use by controller 200. The controller 200 and server 202 are connected by computer network 206, which also receives inputs from a number of different STB output interface systems, as described further below.

JR blasting is performed by an JR distributor 208, coupled to controller 200 and, via JR distribution cables 210, to each STB 108. Each cable preferably terminates adjacent an JR remote control sensor for an STB. The hR distributor receives commands 208a from controller 200, performs a protocol look-up 208b to determine the corresponding hR signal, and sends 208c the JR command to the relevant set-top box 108. A table or map is stored indicating for each STB which protocol to employ, and the IR distributor 208 performs a look-up in this table at step 208b, to determine which protocol to employ. A serial distribution unit 212 is also coupled to controller 200 (although this is optional depending upon the implementation). Serial distributor unit (SDU) 212 receives (shell) commands from controller 200 and sends serial data to the STBs 108 via serial control links 214. Serial data responses from the STBs are received by one or more terminal servers 216, which are also coupled to network 206.

A power distribution unit 218 provides power supplies to the STBs 108, again under control of controller 200. This allows the controller to momentarily interrupt power to selected boxes to cause a reboot, for example to test reboot timing.

An RF feed 220 to the STBs is provided by an (optional) RF selection unit 222 to select between a recorded feed 224 and a live feed 226, again controlled by controller 200.

The recorded feed is provided by a recorded stream jukebox 228 which, in one embodiment provides up to 128 recorded streams, again under control of controller 200.

The out-of-band RF from the STBs is processed by an out-of-band interface 230, also coupled to RF feed 220, which outputs out-of-band data 232. The OOB data in particular includes M data which is provided to controller 200, which runs an Mffi manager and which provides Mffi data to LSA 204.

A composite video analyser/VBI analyser 234 receives a plurality of composite video outputs from the STBs and processes these to extract VBI data output by the STBs and, optionally but preferably to perform optical character recognition on the composite video to read at least parts of STB set-up, diagnostic and optionally other screens displayed by the STB for example EPG screens. The analyser 234 provides output data to network 206 for capture by LSA 204.

A SCART interface unit 236 preferably provides a number of ft1nctions relating to analysis data available on a SCART connector of an STB. For example left and right audio outputs are available and analyser 236 may therefore be configured to implement audio level andlor quality (distortion) measurements. These can be used, for example, to determine whether a volume setting on an STB produces approximately the correct RMS (Route Mean Square) audio output level. Preferably analyser 236 also includes a DC coupled input responding to DC voltages on the SCART connectors -for example 6 volts on pin 8 indicates 4:3 format whereas 12 volts indicates widescreen format.

The data analysis units 234, 236 are highly advantageous in implementing a closed loop automatic test system for the STB. Preferably one analysis unit, 234, 236 is provided for each STB (recognising that multiple units may be provided within a single physical housing) although in other arrangements units may be shared between STBs, for example one per rack.

In one embodiment six racks each capable of holding 16 STBs are provided. Each STB preferably has an 11 address which allows it to be addressed, for example by controller 200.

Other optional units which may be included include a "wandering remote" unit 238 which provides signals to controller 200 for JR remote control of a selected STB via JR distributor 208. An optional voltage generator 240 may be employed to provide a controllable voltage output, for example for comparison with other voltages such as a SCART voltage. Preferably controller 200 provides a trigger output 242 which may be used for a number of purposes including, for example, to trigger recording of a transport stream or video on, say, a hard drive, DVD recorder or the like. Preferably the device is coupled to controller 200 to include "built-in test" software so that controller 200 can self-test correct operation of the test system 100.

Other facilities which the test system may incorporate include: audio presence/absence detection; a "black screen" detector/timer; and a hardware stopwatch which can be started and stopped depending upon defined conditions preferably without using the server-based closed ioop.

Although in preferred embodiments the STBs test system operates as a closed loop system, in principle a separate system may be employed to provide IR remote controls to the STBs, in which case controller 200 may be provided with a facility to detect the IR commands issued by the separate system, in other respects operating generally as described above.

The controller 200 runs software to implement the test system, in one embodiment written in Visual Basic. Controller 200 may comprise aconventional personal computer linked to the various interface units by USB (Universal Serial Bus) connections. Preferably software is included to implement SNIv1P manager classes for running SNMP commands. Test scripts may be written using any convenient script writing software and stored locally for implementation. As previously mentioned, commands and responses are received over network 206 by LSA 204 and are logged and time-stamped.

Figure 3 shows a flow diagram of software on controller 200 for running a set of test scripts.

At step S302 a script count is initialised and then, at step S304, the next test script is loaded. Each command is read in turn; if the command is a non-blocking command (S306a) the controller 200 controls the system to implement the command (and the command is logged by LSA 204). If the command is a blocking command (S306b) then the command is sent to the LSA 204 to look for the defined condition from the set of STBs, with a mask applied to determine which STBs are monitored. At S308 there is an optional intercommand delay, which may be scripted (non-blocking).

At step S3 10 if the previous command was an unblocking command the procedure loops to step S304 to read the next command from the test script. If the command was a blocking command then, at step S3 12, the procedure waits for a response from the STBs to be signalled by LSA 204 or alternatively, for a Timeout condition.

Afterwards, at step S314 (for a script containing only non-blocking commands, at the end of the script) the procedure reads the output matrix data, i.e. the data collected by LSA 204 and translates this to response data as necessary. The procedure then (S3 16) determines whether the test has been passed or failed and, preferably, enters this data into a multibox pass/fail matrix. The script count is then incremented (S318) and the procedure loops back to step S304 to load and read the next test script.

Preferably (but optionally) at the end of the procedure an automatic pass/fail matrix analysis (S320) is performed to provide test results 106 (of Figure 1). This analysis may, for example, look for a pattern of faults which may indicate a particular problem and/or may compare different software versions, for example to indicate whether a problem in an earlier version has been resolved in a later version, and/or prove a bug fix. The analysis may implement more than one of these analyses if desired.

In embodiments the software of Figure 3 may be employed to drive multiple boxes in parallel with the same test script. Additionally or alternatively, multiple instances of the software may be run concurrently for testing a plurality of different hardware and/or software versions simultaneously, optionally to perform comparative testing.

An extract from a first, non-blocking example script is shown below: lB. "10l"<Mask> II Select BBCI IR "U"<Mask> Ii Up to BBC2 *N Delay n II Wait a bit N ADT "VR"<3> /1 Scart Pin 8 voltage The "Mask" term defines a Mask indicating to which STBs to apply the command; ADT refers to SCART data analysis unit 236 and "VR" refers to voltage read, in this case from input 3 of the unit.

A second example script including a blocking command is shown below: N lit "G"<Mask> *B LSA Lookfor "01 "<Mask>Timeout The first command sends the JR blasting command "G" to select the electronic program guide and this is then followed by a blocking command to LSA 204 to look for data "Gl" (again with a Mask applied) which may be received from terminal service 216 or, for example from VBI analyser 234. The Timeout value specifies a Timeout period for the command, as mentioned above.

Referring next to Figure 4, this shows a block diagram of a set-top box 400 configured to output test data in the VBI interval of a composite video output of the box, and also on the S CART connector (using pin 16).

In a conventional manner the STB 400 comprises an input 402 from an RF cable (although in alternative embodiments this could be from a satellite antenna block down converter, from a terrestrial antenna or from some other source) which is provided to an input tuner 404. The tuner selects a mulitiplex which is provided to demux (demultiplexer) 406 which, under the control of processor 408, provides video and audio PIDs and MPEG encoded data to a video/audio decoder 410 which may also be in communication with processor 408. Demux 406 receives control information from processor 408 and provides data to processor 408 such as EPG data, system information data (including MPEG transport stream data) and the like.

The decoded video and audio for the selected channel is provided to modulator 412 which provides a video output signal for the television to SCART socket 414. An infrared remote control sensor 416 is also coupled to processor 408 to provide an interface to JR remote control device (not shown in Figure 4).

The processor 408 provides graphics overlay information to modulator 412 and, in preferred embodiments, VBI data incorporating STB output data such as that described above, for example video/audio PID data, and other data relating to the operation of the STB. The processor also provides STB output data to SCART socket 414, for example to pulse pin 16 of the SCART socket to indicate a reboot, again previously described in detail above.

The processor 408 is coupled to non-volatile memory 418, for example Flash memory.

This stores a software stack typically comprising an operating system, platform and drivers (normally provided by a vendor for the STB), middleware (which typically implements a presentation layer, user interface, internet access, graphics/HTML page display and the like) and other application software such as an electronic program guide.

Other software enables the STB to operate in "shell" mode so that the serial interface (not shown in Figure 4) can be used to control the STB and to receive data back from the STB.

In preferred embodiments of the STB the software stack shown in Figure 4 is extended to include a code for providing STB output data in the VBI and, preferably, code to provide SCA.RT data comprising STB output (test) data to one or more pins of the S CART connector. In order to be able to implement black box testing of the STB, these features are preferably implemented by extending the normal build of the software so that these features are always present as standard in customer/user devices. In this way black box testing of an STB can be implemented without the need to control the STB or receive back responses from the STB using the serial port.

No doubt many other effective alternatives will occur to the skilled person. It will be understood that the invention is not limited to the described embodiments and encompasses modifications apparent to those skilled in the art lying within the spirit and scope of the claims appended hereto.

Claims

CLAIMS: I A set top box (STB) test system, the system comprising: an

STB command system for providing commands to at least one said STh; an STB output interface system for receiving an output from said at least one STB; and a test control system coupled to said STB command system and to said SIB output interface system and configured to control said command system to issue commands to said at least one said STB and to read results of said commands from said STE output interface system, to test said SIB.

2. An STE test system as claimed in claim 1 wherein said STB command system includes an infrared (IR) command system for commanding a said STB using an infrared remote control sensor of the STB.

3. An SIB test system as claimed in claim I or 2 wherein said output interface system includes a composite video analysis system to receive a composite video output from said STB and to analyse said composite video output from said STB to provide video output analysis data for reading by said test control system.

4. An STB test system as claimed in claim 3 wherein said video output analysis data comprises one or more of text data, graphical data, and screen pattern identification data.

5. An STE test system as claimed in any preceding claim wherein said output interface system includes a vertical blanking interval (VBI) data extractor to receive a video output from said SIB and to extract STB data from a vertical blanking interval of said STE video output for reading by said test system.

6. An STB test system as claimed in claim 5 wherein said VBI data extractor is configured to extract said SIB data from one or both of lines 19 and 20 of the VBI.

7. An STB test system as claimed in claim 5 or 6 wherein said SIB data includes one or more of audio programme information data (PD), video PD, subtitle text, STB time, STB up-time, and SIB heartbeat data.

8. An SIB test system as claimed in any preceding claim wherein said output interface system includes a SCART interface to interface with a SCART connector of said STB and provide SCART-derived data to said test system.

9. An SIB test system as claimed in claim 8 wherein said SCART-derived data includes data indicating a re-boot of said STB.

10. An SIB test system as claimed in claim 9 wherein said re-boot data comprises a pulse on SCART pin 16.

11. An SIB test system in claim 8, 9, or 10 wherein said SCARI-derived data comprises data derived from a variable length pulse on a pin of said SCART connector.

12. An SIB test system as claimed in any preceding claim further comprising an rf feed system for said at least one SIB.

13. An STB test system as claimed in claim 12 wherein said output interface system includes an out-of-band interface to receive a signal sent back down said rf feed by said SIB.

14. An STB test system in claim 12 or 13 wherein said rf feed system includes a channel multiplex storage device for providing one or more stored channel multiplexes to said at least one STB.

15. An STB test system in claim 12, 13, or 14 wherein said test control system includes a trigger output for signalling a SIB fault, and further comprising a system for storing and/or identifying a portion of a channel multiplex provided on said rf feed associated with said STB fault.

16. An STB test system as claimed in any one of claims 12 to 15 wherein said rf feed comprises one or more items of information controllable by said test control system in addition to STB video and audio data.

17. An STB test system in any preceding claim wherein said test control system comprises a first computer system for controlling said STB command system and a second computer system coupled to said STB output interface system for identifying an expected result of a command sent to a said SIB and reporting said identification to said first computer system.

18. An STB test system as claimed in claim 17 for testing a plurality of said STBs simultaneously, and wherein said second computer system is configured to report said identification responsive to identification of said expected result from a defined number of said STBs.

19. An STB test system as claimed in claim 17 or 18 wherein said test control system further comprises a script interface to receive a test script comprising a list of instructions for testing said at least one STB, and wherein said instructions comprise two of types of instruction, a first type for execution by said first computer system and a second type for at least partial execution by said second computer system.

20. A method of testing one or more set top boxes (STBs), the method comprising issuing commands to said STBs, receiving responses from said STBs, and comparing said responses with expected responses in a closed-loop automatic test system.

21. A method as claimed in claim 20 wherein said responses are received from said STBs at least in part in a vertical blanking interval (VBI) of a video output of a said STB.

22. A set top box (STB) including one or more automatic test facilities selected from the group consisting of: a system to include STB test data in a vertical blanking interval (VBI) of a video output from the STB; and a system to control one or more pins of a SCART connector of the STB to output STB test data from said SCART connector.