SYSTEM FOR AND METHOD OF DISPLAYING AN OUTPUT IMAGE ON A DISPLAY DEVICE
The invention relates to a system for displaying an output image on a display device.
The invention further relates to a method of displaying an output image on a display device.
Some televisions provide a feature which is known as "picture-in-picture". E.g. in patent specification US 5,130,800 is described that multiple pictures from distinct video signals are simultaneously displayed on a display device of one image display apparatus. In that case the dimensions of the pictures from at least one of the video signals are strongly reduced by means of sub-sampling. This sub-sampling has a negative influence on the readability of textual information.
It is an object of the invention to provide a system of the kind described in the opening paragraph with which reduced images originating from a first series of input images are simultaneously displayed with images from a second series of input images with improved readability of information related to the first series.
The object of the invention is achieved in that the system for displaying an output image on a display device of, comprises:
- receiving means for receiving a stream comprising a first series of input images and a second series of input images;
- scaling means for calculating a scaled down image by scaling a first image of the first series of input images; - extraction means for extracting data from the stream, with the data being related to the first image; and
- merging means for creating the output image by merging a first image of the second series of input images, the scaled down image and a readable representation of the extracted data.
Displaying a reduced, i.e. scaled down image, usually results in unreadable text. The invention is based on the insight that some data related to the scaled down image is of particular importance and hence should be displayed in a format which enables reading by the user who is primarily focussing on the second series of images which is displayed on the same display device. As a consequence the user, looking at the display device, can observe the second series of images, the scaled down images of the first series and also relevant data related to the images of the first series which are displayed in a reduced format.
An embodiment of the system according to the invention is designed to display the readable representation of the data with a size which is larger than 2 percent of the width of the display device. In order to be readable under normal viewing conditions, i.e. a distance of some meters between the user and the display device, the size of the readable representation of the data should be large enough. Each character of the readable representation of the data should be at least 0.5 cm high and width. Typically a font size of 20 pixels is to be used in the case of a display resolution of 720x625 pixels. An embodiment of the system according to the invention is designed to display the readable representation of the data adjacent to the scaled down image. Preferably the readable representation of the data is displayed in the neighborhood of the scaled down image, e.g. at the border. In this way it is clear that the readable representation of the data is related to the scaled down image. The readable representation of the data might be displayed above, below, left or right relative to the scaled down image. Optionally the readable representation of the data is wrapped around the scaled down image.
In an embodiment of the system according to the invention the extraction means are arranged to extract values of a group of pixels from the first image and is the system arranged to generate the readable representation of the data on basis of the values of the group of pixels. Preferably the group of pixels correspond to a block of pixels. These pixels are selected from the first image and used to generate the readable representation of the data. This generation might be a straight copy of pixel values. Optionally image processing is performed to enhance readability, e.g. filtering for edge-enhancement or contrast enhancement, interpolation for enlargement, or color adaptation by means of a Look- Up-Table.
In an embodiment of the system according to the invention the extraction means are arranged to extract the values of the group of pixels from the first image on basis of character recognition. The selection of the group of pixels might be based on known pixel coordinates, e.g. user defined. But preferably these pixels are selected by means of an
automatic selection algorithm. Such an algorithm might be based on character recognition: which pixels correspond to characters, e.g. as specified in patent specification US 6,195,459. Optionally the character recognition is combined with text or word recognition. Recognized characters can be input for a graphics generator which is designed to render characters of various fonts and size.
Another embodiment of the system according to the invention is characterized in that the extraction means are arranged to extract a data-element from a digital video broadcast transport stream and that the system is arranged to generate the readable representation of the data on basis of the data-element. In this embodiment the readable representation of the data is not based on values of pixels of the first image of the first series but on data-elements which are exchanged from a broadcast to the system according to the invention by means of a separate stream of data. In this case, the received information stream is a transport stream with services, each comprising respective elementary video streams and elementary data-streams (meta-data included), which are mutually related. For example a video stream represents the images being captured of a football match and a data-element of the related elementary data-stream represents the current score of the football match.
An embodiment of the system according to the invention comprises the display device. The system might comprise a separate set top box and television. But preferably these two are integrated into one unit. It is a further object of the invention to provide a method of the kind described in the opening paragraph with which reduced images originating from a first series of input images are simultaneously displayed with images from a second series of input images with improved readability of information related to the first series.
This object of the invention is achieved in that the method of displaying an output image on a display device, comprises the steps of:
- receiving a stream comprising a first series of input images and a second series of input images;
- calculating a scaled down image by scaling a first image of the first series of input images; - extracting data from the stream, with the data being related to the first image; and
- creating the output image by merging a first image of the second series of input images, the scaled down image and a readable representation of the extracted data.
Modifications of the system and variations thereof may correspond to modifications and variations thereof of the method described.
These and other aspects of the system and of the method according to the invention will become apparent from and will be elucidated with respect to the implementations and embodiments described hereinafter and with reference to the accompanying drawings, wherein:
Fig. 1 schematically shows an embodiment of the method according to the invention;
Fig. 2 schematically shows an embodiment of the system according to the invention;
Fig. 3 schematically shows an alternative embodiment of the system according to the invention in its context; and Fig. 4 schematically shows the components of the alternative embodiment of the system according to the invention. Corresponding reference numerals have same or like meaning in all of the Figures.
Fig. 1 schematically shows an embodiment of the method according to the invention. Fig. 1 shows a first image 100 of a first series of consecutive input images representing a football match and a second image 102 of a second series of consecutive input images representing a car program. The group of pixels 106 in the upper right corner of the first image represent the current score. Based on a scaled down copy 108 of the first image 100 and on the second image 102, an output image 104 is created. The output image 104 is conform the so-called "Picture-In-Picture" concept. The output image 104 also comprises a readable representation 110 of data which is extracted from the first image. In this case a copy is made from values of the group of pixels 106 of the first image which represent the current score. The result of the extraction and pasting of the group of pixels 106 is that the current score is still readable. Note that the scaled down copy 108 also comprises a representation 112 of the current score. But because of the scaling this representation 112 has a limited size and hence it is hardly readable.
Fig. 2 schematically shows an embodiment of the system 200 according to the invention being arranged to perform the method as described in connection with Fig. 1. The system 200 comprises:
- a first receiving means 202 for receiving a signal representing a first input image 100 of the first series of consecutive input images. The signal may be a broadcast signal received via an antenna or cable but may also be a signal from a storage device like a NCR (Video Cassette Recorder) or Digital Versatile Disk (DND). The signal is provided at the input connector 214. The first receiving means 202 is arranged to be tuned to a first predetermined frequency band to extract the first input image 100 from the received stream of information.
- a second receiving means 204 for receiving a signal representing a second input image 102 of the second series of consecutive input images. The second receiving means 204 is arranged to be tuned to a second predetermined frequency band to extract the second input image 102 from the received stream of information. - scaling means 206 for calculating a scaled down image 108 by scaling the first image 100 of the first series of consecutive input images;
- extraction means 208 for extracting data from the stream of information, with the data being related to the first image 100;
- merging means 210 for creating an output image 104 by merging the second input image 102 of the second series of consecutive input images with the scaled down image
108 and with a readable representation of the data 110 being extracted from the first input image 100.
- display means 212 for displaying the output image 104.
A television is an example of an embodiment of a system 200 as described in connection with Fig. 2.
Fig. 3 schematically shows an alternative embodiment of the system 300 according to the invention in its context. A set top box is an example of an embodiment of a system 300 as described in connection with Fig. 3. The system 300 has a number of connections with other systems 302 and 310: - The system 300 is connected to a broadcast server 310 by means of a broadband network 306. The broadcast server 310 is responsible for providing a transport stream to the broadband network 306.
- The system 300 is connected to a television 302 which comprises a display device 212. Video and audio content and data being exchanged via the system 300 are
displayed on the television 302. That means that the output images which are generated by the system 300 are provided to the television 302.
- The system 300 is optionally connected to a broadcast server 310 by means of a second network 308, e.g. Ethernet. This second network is primarily used for data exchange from the system 300 to e.g. the broadcast server 310.
The broadcast server 310 comprises a first storage means 312 for storage of video and audio content being provided by means of input connector 320. The broadcast server 310 also comprises a second storage means 314 for storage of data. This data comprises application data, i.e. software code which is the base of an application to be run on the system 300, and of data to be processed by the application. Data of this type will be extracted by the system 300 according to the invention, converted to a readable format and merged with images of streams of video. The video and audio content are multiplexed with the data by means of multiplexer 316. The resulting transport stream is modulated by means of the modulator 318 which is designed to convert the transport stream to a higher frequency such that it can be transmitted on the broadband network 306 (e.g. terrestrial or satellite).
Fig. 4 schematically shows the components of the alternative embodiment of the system 300 according to the invention. The system 300, e.g. a set top box, is connected to a television 302. The system 300 is provided with a transport stream at the input connector 214 and provides the television 302 with a video signal representing the series of output images at the input connector 303 of the television 302. The output images are displayed on the display device 212 of the television 302. The system 300 comprises:
- a receiving means 203 for receiving a transport stream, e.g. a MPEG-2 stream, comprising a number of services. Each service comprises at least one video stream, at least one audio stream and at least one stream of data-elements. The receiving means 203 is arranged to split the transport stream into separate services and to split the services into separate elementary streams. One of the video streams of a first service corresponds to a first series of consecutive input images. One of the video streams of a second service corresponds to a second series of consecutive input images.
- scaling means 206 for calculating a scaled down image 108 by scaling the first image 100 of the first series of consecutive input images;
- extraction means 208 for extracting data-elements from an elementary stream of data corresponding to the first service. The extraction is based on a selection made by the user. The user has indicated which type of information he is interested in. This is done by selecting from provided menu options displayed on the television 302. The menus are
generated by the system 300 on basis of data-elements received. Typically, use is made of the so-called On Screen Display feature of the television 302.
- merging means 210 for creating an output image 104 by merging the second input image 102 of the second series of consecutive input images with the scaled down image 108 and with a readable representation of the data-elements being extracted from elementary stream of data corresponding to the first service. The merging means 210 comprises rendering means to render the readable representation on basis of the extracted data-elements. Extracted data-elements might correspond to the score of a football match.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be constructed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements and by means of a suitable programmed computer. In the unit claims enumerating several means, several of these means can be embodied by one and the same item of hardware. There are several ways for deployment of the processing means over e.g. a television and a set top box.