KR20050016696A - System for processing video signals - Google Patents

Abstract

The present invention relates to a video signal processing system, the system comprising a receiver (100) arranged to receive a video signal, wherein at least one video frame of the signal corresponds to at least one of each of a plurality of broadcast data sources. It contains one area. The system includes a processor 150 arranged to process the video frame to extract at least a portion of the region from at least one of the video frames. Frame 200 of the mosaic video signal may include one or more small pictures 210, each of which may render a reduced version of a television program. Edge detection and line detection may be performed to extract the region 210 or sub-region 240 from the frame. The television channel from which the extracted region or sub-region is received or associated can be identified. The user can define the display of the extracted area or sub area on the display screen.

Description

Video signal processing system {SYSTEM FOR PROCESSING VIDEO SIGNALS}

The present invention relates to a video signal processing system, the system comprising a receiver arranged to receive a video signal, wherein at least one video frame of the video signal corresponds to at least one corresponding to each one of a plurality of broadcast data sources. It includes an area.

US 5,633,683 describes a television receiver for receiving and displaying a mosaic video signal comprising a plurality of sub-pictures, each sub picture representing one of a plurality of video signals, and position data representing each sub on a display screen. The position of the picture is linked to the program number of the associated video signal. The receiver comprises a decoder for decoding the program number of the video signal indicated by the position of the sub picture displayed and indicated by the user. Thus, the user can select the desired program for viewing, recording, or the like by simply " indicating and clicking " on the corresponding sub-image on the mosaic screen. The receiver further comprises means for deriving a selected sub picture from a mosaic signal and means for simultaneously displaying the selected sub picture and associated video signal for a predetermined time period after selecting the video signal. In this way, the receiver comprises a first decoder for decoding and displaying the selected program and an additional decoder for processing the mosaic video signal. The sub picture peculiar to the selected program is cut out and displayed as Picture-In-Picture (PIP).

The sub picture can be derived only when the position data is obtained by the television receiver. Such devices require special transmitters adapted to generate position data, which is a limitation of using such techniques. For example, if the device of the television service provider cannot be arranged to generate position data, the sub picture cannot be derived by the receiver.

1 is a functional block diagram illustrating a receiver of a system suitable for implementing the present invention.

2 is a view showing a visual representation according to an embodiment of the present invention, a mosaic screen.

3 is a view showing an example of a visual representation according to an embodiment of the present invention, in which a sub-region extracted from a mosaic screen is shown in an auxiliary screen area, while an area extracted from a mosaic screen is shown in another auxiliary screen area. Figure showing the loss.

4 illustrates an example of extracting a sub region corresponding to one broadcast data source to be viewed simultaneously with a frame corresponding to another broadcast data source.

It is an object of the present invention to provide a system of the type defined in the beginning which can operate suitably with a video signal irrespective of obtaining angular position data linking the position of each sub-picture on the display screen with the program representation of the associated video signal. To provide.

The object of the invention is realized in that the system comprises a processor arranged to process the video frame to extract at least a portion of the area from at least one of the video frame.

The frame received by the receiver includes one or more regions corresponding to each broadcast source. For example, a frame may contain small pictures, each of which may cause a reduced version of a television program to be broadcast on each television channel. Typically, the image has a rectangular shape. The receiver includes a processor arranged to graphically process video data of each frame and extract a complete picture from the frame. In another example, video information broadcast on one television channel occupies all or part of an area of a frame, and only the video information, sub area or part of a sub picture is extracted by a processor. The sub area (eg, weather map or graph shape) may have a rectangular, circular, or any other shape that forms a closed figure.

First, edge detection may be performed by the processor to retrieve the region or sub region from a frame. Thus, an edge between regions, or an edge between a subregion and other video data is determined. In one of the embodiments, edge detection is performed on a plurality of sequential frames where the location of the region or subregion is the same on these frames, for example by using a time filter. Edge detection is more reliable when the processor analyzes more than one frame to determine region edges on a frame. Thus, the edge between regions within a frame, or between sub regions within an area, is made stronger than the other edges, which are present in the frame and change over time due to, for example, movement of objects, characters therein. . The detection can be improved by further performing detection of lines defining regions or sub-regions from other video data, for example a Hough transform can be used for such detection. In the case of a mosaic frame, the characteristic that the lines between the images are equidistant can be used during the extraction process to reduce or avoid possible false detections. Once the horizontal and vertical lines for the corresponding area or sub area are determined, the video data of the area or sub area can be further processed and / or transferred to presentation means such as a display device for showing the extracted information. An advantage of the system according to the invention is that no data representing the area position in the frame is needed. This offers the possibility of extracting video information regardless of whether auxiliary data for processing the information is available without unnecessary constraints given in the prior art.

Alternatively, the receiver comprises a marker by which the user can indicate an area or sub area of the frame viewed by the presentation means, which area or sub area will be extracted from at least one of the video frames. The processor does not need to determine the area or sub area by using edge and line detection because the area or sub area to be extracted is easily specified by the user. That is, the user will be able to mark any part of the frame (s) shown on the screen, and the area or sub area will be extracted from one or more frames. Another advantage of the present invention is that it provides more ways of processing video information than are known in the art.

In one of the embodiments, the receiver further comprises means for identifying a broadcast data source or television channel, from which the extracted region or sub-region is received or associated with said means. When an area or sub area is extracted, it can be provided as video data without knowing what happens next in this data. If regions or sub-regions are extracted from the mosaic video signal, these regions may not know about the corresponding channels. There are many ways to identify each channel. For example, the logo of the TV channel may exist in the extracted region or the region in which the sub region is extracted. The user can also specify the channel manually.

In another embodiment, the rendering means may allow the user to specify a representation of the extracted area or sub-region on the screen. For example, the location of the extracted information may be customized to the user's preference, and the screen area where the extracted area or sub-area is viewed may be changed.

These and other aspects of the invention will be more apparent and described with reference to the accompanying drawings.

1 is a functional block diagram of a receiver according to the present invention. The system includes a receiver 100 that may be connected to or include a display device (not shown), a video cassette recorder (VCR), and a loudspeaker or other device. The receiver can also be integrated into different devices, such as set-top boxes, or other devices designed to operate with AV (audio-video) signals. Receiver 100 receives a plurality of video signals transmitted via satellite, terrestrial, cable or other link. The command may be input to the receiver by an infrared signal transmitted from a remote control unit (not shown). Thus, the receiver includes a receiver (not shown) that operates with a control signal. The remote control unit may have a special button associated with a possible command to control the receiver as described herein. Currently, MPEG-based systems for transmitting and / or receiving digital video signals are well known. The receiver according to the invention can be arranged to receive digital and / or analog video signals.

The receiver includes at least one tuner 110, demultiplexer 120, optical audio decoder 130, at least one video decoder 140, and a video processor 150. The received video signal is applied to the tuner 110.

The video signal may merge a small size image occupying a relatively small area of each frame with a mosaic video signal comprising the frame. Each picture represents a video signal associated with each TV channel, Internet broadcast center, or other broadcast data source. Alternatively, the video signal may include information received from a single broadcast data source. For example, the tuner can receive a video signal of only one TV channel.

The tuner 110 may include a demodulation circuit for demodulating the received signal and an error correction circuit for detecting and correcting an arbitrarily generated error. The output of the tuner is supplied to a demultiplexer 120 for decrypting the signal. The demultiplexer provides an output audio signal to the audio decoder 130 and an output video signal to the video decoder 140. Decoders 130 and 140 decode audio and video signals, respectively, which may be MPEG-compressed signals. Through future developments in video systems, implementations of the present invention may be modified by those skilled in the art.

The receiver may comprise more than one tuner and more than one decoder, for example two tuners and two video decoders. Each decoder may include a memory (not shown) for storing the video signal. One of the tuners can be used to receive a signal of a user-selected channel, while the other tuner is used to receive a video mosaic signal. In this way, one of the video decoders can be used to decode the video mosaic signal. Signals corresponding to different TV channels are received as mosaic video signals. An advantage of the present invention is that the receiver can alternatively comprise only one tuner and perform the same function as when two tuners are available. For example, the tuner may be arranged to receive a signal of a channel selected by the user and to receive a signal of a mosaic. A single tuner can tune to, for example, a mosaic channel for an x (eg 3) picture, and a main (eg user selected) program for a 50-x picture. The x picture period leaves the tuner sufficient time to extract at least one picture from the mosaic channel. Lost pictures for the primary channel may be generated by picture repetition or in high-end systems using natural motion interpolation. Since only a few images are lost from the main program, this becomes almost invisible to the viewer.

The decoder 140 supplies the decoded video signal to the video processor 150. The processor processes the received signal according to the present invention to extract video information.

If a mosaic video signal is found available, the frame of the signal is a simple picture, or video information, that is analyzed without any additional data. 2 shows an example of a frame 200 of a mosaic video signal. In order to identify the area 210, the coordinates XY of the corresponding area having a generally rectangular or similar shape on the frame must be determined. This can be accomplished by first using edge detection techniques commonly known from a book entitled, "2D Signal and Image Processing" (1990, Jae S. Lim, Prentice-Hall PTR, New Jersey, pp. 476-483). Edges between regions 210 corresponding to different TV channels should be detected. An edge is a boundary or contour in which a change in the physical aspect of an image occurs, for example, a change in pixel gray value, color, and texture. If the physical aspect changes rapidly, the edge line from the strip of candidate edge points is determined according to the algorithm described in the cited document above. For example, the threshold for the physical parameter can be determined by calculating the gradient of the vector in the x and y directions. At this time, the magnitude of the gradient is compared with the threshold value to determine the candidate edge point. After that, the edge will appear as a strip, for example an edge thinning algorithm is applied to determine the edge curve. In one simple edge thinning algorithm, the edge point is selected by checking whether the module of the gradient is at a local maximum in at least one direction. Edge detection can be performed on sequential frames for the same area to ensure accurate detection.

Detection of line 220 is performed by processor 150 after edge detection. Many methods for finding point alignment and feature arrangement in an image are known. For example, a so-called "least feature" method in which the sum of the squares of the vertical deviations of each point from the line is minimized can be used to fit a straight line to the data point. A good thin method can be advantageously used, as known from the "Image Processing Guide" (1995, by John C. Russ, Florida, Boca Raton, CRC Publishing, pp. 495-500). It should be noted that the method can also be used to detect regions bounded by non-straight lines in a frame. Once it is determined to fit the line to the detected edge, the required video area is known to the system. If the region has a rectangular shape, the coordinates XY of the region on the frame are determined, while the video data corresponding to the region can be identified and separated from the video data of the complete frame.

When the region 210 corresponding to each broadcast data source, or TV channel, is extracted, uncertainty may still remain about which TV channel the region corresponds to, or is simply unknown to the system from a received frame or simple image. You may not. To this end, the processor 150 may be further arranged to identify the source of the extracted video region. One possibility is to analyze the entire area 210 to place the logo 230 of the TV channel if a processor is available. At this time, recognition techniques known in the art are applied to recognize codes, logo-images, texts, etc., which are then compared with identification information of channels previously determined by the system or predetermined by the manufacturer and stored in the receiver. do.

Alternatively, in determining the position of the logo, the user may “highlight” the logo video area on the screen. Temporal analysis can be applied to logo video data, for example, by detecting logo video data in sequential frames over multiple frames. Logo data may be identified by using the above-described detection technique, and may be extracted and stored in a memory coupled to the processor. Logo identification data, a logo template, obtained in such a manner can be used to identify TV channels. To identify the channel, the logo in the video signal can be correlated with the logo template stored in memory, for example using well known least squares methods.

It can happen that no logo is present in the frame when the advertisement occurs. However, in general, positioning information broadcast on a specific TV channel in an area does not change every other frame. This problem can be solved by storing a table of TV channel identifiers associated with each region in memory means (not shown) coupled to the processor, as shown for the mosaic frame of FIG. 2 in Table 1. This Table 1 can be derived once and stored by the system itself.

SBS-6 Yorin Ned1 CNN BBC RTL4 V8 RTL5 Ned2

Alternatively, audio information corresponding to the image on which the TV channel is established can be analyzed. Some TV channels periodically repeat their promotional pause and accompany video information with specific music. All of these are also used to identify TV channels and can be implemented by those skilled in the art without difficulty.

In another example, data for identification of a broadcast source may be received from a remote transmitter. For example, a sequence of channel names (IDs) or abbreviations received by receiver 100 is sufficient for this purpose. The data may include one special character (SC) that specifies whether the right or other direction should be used to associate the identifier with the next area. In this way, only little additional information is needed. One example of such identification data for the mosaic frame of FIG. 2 is shown in Table 2 below. Problems with the implementation of such identification methods can be found in US Pat. No. 5,633,683.

According to a further aspect of the invention, the processor 150 is arranged to extract only a portion of the region 240, or sub-region, corresponding to each broadcast data source, or TV channel. Such sub-regions may be extracted and used by a processor when position data for identification of the sub-regions in the region is generated by a broadcast data source to a transmitter or receiver. For example, a TV channel broadcaster may include such data in a digital video signal. The provider of the mosaic video signal, if available, may merge the data into the mosaic signal for the region corresponding to each TV channel. Such a system including a transmitter and a receiver can be realized by appropriate modification of the system known from US Pat. No. 5,633,683.

Alternatively, the user can define the sub area to be extracted. For example, a cursor, marker or other pointer shown on frame 200, shown on the display device, can be used for that purpose. The user can use the direction keys of the remote control unit to displace the markers, cursors and special keys for marking the sub area 240 on the frame. When the sub region is selected to be user-operable, the sub region may be extracted from the next frame of the mosaic video signal.

If the user manually selects the entire rectangular area 210, it should be noted that the selected area corresponds to the rectangular area automatically detected using the described technique, for example, a power transform.

Another way of extracting subregions may be related to the MPEG-4 standard, which offers the possibility of adjusting to video objects. Other ways may be derived within the scope of the present invention. If necessary, identification of the TV channel corresponding to the extracted sub-region may be performed as described above.

4 shows a frame 400 of a video signal of a TV channel. In contrast to the above example, all the information in this frame corresponds to the same channel "A". The sub region 410 may be extracted by one of the methods described with reference to the mosaic frame.

Identification of objects such as people, animals, characters, cars, etc. within the area can be performed by using the techniques described above. The user can select one or more identified objects to be further identified in subsequent frames. Of course, if the object is a three-dimensional graphical shape, the representation of the object may change over time. Therefore, analysis of the presence of the same object in different frames can be implemented. For example, in the MPEG-4 standard, control of video objects is known, which can be matched to the object shown in the next frame. Extraction from Frame The image of the selected object may be further displayed or processed. The set of extracted object images corresponding to the defined objects may also be stored for further display and the like.

In general, two types of extraction according to the present invention can be summarized. In the first case, the region or subregion is extracted regardless of what the content of the region or subregion is with respect to the position and size of the region or subregion on the frame. In other cases, the processor extracts the objects provided to the area, which objects may be extracted from any portion of the area that is physically present.

It should be noted that the mosaic frame does not necessarily need to be received from the broadcast TV signal. The receiver may comprise communication means for reception and will also transmit digital video and audio content from the Internet.

The extracted video information can be used in addition to the display in many ways. The video processor may include a picture-in-picture processor (P-in-P) (not shown) or video switch, each operating as known from US5.633.683. The extracted video information is additionally applied to the video switch. Video processor 150 may be suitably programmed to perform all of the functions described herein. The audio and video output of the receiver 100 is further provided to an AV device for rendering audio and / or video content.

3 shows an example of a frame 300 representing the extracted information. On frame 300, there is a main screen 310 where the program selected by the user or in another way is shown. The secondary screen 320 is smaller than the main screen 310, for example, because the user does not like the TV program shown on the secondary screen as much as the program on the main screen. The video content shown on the secondary screen provides the user with information broadcast on different TV channels and can be extracted, for example, from the mosaic video signal as described above. The frequency of refreshing the information on the secondary screen should be sufficient to inform the user, for example one frame of each TV channel per second, or one frame every 1/2 second. The frequency depends largely on many factors such as the number of tuners 110 available at the receiver 100, the processing power of the processor 150, the type of information provided to the receiver, and the like. It is also possible for the live video content of "live" to be shown on the primary and secondary screens.

The extracted sub-regions may be shown in the same auxiliary region as the extracted region. Both the extracted area and the sub area can be scaled or mapped to an area on the display screen predetermined by the user or the system. In one example, the CNN “ticker tape” with the “moving text” of the news is extracted from the mosaic 240 and shown on the secondary screen 330.

The user may change the arrangement of the auxiliary screen 320 in the frame 300. The position of the auxiliary screen can be changed and the screen can be moved within the frame area. For example, the user wants to have a screen 330 on the upper side of the frame 300. For this purpose, the remote control unit can have a special button for switching the TV set to the editor mode. Display menus with commands for creating, deleting, moving, resizing, etc. new auxiliary screens of the auxiliary and / or main screens can be used in the editor mode. The user can also indicate to the information source the rerenewal frequency or other parameters to be shown on each screen 310, 320 or 330. The user may select an arrangement in which the extracted area or sub-area is scrolled sequentially through the sub-screen area. The boundary, edges of the auxiliary region may be made bold, accentuated, or the like. The result of editing the placement of frame 300 is further passed to a video processor for controlling the extraction and proper presentation of video content.

Another example of displaying the extracted information is shown in FIG. 4. Sub area 410 is selected for extraction from frame 400 of TV channel "A". Sub-region 410 is shown on frame 450 and is resized to sub-region 460 with a black mirror arrangement. The image on the screen 460 may be shown in scaled or semi-transparent form and may be distinguished by black borders from other TV programs of channel "B" shown on the rest of the screen. Thus, a user can watch content from two channels A and B.

In one of the embodiments, the data retrieval system comprises a receiver according to the invention. A portion of the region or subregion may be stored in memory along with a description of its content. For example, "ticker tape" 240 may be identified in the system with several descriptors, such as "CNN news headlines", "news banners", and the like. In general, this can be done for TV programs in which such sub-regions are permanently present. The location of such subregions in the frame can be stored with the above description. The user can also provide a false-name to the sub area. The search interface for searching for a desired sub-region on the display can be enabled at the request of the user. Thus, whenever the user wants to watch such a sub area on the screen, the user can easily search for the sub area.

Other implementations that provide similar functionality can be replaced with the implementation described above without departing from the scope of the present invention. Various program products may implement the functions and methods of the systems of the present invention and may be combined with hardware or located on different devices in various ways. Modifications and variations of the described embodiments are possible within the scope of the inventive concept.

As described above, the present invention includes a receiver arranged to receive a video signal, wherein at least one video frame of the video signal includes at least one region corresponding to each one of a plurality of broadcast data sources, It is used for a video signal processing system.

Claims (14)

A video signal processing system, comprising: a receiver arranged to receive a video signal, wherein at least one video frame of the video signal includes at least one region corresponding to each one of a plurality of broadcast data sources In the system, By using an image analysis algorithm, a processor is arranged to process the video frame to extract at least a portion of the region from at least one of the video frames. And a video signal processing system. The video signal processing system of claim 1, wherein the area or portion of the area has a rectangular shape. 3. The video signal processing system of claim 1 or 2, wherein the processor is arranged to perform detection of an edge on the area or portion of the area. 4. The video signal processing system of claim 3, wherein the processor is further arranged to perform detection of a line defining the area or portion of the area from each frame. 5. The video signal processing system of claim 3 or 4, wherein the processor is further arranged to perform detection of an edge on a plurality of sequential frames for the region or portion of the region corresponding to the same source of data sources. . 2. The video signal processing system of claim 1, further comprising a marker for user-operably indicating the region or portion of the region to be extracted from at least one of the video frames. 7. The apparatus of any one of claims 1, 4 or 6, further comprising identification means for identifying a correspondence to one of the plurality of broadcast data sources of the extracted region or portion of the region. , Video signal processing system. The video signal processing system of claim 1, further comprising presentation means arranged to show at least one extracted region or portion of the region mapped to each screen region. The system of claim 1, wherein the system is further configured to receive a video frame including at least one area corresponding to each one of the plurality of broadcast data sources, and to receive a video frame from a selected broadcast data source among the broadcast data sources. Operable to switch between, Said presentation means being arranged to show said received video frame of said selected broadcast data source on a main screen area and to show at least one extracted area or part of said area on at least one of a plurality of auxiliary screen areas, Video signal processing system. 2. The apparatus of claim 1, further comprising an additional receiver arranged to receive a video frame of a video signal from a selected one of the broadcast data sources, The presentation means provides the video frame of the selected broadcast data source received by the additional receiver on a main screen area, and at least one extracted area or part of the area on at least one of a plurality of auxiliary screen areas. And a video signal processing system. 11. The method of claim 9 or 10, wherein the presentation means is arranged to user-operably define a representation of at least one extracted area or portion of the area in each one of the plurality of auxiliary screen areas. , Video signal processing system. 12. The video signal processing system according to claim 11, wherein the presentation means is arranged to allow a user to define the size and / or location of at least one auxiliary screen area. A receiver arranged to receive a video signal, the at least one video frame of the signal comprising at least one region corresponding to each one of a plurality of broadcast data sources, the receiver comprising: And the receiver comprises a processor arranged to process the video frame to extract at least a portion of the region from at least one of the video frames. A computer program product, when performing the computer program product, causing a programmable device to act as the system of claim 1.