FR2898235A1 - METHOD FOR DISPLAYING INFORMATION EXTRACTED FROM A COMPOUND DOCUMENT OF REPORTS AND RECEIVER USING THE METHOD - Google Patents

Abstract

The invention relates to an image display method extracted from an audiovisual document recorded in an apparatus. The recorded document comprises at least one report comprising at the beginning a first video sequence having a particular characteristic, such as the image of a presenter. This first video sequence having only limited information content, the invention consists in selecting at least one image extracted from the report but not in the first video sequence, and in creating and displaying a composite image comprising both the first video sequence with the presenter and at least one image extracted from the report. An improvement consists in that this first video sequence comes from a summary of the document, images of the report are added to the images of the summary to increase the informative content. The invention also relates to the receiver capable of implementing the method.

Description

The invention relates to a method for displaying information extracted from a

  recorded document composed of reports, and a receiver implementing the method.

  Today, many devices allow the reception and reproduction of recorded audiovisual works. Recently, high-capacity hard drives have been integrated into these audiovisual receivers; stored contents can be stored either from a unidirectional broadcast network or from a point-to-point network (Internet for example). Among these devices, PVRs (Acronyms for Personal Video Recorder) usually have an Electronic Program Guide (EPG for short) that allows a user to view the contents recorded in the hard drive and reproduce them to the user. using an order. Today, the size of the hard drives can store more than 200 hours of programs. This amount of available information requires the addition of new features in the PVR allowing inter and intra-fast navigation. The EPG allows you to group documents according to many criteria and present them in groups, for example: theme, type, language, parental control, etc. The classification is made using 20 metadata associated with the audiovisual content. Alternatively, the receiver analyzes the content and is able to classify it. Another example of storage means relates to video recorders that record audiovisual works on magnetic tapes. In this case, access to audiovisual content is sequential. On these products, it is possible either to immediately record the video being received on the current channel, or to schedule the recording of a broadcast event later. Regardless of how you store the content, the user interface allows you to control the selection and reproduction of saved documents. In a first step, the user selects, using a program guide, the program he wishes to view. Then, by pressing the play key it triggers the visualization of the work.

  In the case of digital recordings (for example: a DVD), the work is segmented into several chapters. In this way, the user can choose a chapter and start viewing at the beginning of the selected chapter. Pressing the next track key displays the visualization jumps to the next segment. Some broadcasts received by a broadcast network are also segmented, so that, once recorded in the receiver's hard disk, the user can select a specific segment to view it or navigate from one segment to another by not viewing as the first few seconds to search for the desired segment. If the transmission is received unsegmented, there are methods of segmentation to do so at the receiver. The patent application referenced FR02.13977 filed in France on October 28, 2002 describes a segmentation method that exploits the audiovisual content of the work is to extract parts having a semantic unit. The method described in this application, allows to develop a segmentation of sports videos at several levels. There are also methods for automatically extracting the tree structure of a movie or documentary video. Conventionally, the video is cut into planes based on abrupt changes in the color content of the images. Then, the shots are grouped into scenes based on the visual resemblance of the contents of the shots and their temporal proximity. When the recorded document is a television newscast, the indexing according to current methods will allow direct access to the reports. When a user selects a report, it is extracted from the memory of his device and sent to the screen. The reports begin visually with a close-up on the presenter of the newspaper. If the sound message that accompanies it is important because it introduces the report, the image gives no relevant information and does not allow the user to distinguish a report from another, nor to have a view of together, so that the user is aware of the report only by his auditory perception. One way to shorten this time would be to cut the introductory sequence of the report where the presenter is alone on the screen and begin directly with the images of the report. But this technique has two disadvantages: the introductory and informative sound message is lost, and no complete representation of the report is finally displayed. The present invention proposes an original interface for representing a report which preserves the structure and the elements of the report and which makes it possible to inform the user more quickly of the subject of the report. The invention relates to a display method in an audiovisual document recorded in an apparatus, the document comprising at least one report comprising at the beginning a first video sequence having a particular characteristic, such as the image of a 15 presenter. The method comprises the following steps: selecting an image at least extracted from the report but not the first video sequence; displaying a composite video sequence comprising both the first video sequence and at least one previously selected image; . In this way, the sound message delivered by the presenter is always present but its associated image no longer monopolizes the screen, because it is enriched with visual parts of the report. According to a first improvement, the reports begin with a summary comprising several first video sequences displaying the presenter. The method then consists in cutting the document recorded in several reports, in cutting the summary into parts announcing a report, each part of the summary being thus associated with a report and finally in displaying the composite video sequence while at the same time displaying a portion of the report. of the summary and an image previously selected in the report associated with this part of the summary. In this way, the user can view the summary of a newspaper by having on the screen also key images reports reported in this summary. According to a refinement, the user can navigate in the summary and in particular trigger the full-screen viewing of the associated report According to another improvement, the document comprising several reports, the user can interrupt the display of the composite video sequence which triggers a a new step of displaying the composite video sequence corresponding to the following report. In this way, the user can quickly reach a report that interests him. According to another improvement, the user can select a selected image as a key image of the report, which triggers the visualization of the sequence plan containing this key image. In this way, the user can quickly reach a part of the report whose key image has caught his attention. According to another improvement, the display of a composite video sequence lasts a determined duration at the beginning of each first video sequence. According to another improvement, the step of selecting at least one image of the report consists in selecting a middle image in a sequence plane. According to another improvement, the step 20 of displaying a composite video sequence also consists in displaying textual information broadcast in service information. According to another improvement, the step of displaying a composite video sequence also consists in displaying, in association with each selected image, a time indication showing the moment of appearance of this image within the report.

  The invention also relates to an audiovisual receiver comprising a central unit, a means for reproducing at least parts of an audiovisual document recorded in storage means, the document comprising at least one report comprising at the beginning a first video sequence presenting a particular characteristic, such as the image of a presenter. The receiver comprises a first means for selecting an image at least extracted from the report but not the first video sequence, and means for composing a composite image comprising both the first video sequence and at least one previously selected image .

  Other features and advantages of the invention will now become more apparent in the context of the following description of exemplary embodiments given by way of illustration with reference to the appended figures which represent: FIG. 1 is a diagram block of an audiovisual receiver for implementing the invention, - Figure 2 is a screen appearance showing the beginning of a report according to a preferred embodiment, - Figure 3 is a flowchart of the main steps taken by the browser according to a preferred embodiment, - Figure 4 is a flowchart of the main steps performed by the browser according to an alternative embodiment, Figure 5 is an illustration of the cutting of a recorded document.

  First of all, the operation of an audiovisual receiver 1 equipped with a display device 2 will be described. The receiver comprises a central unit 3 connected to a program memory 12, and an interface 5 for communication with a bus. high-speed local digital subscriber 6 for transmitting audio / video data in real time. This network is for example an IEEE 1394 network. The receiver can also receive audio / video data from a broadcast network through a reception antenna associated with a demodulator 4. The receiver furthermore comprises an infrared signal receiver. for receiving the signals of a remote control 8, a memory 9 for storing the downloaded data, and an audio / video decoding logic 10 for generating the audiovisual signals sent to the television screen 2. According to an exemplary embodiment, the memory 9 is a large capacity hard disk for recording audiovisual documents. According to another exemplary embodiment, the audiovisual document is contained in a removable medium such as a compact disk or a DVD disc, the medium being inserted into a reader device (not shown in FIG. 1) incorporated in the receiver. The remote control is equipped with the arrow keys: and F and function or validation keys such as: Summary, OK and STOP. According to a preferred embodiment, the direction arrows make it possible to move a cursor on the screen, the commands accessible to the user are presented in the form of icons and are activated when the user presses OK with the cursor positioned on the screen. the icon corresponding to the desired command. The receiver also includes a circuit 11 for displaying data on the screen, often referred to as the OSD circuit, of the English "On Screen Display" (literally meaning "on-screen display"). The OSD circuit 11 is a text and graphics generator which makes it possible to display on the screen menus, pictograms (for example, a number corresponding to the channel displayed) and which makes it possible to display the navigation menus according to to the present invention. The OSD circuit is controlled by the Central Unit 3 and a browser 12. The browser 12 is advantageously made in the form of a program module stored in a read-only memory. It can also be realized in the form of a specialized circuit 25 of the ASIC type for example. The digital bus 6 and / or the broadcast network transmit to the receiver data including audiovisual documents. Eventually, these documents are associated with descriptive data. These data come either from a broadcast network or digital network 6. Descriptive data include segmentation information defining the various segments of the document. Advantageously, the descriptive data allow a fine decomposition of the document and include text making it possible to give at least one title to each segment. The descriptive data is for example contained in the service information specified by the DVB-SI standard. They are recorded in the memory 9 of the receiver.

  After describing the elements constituting the invention according to a preferred embodiment, we will now explain how these cooperate, with the help of FIG. 3. In a first step, the user records the television newscast in the memory 9 of its receiver 1 (step 3.1). The user then starts the execution of his resident browser from his receiver and requests the viewing of the recorded document, this document consisting of at least one report. Typically in a newscast, a story starts with a video showing the presenter introducing the topic to be treated orally. At the visual level, no relevant information is provided. At the sound level, on the other hand, the presenter's words are essential for the user to determine the subject of the report. In step 3.2, the browser selects one or more images extracted from the body of the report. These images do not show the presenter, their visual content is rich in teaching. According to the present invention, the receiver 1 will develop a new image 2.1 by resizing the image of the presenter 2.2 which appears at the beginning of the report in order to free up space on the screen to display other images 2.3, which are fixed there, extracted at other times of the report (step 3.3). An example of this new so-called composite image 2.1 is shown in FIG. 2. A 2.2 reduction of the video with the present presenter can be seen at the top left of the screen. In the lower part of the screen, thumbnails 2.3 automatically 30 extracted from the report and representative of its visual content are displayed. In the example of Figure 2, twelve thumbnails 2.3 are displayed that correspond to twelve particular moments of the report. Other arrangements are also possible, for example the presenter appears in the middle of the screen without size change, and the thumbnails are arranged in incrustation all around him. It is preferable that the arrangement of the thumbnails respects the chronological order of their appearance in the report. If necessary, they appear with a number or by displaying the moment of appearance in the report. The moments of appearance in minutes and seconds of the images displayed on the first line are displayed in the example illustrated in FIG. 2. The user therefore has at his disposal both audio and visual information summarizing the content of the report. and allowing him to guide him in his choice of viewing the full story. When the presenter no longer appears on the screen, it becomes unnecessary to display the composite image 2.1 and the viewing of the report continues in a conventional manner, without image processing at the receiver. If the user in view of the composite image 2.1 does not want to view this report, he presses the - key, which ends the viewing of the report. The browser then displays the following report, the screen having the same appearance as that shown in Figure 2. A first way to extract thumbnails is to analyze the reports to determine the different sequences. A shot shot (or shot in English) is an uninterrupted sequence shot by a camera or a camcorder in a single shot. The thumbnails displayed are the first images of the clips of the 25 reports. Other techniques are usable. For example, the browser may choose to take the middle image in the sequence plane, that is, the image that is in the middle of the plane. Another technique is to extract the images corresponding to a minimum of activity. Activity measurement is a technique known to those skilled in the art, it is generally based on movement in the images. You can also select thumbnails with close-up faces. Other techniques based on semantic analysis of video content can also be used. According to a preferred embodiment, the images are an inanimate photo extracted from the report. This mode is easy to produce with simple inlay methods. Alternatively, a thumbnail is a sequence shot selected in the report. As a result, at the beginning of the report and while the presenter introduces the subject, the sequence shots appear one after the other, thus giving an impression of movement. The clip plane disappears from the screen when it finishes. A few seconds later, another clip shot appears in another place or another. The appearance and disappearance of animated images add a fun side to the screen appearance. Tests have shown that three or four sequence map views provide a good compromise between the screen occupancy and the amount of visual information available to the user.

  The first part of a newscast usually consists of a summary in which the presenter presents the main stories that will be developed in the rest of the newspaper. Alternatively, the receiver 1 modifies the appearance of the summary video sequences by adding thumbnails so that the user perceives more information about the subjects than the information given only by the presenter. According to this variant, a report is divided into two: the video sequence of the summary where the presenter introduces the subject of the report, and the body of the report which then appears in the newspaper, and from which the thumbnails are extracted. FIG. 4 presents a flowchart of the analysis steps of a recorded video document making it possible to generate a list of links between the summary and the reports and to display the thumbnails, for the implementation of improvement.

  In step 4.1, the video document is recorded in the audiovisual receiver 1. It is for example received from a broadcast network. In step 4.2, the video document is segmented into reports. Many methods in the literature offer solutions to this problem. For example, a possible technique consists, after cutting into shots of the video to identify the recurring shots showing a determined image, typically the presenter of the newscast. The different planes present between the two appearances of the presenter are grouped together in the same video segment characterizing a report. In another example, the video is split into planes, and then the shots are grouped into scenes based on the visual resemblance of the plan contents and their temporal proximity. We can observe the audio component by analyzing the sound environment (a report on a football match, an ecological documentary or the last intervention of a politician have 15 different characteristics), or the continuity of the voice of the same journalist commenting on his report. The different planes present between the two appearances of the presenter are grouped together in the same video segment characterizing a report. We thus obtain at the output of this module a list of reports as well as a list of plans containing the presenter. In step 4.3, the browser analyzes the document to determine the summary. Typically, the summary is at the beginning, more rarely at the end of the document. The summary can be a single sequence shot showing a front presenter. Optionally, a determined music is detectable in the background of the commentary. The search for the images of the summary can take place according to different hypotheses, for example: - the images of the summary contain rectangular thumbnails generally fixed at the top left or at the bottom right of the presenter, - the summary is diffused at the beginning of the newspaper before the appearance of the presenter in the form of a succession of video clips.

  Other visual recognition techniques exist. For example, a selection of all the plans preceding the first report is first made. On all of these plans, a thumbnail detection is implemented on all the images of each plan.

  This detection can be based on the fact that the vignettes appear in part of the image, the rest of the image remaining almost fixed. For example, an algorithm based on the use of images resulting from the difference between successive images associated with a detection of rectangular shape by mathematical morphology can be used. If the summary is distributed at the beginning of the newspaper and the images of reports do not appear inset in the image but in full screen, it is possible to use a detection method based on the fact that: the presenter gives the oral summary accompanying the summary, 15 which may make it possible to use speaker segmentation and then select the images where the presenter is speaking but is not visible; - the summary is generally between the credits and the first appearance of the presenter; in this case, the detection of the credits based on a detection of the music tracks or a prior knowledge of the generic can find the summary as anything that is not generic.

  Once the summary is detected, the next step is to detect the similarities between each portion of the summary and each report. The method here consists in comparing the images of the summary with the images of the reports. Standard techniques for extracting characteristic attributes of the image associated with a distance measurement make it possible to provide a measure of similarity that can be traced back to the association of summary images and reports. In many cases, the summary may be considered as video summaries of the associated reports, the invention may advantageously use the time component to make the image matching process more robust. For example, the order of presentation of reports in the summary is generally the same as that of newspaper reports. The measure of similarity is thus reinforced if the two images presenting similarities are presented in the same order, some being in the summary, the others in the reports. Once this stage is completed, couples formed from a part of the summary and a report are drawn up. Thanks to an adapted user interface, these associations will now be exploited in order to give more visual information to the images of the summary. Figure 5 illustrates the division of a document and the determination of the similarities between summary parts and reports on a television newscast. Following the segmentation, the treatment program determined that the summary had five parts and that the journal had five reports. By analyzing the similarities between the summary parts and the reports, the treatment program was able to establish four similarities: the first part of the summary and the first report, the second part of the summary and the second report, the third part of the summary and the fourth report and the fourth part of the summary with 20 the fifth report. In the example illustrated in Figure 5, the third report was not announced in the summary, this report can not be indexed by a portion of the summary, he saw at the time of full viewing of the recorded log.

  In step 4.5, the user has requested to view the document, the summary is displayed at first. The browser displays a composite image 2.1 by adding to the images of parts of the summary, images extracted from the associated report. The images extracted from the associated report are embedded in a screen window at the top right of the screen. In this way, the user perceives in addition to the sound content produced by the presenter a visual content informing him of the object of the associated report. The same image example as that shown in FIG. 2 is thus generated. The user perceives both audio and visual information summarizing the content of the report and allowing him to guide him in his choice to see a specific report. During the visualization of this part of the summary associated with this report, the receiver 1 transmits the sound of the summary. If the user presses the OK button on the remote control 8 during the viewing of this part of the summary, the receiver 1 extracts the report associated with this part from the memory and displays the video of this report in full screen by the beginning. At the end of the video, the visualization automatically returns to the next part of the summary. According to an improvement, the user presses the summary key, the visualization of the video stops, and the first thumbnail of the corresponding report is highlighted graphically. The user can then navigate to the thumbnails using the arrow keys. After having selected one, by pressing OK, the receiver 1 extracts from the memory the sequence of the report which begins with the selected thumbnail and displays it on the screen. As before, at the end of the report, the visualization automatically returns to the next part of the summary. According to one improvement, the thumbnails of the composite image 2.1 serve as an index to access the part of the report containing them. When the composite image 2.1 is displayed, the user presses the STOP key which freezes the image of the screen, the image of the presenter being then graphically highlighted. Using the navigation keys, the user navigates and highlights the thumbnails. Then, by pressing the OK key, the user triggers the display of the sequence shot of the report containing the image thus selected. When the end of the clip plan is reached, the screen again displays the composite image 2.1 at the moment when its display has been stopped.

  According to one improvement, the document comprises in association audiovisual data, textual information. Typically this information is disseminated for the hearing impaired and displayed at the bottom of the screen at the request of the user. This textual information contained in metadata is associated with reports. When the user displays the composite image 2.1 created by the receiver 1, an area of the screen displays at least a portion of the text contained in this additional information. Alternatively, the receiver 1 has a voice recognition means for analyzing the presenter's speech during the report and converting it to text. The text transcribed from the audio signals is displayed while viewing the contents. According to another improvement, the visualization of a composite image 2.1 as shown in FIG. 2 takes place during a determined period of time. Indeed, at the beginning of the part of the summary corresponding to a report, the presenter introduces the subject but at the end of this part, the user is well aware of the report and no longer needs to be informed by the thumbnails . So, these thumbnails can be removed and the image of the presenter can appear alone in full screen.

Tests have shown that after ten seconds, the subject is well introduced and the thumbnails provide little information in addition to what has been said. Although the present invention has been described with reference to the particular embodiments illustrated, it is not limited by these embodiments, but only by the appended claims.

It will be appreciated that changes or modifications may be made by those skilled in the art.

Claims (11)

claims   1. A method of display within an audiovisual document recorded in an apparatus, the document comprising at least one report comprising at the beginning a first video sequence (2.2) having a particular characteristic, such as the image of a presenter; characterized in that it comprises the following steps: - selection (3.2) of at least one image (2.3) extracted from the report which image does not belong to the first video sequence, io - display (3.3) of a sequence composite video (2.1) comprising both the first video sequence (2.2) and at least one previously selected image (2.3).   2. Display method according to claim 1, the reports being introduced by a summary comprising a plurality of first video sequences (2.2) having said particular characteristic; characterized in that it comprises within the apparatus the following steps: -découpage of the document (4.2) recorded in a plurality of reports, 20 - cutting of the summary (4.3) in parts announcing a report and association of each party with the corresponding report (4.4), the display step (3.3; 4.5) of the composite video sequence (2.1) of concomitantly displaying part of the summary (2.2) and at least one image (2.3) previously selected in the report . 25   3. Display method according to claim 2; characterized in that it further comprises a step of introducing a command at the time of displaying the composite video sequence triggering the full-screen viewing of the associated report. 30   4. Display method according to any one of the preceding claims, the document comprising several reports; characterized in that it comprises a step of introducing an interruption control of the display of the composite video sequence, triggering a new step of displaying the composite video sequence corresponding to the next report.   5. Display method according to any one of the preceding claims; characterized in that it comprises a step of introducing a command for selecting a selected image, triggering a step of displaying the sequence plane containing said selected image.   6. Display method according to any one of the preceding claims; characterized in that a composite video sequence is displayed for a specified duration at the beginning of each first video sequence.   7. Display method according to any one of the preceding claims; characterized in that the step of selecting at least one image at least of the report consists in selecting a middle image in a sequence plane.   8. Display method according to any one of the preceding claims; characterized in that the step of displaying a composite video sequence comprises also displaying textual information broadcast in service information.   9. Display method according to any one of the preceding claims; characterized in that the step of displaying a composite video sequence consists in also displaying in association with each selected image a time indication showing the moment of appearance of this image within the report.   Audiovisual receiver (1) comprising a central unit (3), means for reproducing (10) at least part of an audiovisual document recorded in a storage means (9), the document comprising at least one report comprising at the beginning a first video sequence having a particular characteristic, such as the image of a presenter; characterized in that it comprises a first means (3, 12) for selecting an image at least extracted from the report, which image does not belong to the first video sequence, and a composition means (3, 12,   11) of a composite image comprising both the first video sequence and at least one previously selected image.