JP2010147509A - Video processor and video distribution system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To notify a viewer about presence of an important scene included in a scene skipped by fast-forward reproduction by the viewer, in viewing a video. <P>SOLUTION: An object in a reproduced video is stored previously as a preference object of a user in reproducing the video. When the user executes fast-forward reproduction, a video in the fast-forwarded part is checked against the user preference object. If there is a scene where an object suiting the user preference appears in the video in the fast-forwarded part, video display of the scene or display by characters is carried out; and information notification is executed to the user. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a video processing apparatus and a video distribution system that perform recording and reproduction such as a camera and a recorder, and more particularly to a video processing apparatus and a video distribution system having a function of supporting viewers by using video information.

  In recent years, with the increase in capacity of recording media, it has become possible to record moving images for a long time with a camera or a recorder. As for editing the recorded video, there are editing methods such as using the editing function attached to the camera or recorder that has performed recording, and capturing a moving image with a personal computer, generating and recording a new moving image. However, in the editing work, it is necessary to work while viewing video and audio in order to determine a scene to be left, and in the work of editing a moving image for a long time, much labor and time are required.

Therefore, a technique that enables automatic editing of moving images has been proposed.
For example, Japanese Patent Laid-Open No. 2000-350165 (Patent Document 1) is known as a technique for performing automatic editing using a user's reproduction history. This publication describes, as a problem, “Providing a moving image recording / reproducing apparatus that displays summary information of the contents of a recorded moving image using a representative image of a user's favorite scene”. “Recording / playback unit 110 that performs video recording and playback processing, control unit 120 that controls recording / playback unit 110, moving image storage device 31 that stores moving images, and control information that is used by control unit 120 And a control information storage unit 130 for storing the video, the video to be recorded is divided for each scene change, a representative screen is selected from each divided scene, and the importance of each scene is determined based on the user input operation type And a representative screen of a scene with high importance is displayed as image summary information. "

  Japanese Patent Laid-Open No. 2002-176613 (Patent Document 2) is known as a technique for extracting a portion in which a specific person is shown. In this publication, as a problem, “Providing a device for automatically editing a moving image” is described, and as a means for solving this problem, “in a moving image processing device in which a plurality of moving image files 142 are stored in a fixed disk 104, The determining unit 21, the extracting unit 22, and the synthesizing unit 23 are provided, and the determining unit 21 specifies a part in the moving image in which the image signal or the audio signal satisfies a predetermined standard, for example, a face of a specific person is displayed. A part, a part that has been zoomed up at the time of shooting, a part whose volume exceeds a predetermined value, etc. are specified, etc. The extraction unit 22 extracts a partial moving image including the specified part from the moving image file 142. The synthesizing unit 23 combines the partial moving images to generate the edited file 143. Thereby, automatic editing of the moving images is realized. "

JP 2000-350165 A JP 2002-176613 A

  In Patent Document 1, scene weighting is performed according to a user's playback history (normal playback, slow playback, high-speed playback, reverse playback, and stillness), and by dividing the video based on this weighting, the user's A technique for automatically editing video that reflects preferences is disclosed. In this method, the user does not need to be aware of editing work. Scenes you are not interested in will be skipped by fast-forwarding, and you will watch the scenes that interest you. This is because the editing operation is automatically performed only by this. However, there may be a case where there is an important scene in the fast-forwarded scene that is not of interest to the user. When a scene is fast-forwarded, the priority of the scene is naturally set low. Even if a scene that deserves the best shot is included in the fast-forwarded scene, the user ends without noticing. In addition, since the priority is set low, the video is not automatically edited. As a result, although a good scene is actually recorded, it is overlooked without viewing it. This is a problem that particularly occurs when fast-forward playback is performed in units of 5 seconds, 10 seconds, 30 seconds, etc. in order to roughly check the content of the recorded video. In this case, since the user can view the recorded video only in fragments, it becomes easy to overlook important scenes.

  Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique for editing a video by automatically extracting a scene in which the person is reflected by registering a video of the person that the user wants to extract. By combining with the above-mentioned Patent Document 1, it is possible to view without missing a scene in which the person intended by the user is shown. However, in this method, the user needs to register the video of the person to be extracted. The video to be registered must satisfy certain conditions. For example, if the front face of the person to be extracted is not registered, detection may be difficult and extraction may not be possible. Therefore, the user needs to consciously perform the registration work as in the case of editing work.

  The present invention has been made in consideration of the above points, and proposes a method of notifying the viewer of the existence of an important scene included in a scene that the viewer has skipped when viewing a video.

  The above object can be achieved by the configuration described in the scope of the claims as an example.

  According to the present invention, when an important scene is included in a scene that the viewer skips in viewing the video, it is possible to notify the viewer of the existence of the scene. As a result, the viewer can watch without missing an important scene, and the usability is improved.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a video processing apparatus according to the first embodiment of the present invention. In FIG. 1, 1 is a video input unit, 2 is an operation IF, 3 is a recording / playback control unit, 4 is a recording medium, 5 is a video synthesis unit, 6 is a video output unit, 7 is a monitor, 8 is a user preference detection unit, 9 is a user preference storage unit, 10 is a scene detection unit, and 11 is a notification information generation unit.

  Here, the configuration of the present invention will be described with reference to FIG.

  The video captured from the video input unit 1 is recorded on the recording medium 4 by the recording / playback control unit 3. The video recorded on the recording medium 4 can be reproduced by operating the operation IF 2. When reproducing the video, the recording / reproduction control unit 3 reads the reproduced video from the recording medium 4, outputs the reproduced video from the video output unit 6, and displays it on the monitor 7. At the time of video playback, the user preference detection unit 8 extracts a subject from the playback video and stores it in the user preference storage unit 9 as a user preference target. In the scene detection unit 10, when the user performs fast-forward playback, the user preference target stored in the user preference storage unit 9 is compared with the video of the fast-forward portion. If there is a scene in the fast-forward portion where the user preference object is present, the notification information generation unit 11 generates notification information, and the video synthesis unit 5 synthesizes it with the playback video and outputs it to the user. Information notification is performed.

  Next, a specific processing flow when the present invention is applied will be described.

  FIG. 2 is a diagram showing an example of a processing flow when the video processing apparatus according to the first embodiment of the present invention is used. The flow of processing is shown in order. First, the user starts reproduction (SP1). During reproduction, subject extraction from the reproduced video is performed (SP2). The subject to be extracted may be anything such as a person, a figurine, or a vehicle. For example, when the subject is a person, the presence or absence of the subject can be determined and extracted by a conventional method such as a background difference method or a skin color detection method. Further, when the subject is a moving object such as a car, the presence / absence of the subject can be determined and extracted by a conventional method such as a motion detection process for extracting a difference between images between frame images. Feature points are extracted from the subject extracted in this manner and managed for each subject. For example, when the subject is a person, characteristic parts such as eyes, eyebrows, nose, mouth, and characters drawn on clothes are stored as template images and managed as constituent elements of the subject. By having such management information, when the same subject appears in the video, it is possible to determine that it is the same subject by performing pattern matching. Therefore, it is possible to count the reflected scenes for each subject. A subject having a large number of extracted scenes, that is, a subject whose time shown in the video is a predetermined time (SP3) or more is stored as a user's preference target (SP4). If fast-forward playback has been performed (SP6), it is determined whether or not the user's preference target is reflected in the video portion of the fast-forward playback (SP7). The image of the part that has been fast-forwarded is compared with the management information (template image) of the user's preference object, and the determination by pattern matching is performed. Here, when it is determined that the user's preference object is reflected, this is notified to the user (SP8). The above processing is repeated until the playback is stopped (SP5) and the playback is ended (SP9). Note that information such as the number of scenes of each subject in the playback video, the management information of each subject, and the user's preference target is retained regardless of the various operations and power status of the device. This improves usability. In addition, here, a method of using template matching for extracting a subject in a video has been shown, but any other method may be used. In this embodiment, an example in which a subject having a certain time or longer in the video is set as a user's preference target is shown, but the user's preference target may be determined according to the user's reproduction operation. For example, when the user performs fast-forward playback, it can be considered that the user's preference object disappeared from the video immediately before that. That is, in this case, it is possible to detect the user's preference target by extracting the subject from the video immediately before fast-forward playback.

  Next, a method of information notification to the user when the present invention is applied will be described.

  FIG. 3 is a diagram showing an example of information notification when the video processing apparatus according to the first embodiment of the present invention is used. Here, a situation is considered in which fast-forward playback is performed during video playback and the screen returns to the playback screen after fast-forward playback is completed. When the user's preference object is shown in the video of the fast-forward playback portion, the scene is displayed on the screen as a thumbnail video. FIG. 3A shows an example in which thumbnail images are displayed in a small superimposed manner with a playback screen as a background. FIG. 3B shows an example in which the playback screen is reduced and thumbnail images are displayed in a small size in the remaining area. Although FIG. 3 shows an example in which two scenes are detected, each scene image is also displayed simultaneously when one scene or three or more scenes are detected. When there are many detected scenes, each scene image may be displayed sequentially. By performing such a screen display, it is possible to easily notify the user of information. Upon receiving this notification, the user can confirm again the scene that has been fast-forwarded. Note that the thumbnail video may be a still image or a moving image. In the case of still image display, displaying a scene in which the user's preference object is greatly displayed makes it easier to understand. Further, in the case of moving image display, only the scene in which the user's preference object is shown may be reproduced, and at the same time as the corresponding scene ends, the process may shift to still image display.

Here, the method of notifying the user of information is not limited to thumbnail images, and character information may be used. FIG. 4 is a diagram showing a different example of information notification when the video processing apparatus according to the first embodiment of the present invention is used. Here, as in FIG. 3, a situation is considered in which fast-forward playback is performed during video playback and the screen returns to the original playback screen after fast-forward playback is completed. FIG. 4A is an example of a case where the message “There was an important scene” is displayed when the user's preference object is shown in the video of the fast-forward playback part. FIG. 4B shows an example in which a thumbnail video is displayed as in FIG. 3, and an example in which the time code of the corresponding scene is simultaneously displayed. In this way, not only the video but also the character information is displayed on the screen, so that the user can be notified of information more easily.
In the present embodiment, a process of extracting a subject that is a user's preference target at the time of playback and matching with a video of a part that has been fast-forward played has been shown. However, the extraction of the subject may be performed at the time of recording. In this case, it is not necessary to detect the subject that is being played back, and the user preference target can be acquired by referring to the subject extraction information at the time of recording. Furthermore, with respect to matching with a fast-played portion of the video, it is possible to easily detect the presence / absence of a user preference target by referring to the subject extraction information at the time of recording. As for information notification using video and characters, any combination of display may be performed, and the display position is arbitrary. Note that the information notification may be set to be hidden after a certain period of time such as 5 seconds or 10 seconds, or may be set to be hidden by the user's instruction from the operation IF2. good.

  As described above, according to the present embodiment, when an important scene is included in a scene skipped by the user by performing fast-forward playback during video playback, the presence of the important scene is detected using video or character information. This makes it possible to notify the user in an easy-to-understand manner. This prevents the user from overlooking important scenes and improves usability.

  In the first embodiment, information notification of important scenes is shown using FIG. 3 and FIG. In this embodiment, an example of a method for reproducing the notified important scene is shown. FIG. 5 is a diagram showing an example of the processing flow when the video processing apparatus according to the second embodiment of the present invention is used. FIG. 6 is a diagram showing an example of playback video switching when the video processing apparatus according to the second embodiment of the present invention is used. Here, the flow of the process in a present Example is shown in order. First, the important scene of the fast forward playback portion is displayed (SP10). Next, the user selects a video of an arbitrary important scene (SP11). An example of the screen display at this time is shown in FIG. Here, a case where two important scenes are superimposed and displayed as thumbnail images with a playback image as a background is shown. The user selects an important scene video to be reproduced by the operation IF2. As a selection method, a method of selecting a target by displaying a frame line 13 on the selection target as shown in FIG. When the operation IF 2 is equipped with a button capable of selecting up, down, left, and right, the user can move the frame line 13 to another object by pressing the left and right selection buttons. With the frame line 13 attached to the target to be reproduced, the target image is started to be reproduced by performing a decision operation from the operation IF2 (SP12). An example of the screen display after switching the playback video is shown in FIG. The selected video is played over the entire screen, and the original playback video is displayed as a small thumbnail video. At this time, if the time code of the original reproduced video is stored, the user can reproduce the original reproduced video at an arbitrary timing. This is performed by instructing to stop the playback of the important scene from the operation IF2, or by selecting the original playback video displayed as the thumbnail video using the frame line 13 as shown in FIG. . By returning to the original reproduced video in this way (SP13), the original reproduced video screen is restored (SP14). It should be noted that after the playback of the important scene is completed, the original playback video screen may be automatically returned. In addition, although an example in which the original playback video is displayed as a small thumbnail video on the playback screen of the important scene has been shown, the original playback video may not be displayed.

  As described above, according to the present embodiment, when an important scene is included in a scene skipped by the user by performing fast-forward playback during video playback, the important scene can be easily played back. Become. This improves usability.

  In Example 1, the configuration of the present invention is shown using FIG. Among them, the user preference detection unit 8 indicates that a subject is extracted from the video being reproduced and stored in the user preference storage unit 9 as a user preference target. However, there are many subjects in the playback video that are not the user's preference targets, and it is not preferable to store these subjects as preference subjects. In view of this, a method of handling a portion of the video that is being watched by the user as a subject extraction target video in the playback video will be described.

  FIG. 7 is a diagram showing an example of the processing flow when the video processing apparatus according to the third embodiment of the present invention is used. The flow of processing is shown in order. First, the user starts reproduction (SP15). The user may perform operations such as slow playback and pause during playback. For example, a scene in which slow playback has been performed is considered to be a scene in which a user's preference object is shown. Similarly, a temporarily stopped scene can also be considered as a scene in which a preference object is shown. Further, if there is a scene that is repeatedly played, it is considered that the preference object is also reflected in that scene. That is, the case where playback is performed while performing a specific operation in this way means that the user is paying attention, and it is considered that the user's preference object is reflected in the video. Accordingly, subject detection is performed and user preferences are stored (SP18, SP17), slow playback (SP19), and paused (SP21), respectively, when the scene is frequently played back (SP18). 20 and 22), it is possible to exclude other scenes from being detected as preference targets. This process is repeated until the reproduction is stopped (SP16) and the reproduction is ended (SP23). This makes it possible to detect user preferences with high accuracy.

  Here, a scene with high playback frequency, slow playback, or pause is considered as the video part that the user has paid attention to, but other specific operations may be considered as the user's attention.

  As described above, according to the present embodiment, it is possible to estimate the user's favorite video portion from the video reproduction method, and to detect the user's preference target from the video, so that the user preference can be accurately detected. It becomes possible. This improves the detection accuracy of important scenes.

  In the first embodiment, a method is described in which information notification is performed at the end of fast-forward playback when there is an important scene in which the user's preference is shown in the video skipped by performing fast-forward playback. In this embodiment, a method for generating and displaying a screen for displaying a list of detected important scenes will be described.

  FIG. 8 is a diagram illustrating an example of the flow of processing when the video processing apparatus according to the fourth embodiment of the present invention is used. The flow of processing is shown in order. First, the user starts reproduction (SP24). The user can display the list of important scenes (SP29) by instructing to display the list of important scenes from the instruction IF2 (SP25). A list of important scenes is created in SP26-28. First, when the user performs fast-forward playback (SP26), it is determined whether or not the user's preference object is reflected in the fast-forwarded video (SP27). If it is reflected, addition to the important scene list is performed (SP28). . By repeatedly executing SP26 to SP28, the important scene list is updated as needed. An example of the important scene list screen generated here is shown in FIG. Here, the thumbnail video of the detected important scene and the time code of the scene are displayed. Although three scenes are displayed on the list display screen in FIG. 9, any number of scenes may be displayed simultaneously, and page change display may be performed when there are many detected scenes. By generating such a list display screen, the user can confirm an important scene at an arbitrary timing. Accordingly, it is not necessary to display the reproduced image and the important scene simultaneously as shown in the first embodiment. This can be realized by installing a function of disabling information notification at the end of fast-forward playback as a setting function of the apparatus when generating a list display of important scenes.

  As described above, according to this embodiment, when an important scene is included in a scene skipped by the user by performing fast-forward playback during video playback, a list of important scenes is automatically generated. Is possible. As a result, the user can check the important scene at an arbitrary timing, which improves usability.

  FIG. 10 is a block diagram showing a video distribution system according to the fifth embodiment of the present invention. In FIG. 10, 4 is a recording medium, 7 is a monitor, 8 is a user preference detection unit, 11 is a notification information generation unit, 17 is a video transmission unit, 16 is a server, and 18 is a video search unit.

  In the first to fourth embodiments, the detection of the user preference target and the presentation of the user preference target included in the scene skipped by fast-forward playback are shown for the video processing apparatus having the recording / playback function. In the present embodiment, a video distribution system that detects a user preference target, searches a distribution video for a video that matches the user preference, and presents the video to the user.

  Here, the configuration of the present invention will be described with reference to FIG.

The user views the video distributed by the server 16 on the monitor 7 via the Internet. When the user views the video recorded in the storage medium 4, the user preference detection unit 8 detects the user preference target from the viewed video. Further, the video search unit 18 searches the storage medium 4 for a video corresponding to the user preference target, and if there is a corresponding one, generates the notification information by the notification information generation unit 11 and then sends it to the monitor 7. And send information to the user.
Next, a specific processing flow when the present invention is applied will be described.

  FIG. 11 is a diagram showing an example of the processing flow when the video distribution system according to the fifth embodiment of the present invention is used. The flow of processing is shown in order. First, the user starts viewing (SP30). The user preference detection unit 8 detects the user preference target from the viewing video, and based on this result, the video search unit 18 searches the storage medium 4 for a video that matches the user preference target (SP31). Here, as shown in the first embodiment, the detection of the user preference target and the search for the video corresponding to the user preference target are performed by detecting the subject from the video and storing the result as the user preference target. This makes it possible to search for videos showing user preference targets. If there is a user-preferred video as a result of the video search, this is notified to the user (SP32, 33). As a notification method, as shown in the first embodiment, the search result image may be displayed as a thumbnail, or may be notified by character information. When the user gives an instruction to reproduce the search result, the corresponding image is reproduced and displayed (SP34, 35). The instruction of the video to be played and the playback method can also be realized by the method shown in the first embodiment. When there is no user's preference target video, when the preference target video is not reproduced, and when the reproduction of the preference target video is finished, the viewing ends (SP36).

In addition, about the search of the image | video suitable for a user preference object, you may perform together with the detection of a user preference object, and you may make it start by a user's instruction | indication. Further, regarding the video recorded on the recording medium 4, when the server 16 holds metadata, this may be used for searching.
As described above, according to the present embodiment, by providing a function for detecting a user's preference target on a video distribution server, it is possible to easily provide a video that matches the user's preference, improving usability. To do.

  In addition, this invention is not limited to an above-described Example, Various modifications are included. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Further, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment.

  The present invention can be applied to any video processing apparatus having a video fast-forward playback function.

1 is a block diagram illustrating a video processing apparatus according to a first embodiment of the present invention. It is a figure which shows an example about the flow of a process at the time of utilizing the video processing apparatus which concerns on 1st Example of this invention. It is a figure which shows an example about the information notification at the time of utilizing the video processing apparatus which concerns on 1st Example of this invention. It is a figure which shows a different example about the information notification at the time of utilizing the video processing apparatus which concerns on 1st Example of this invention. It is a figure which shows an example about the flow of a process at the time of utilizing the video processing apparatus which concerns on 2nd Example of this invention. It is a figure which shows an example about the reproduction | regeneration video switching at the time of utilizing the video processing apparatus based on 2nd Example of this invention. It is a figure which shows an example about the flow of a process at the time of utilizing the video processing apparatus which concerns on 3rd Example of this invention. It is a figure which shows an example about the flow of a process at the time of utilizing the video processing apparatus which concerns on 4th Example of this invention. It is a figure which shows an example about the display of the important scene list screen at the time of utilizing the video processing apparatus which concerns on 4th Example of this invention. It is a block diagram which shows the video delivery system which concerns on 5th Example of this invention. It is a figure which shows an example about the flow of a process at the time of utilizing the video delivery system which concerns on 5th Example of this invention.

Explanation of symbols

1 Video input unit 2 Operation IF
3 Recording / playback control unit 4 Recording medium 5 Video composition unit 6 Video output unit 7 Monitor 8 User preference detection unit 9 User preference storage unit 10 Scene detection unit 11 Notification information generation unit 12 Important scene notification screen 13 Video selection frame 14 Important scene playback Screen 15 Important scene list display screen 16 Server 17 Video transmission unit 18 Video search unit

Claims (16)

A video processing device capable of fast-forward playback of video,
A user preference detection unit for detecting a user's preference target from the video;
A user preference storage unit that stores user preference targets detected by the user preference detection unit;
Based on the user's preference object, a scene detection unit that detects a scene in which the user's preference object is shown from the video being reproduced;
An output unit that outputs at least one of the reproduced video or the detection result of the scene detection unit, and
The scene detection unit performs the detection on the video of the fast-played part;
A video processing apparatus characterized by the above. The video processing apparatus according to claim 1,
The user preference detection unit
A subject extraction unit for extracting a subject from the playback video;
An extraction number storage unit for storing the number of extractions of the subject;
A determination unit that determines a subject whose number of extractions has reached a certain level as a user's preference target,
A video processing apparatus characterized by the above. The video processing apparatus according to claim 1,
The user preference detection unit detects a subject disappeared from the video in a scene immediately before the user performs fast-forward playback as a user preference target;
A video processing apparatus characterized by the above. A video processing apparatus according to claim 2 or claim 3, wherein
The information of the user preference storage unit and the extraction number storage unit can be retained without depending on various operations of the apparatus and the power supply state,
A video processing apparatus characterized by the above. The video processing apparatus according to claim 1,
The output unit outputs the playback video and the detection result of the scene detection unit so as to be displayed in different areas;
A video processing apparatus characterized by the above. The video processing apparatus according to claim 1,
The output unit superimposes and outputs the playback video and the detection result of the scene detection unit;
A video processing apparatus characterized by the above. The video processing apparatus according to claim 1,
When two or more scenes are detected by the scene detection unit,
The output unit displays the detected scenes sequentially or simultaneously;
A video processing apparatus characterized by the above. The video processing apparatus according to claim 1,
The output unit outputs the detection result of the scene detection unit so as to be displayed as character information;
A video processing apparatus characterized by the above. The video processing apparatus according to claim 1,
The output unit has a function of outputting the detection result of the scene detection unit for a predetermined time;
A video processing apparatus characterized by the above. The video processing apparatus according to claim 1,
From the detection result of the scene detection unit, having a function of reading and reproducing the detected scene;
A video processing apparatus characterized by the above. The video processing apparatus according to claim 10,
When the scene detected by the scene detection unit is read and played back, the time code of the video before playback is stored,
A video processing apparatus characterized by the above. The video processing apparatus according to claim 11,
When the scene detected by the scene detection unit is read and played back, the video at the position of the time code is automatically displayed after the playback ends,
A video processing apparatus characterized by the above. The video processing apparatus according to claim 1,
The user preference detection unit performs a process according to a user's reproduction method, and detects a user preference from a reproduced video portion focused on by the user;
A video processing apparatus characterized by the above. The video processing apparatus according to claim 1,
Having a function of generating and outputting a screen listing the detection results of the scene detection unit;
A video processing apparatus characterized by the above. The video processing apparatus according to claim 1,
The output unit has a function of displaying only one of a playback video and a detection result of the scene detection unit;
A video processing apparatus characterized by the above. The user preference detection part as described in any one of Claims 1-3,
A video search unit for searching for a video including a user's preference target detected by the user preference detection unit;
A notification information generating unit that generates information for notifying a search result by the video search unit;
Providing
Video distribution system characterized by

Images