JP2008104038A - Information processor, information processing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect a region which is considered that a feature of a picture is well represented. <P>SOLUTION: The same processing is not performed by regarding a picture of all regions of all frames constituting a program as a target, but when it is determined that a certain frame contains an important region considered to be important when viewing from a point of extracting the feature of the picture, the picture of the important region is intensively analyzed and the feature is extracted. Since a region containing a display region of a telop which comes to be displayed in the form of thrusting away subtitles is inserted so as to certainly show to a viewer from the producer side of the program, there is the high possibility that the feature of a scene is well represented and the region is designated as the important region. The present invention is applicable to a device for analyzing the picture. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an information processing device, an information processing method, and a program, and more particularly, to an information processing device, an information processing method, and a program that can detect an area that is considered to well represent a feature of a video. .

  In recording devices such as hard disk recorders, features are extracted by analyzing the video of the recorded television program, and chapter switching is automatically set based on the extracted features by detecting scene switching positions. And a function for automatically selecting a scene to be digest-played.

  For example, a position where the feature of the video has changed significantly is detected as a position where the content has changed, and a chapter is automatically set at that position. In addition, a section in which telops are frequently displayed is detected as an exciting section of the program, and is automatically selected as a section to be played when the user selects digest playback.

  The feature extraction is performed not only on the main video of the program in which people and scenery are reflected, but also on telops inserted in layers. In addition to extracting information on whether or not a telop exists as a feature, the contents (characters) are recognized and the recognized character is extracted as a feature. For example, when a person's name is included in the recognized characters, the name is managed as representing the name of the performer shown in the main video.

Patent Document 1 determines whether or not a still image is an artificial image based on color information, extracts character information included in an image determined to be an artificial image, and extracts the character information. A technique for determining whether an image is an important image from the background color of the image that is the target of the image is disclosed.
JP 2004-70427 A

  There is a tendency to frequently use telops as a trend of recent television programs, and there are some telops that are not so important in terms of extracting video features. In order to efficiently extract the features of the video, it is preferable to extract the features by focusing on important telops rather than targeting unimportant telops.

  The present invention has been made in view of such a situation, and makes it possible to detect a region that is considered to well represent the characteristics of an image.

  An information processing apparatus according to an aspect of the present invention includes: an acquisition unit configured to acquire a position of a subtitle display area included in each frame; and a subtitle display at a position away from an average position of the subtitle display area by a predetermined distance. And a designation unit that designates a predetermined area including an average position of a subtitle display area as an important area among frames including the area.

  It is possible to further provide feature extraction means for performing the process of extracting the video features of the important area designated by the designation means more than the process of extracting the video features of other areas.

  The feature extraction means can perform processing for extracting the feature of the video of the important area designated by the designation means in a large amount of time, amount of processing, or number of times.

  An information processing method or program according to one aspect of the present invention acquires the position of a caption display area included in each frame, and displays the caption display area at a position away from the average position of the caption display area by a predetermined distance. Including a step of designating a predetermined area including the average position of the subtitle display area as an important area.

  In one aspect of the present invention, the position of the caption display area included in each frame is acquired, and among the frames including the caption display area at a predetermined distance from the average position of the caption display area A predetermined area including the average position of the caption display area is designated as the important area.

  According to one aspect of the present invention, it is possible to detect a region that is considered to well represent the characteristics of an image.

  Embodiments of the present invention will be described below. Correspondences between the constituent elements of the present invention and the embodiments described in the specification or the drawings are exemplified as follows. This description is intended to confirm that the embodiments supporting the present invention are described in the specification or the drawings. Therefore, even if there is an embodiment which is described in the specification or the drawings but is not described here as an embodiment corresponding to the constituent elements of the present invention, that is not the case. It does not mean that the form does not correspond to the constituent requirements. On the contrary, even if an embodiment is described herein as corresponding to the invention, this does not mean that the embodiment does not correspond to other than the configuration requirements. .

  The information processing apparatus according to one aspect of the present invention (for example, the information processing apparatus 1 in FIG. 1) acquires acquisition means (for example, the caption position determination unit 33 in FIG. 5) that acquires the position of the display area of the caption included in each frame. ) And designation specifying a predetermined area including the average position of the subtitle display area as an important area among frames including the subtitle display area at a predetermined distance from the average position of the subtitle display area Means (for example, the area designating unit 34 in FIG. 5).

  In this information processing apparatus, the feature extraction unit (for example, FIG. 5) performs more processing for extracting video features of the important region specified by the specifying unit than processing for extracting video features of other regions. Can be further provided.

  An information processing method or program according to one aspect of the present invention acquires the position of a caption display area included in each frame, and displays the caption display area at a position away from the average position of the caption display area by a predetermined distance. Among the frames including, includes a step (for example, step S4 in FIG. 6) of designating a predetermined region including the average position of the subtitle display region as an important region.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram showing an information processing apparatus 1 according to an embodiment of the present invention.

  As shown in FIG. 1, a television receiver 2 is connected to the information processing apparatus 1 via a cable. The information processing apparatus 1 includes a recording medium such as a hard disk, and records (records) a program provided by, for example, digital television broadcasting or broadcasting via the Internet on the hard disk. That is, the information processing apparatus 1 is supplied with a signal from an antenna. The information processing apparatus 1 reproduces a recorded program in response to a user operation performed using a remote controller or the like, and causes the television receiver 2 to output program video and audio.

  Further, the information processing apparatus 1 analyzes the received video of the program to be recorded and the video of the recorded program recorded on the hard disk, extracts features, and based on the extracted features The program has a function of dividing the entire program into scenes made up of predetermined video sections and setting chapters at the scene breaks. The user reproduces only a favorite scene by selecting a chapter automatically set by the information processing apparatus 1 without setting the user himself or performs a digest reproduction of only the scene selected by the information processing apparatus 1. be able to.

  In the information processing apparatus 1 having such a function, features are not extracted by performing the same content and the same level of analysis processing on the video of all areas of all frames constituting the program. If there is an important area that is considered to be important in terms of extracting video features in a certain frame, analysis processing is focused on the video in that important area, and the features are Extracted.

  For the video in the important area, a lot of processing such as time-consuming analysis processing is performed, or analysis processing is performed using a lot of data. Many processes are performed many times, such as many processes are performed or analysis processes are performed many times, and images of other areas of the same frame or areas of other frames Compared with the analysis, it is focused on. More detailed features may be extracted by performing complex content analysis processing on the important region video, or analysis processing may be preferentially performed from the important region video. .

  FIG. 2 is a diagram illustrating an example of an important area.

Videos P _{1 to} P _{7 in} FIG. 2 are representative pictures of a scene of a certain program. The frames shown in the images P _{1 to} P ₇ with the characters “caption” on the inside indicate that the caption is displayed in the area surrounded by the frames. For example, characters representing the content of the utterance of the performer shown in the main video are displayed as subtitles superimposed on the main video.

  Program data broadcast by digital television broadcasting includes caption data in addition to video and audio data, and the user can select on / off of caption display. In addition to text data (text data) to be displayed as subtitles, the subtitle data includes display timing and data for specifying a display position in the frame.

In the example of FIG. 2, the position of the caption displayed in the video P ₁ is the position below the frame. In normal times when no telop or the like is displayed, the subtitle is displayed at a position below the frame.

The position of the subtitle displayed in the video P ₂ is a position near the center of the frame since the telop “Happy Birthday !!”, which is an open caption, is displayed at a position below the frame. With the start of the “Happy Birthday !!” telop display, the subtitle position that was displayed at the lower position of the frame has moved to a position near the center of the frame, and the telop is displayed. Meanwhile, the subtitle continues to be displayed at a position near the center of the frame.

Similarly, in the example of FIG. 2, the position of the subtitle displayed in the video P ₃ is a position below the frame, and the position of the subtitle displayed in the video P ₄ is “failure → failure → Since the “prohibited” telop is displayed at a position below the frame, it is positioned near the center of the frame.

The position of the subtitle displayed in the video P ₅ is the position below the frame, and the position of the subtitle displayed in the video P ₆ is an open caption “Goodbye!” Telop displayed at the position below the frame. Therefore, the position is near the center of the frame. The position of the caption displayed on the video P ₇ is a position below the frame.

For example, when subtitles are displayed in this way, the area A ₁ of the video P ₂ , which is an area near the display area of the telop of “Happy Birthday !!” The area A ₂ of the video P ₄ that is the area near the display area of the “failure → failure → prohibited” telop and the area A ₃ of the video P ₆ that is the area near the display area of the “goodbye!” Telop are important areas. It is specified. Each of the areas A _{1 to} A ₃ is an area where a telop is displayed, but is also an area where captions are normally displayed, and a certain area including the barycentric position of the area where captions are normally displayed is Is designated as an important area.

  The telop that is displayed until the subtitle is pushed is inserted so that the producer of the program surely shows it to the viewer, so it is highly likely that the telop clearly represents the features of the scene. Therefore, features that are meaningful to be used as a reference for determining chapter settings and digest playback scenes are extracted by focusing on the area where such telop is displayed and the surrounding video. It becomes possible.

  There are telops that include a person's name, phone number, and geographical name as the telop displayed by pushing the subtitles. The person's name can be identified by analyzing the telop and recognizing characters. Can be extracted as features.

  FIG. 3 is a diagram illustrating the concept of important area detection.

  For example, the position of the center of gravity of the caption area, which is the area where the caption is displayed, is detected for each frame, and the important area is specified from the detected frame as the position of the center of gravity of the caption area is shifted from the average position. Is done. The important area is designated to include the average position of the center of gravity of the caption area.

In the example of FIG. 3, the position in the vertical direction (y-axis direction) of the center of gravity of the caption area in the video P ₁ is the position a. Similarly, the caption is displayed in the area where the position a is the vertical position of the center of gravity. Since the number of frames to be played is large, the position a is the average position of the center of gravity of the caption area.

In addition, the important areas of the areas A _{1 to} A ₃ are respectively designated from the images P ₂ , P ₄ , and P ₆ detected that the position of the center of gravity of the caption area is located away from the position a.

  A series of processing of the information processing apparatus 1 that designates an important area will be described later with reference to a flowchart.

  FIG. 4 is a block diagram illustrating a hardware configuration example of the information processing apparatus 1.

  A CPU (Central Processing Unit) 11 executes various processes according to a program recorded in a ROM (Read Only Memory) 12 or a recording unit 19. A RAM (Random Access Memory) 13 appropriately stores programs executed by the CPU 11 and data. The CPU 11, ROM 12, and RAM 13 are connected to each other by a bus 14.

  An input / output interface 15 is also connected to the CPU 11 via the bus 14. The input / output interface 15 is connected to a receiving unit 16, an input unit 17, an output unit 18, a recording unit 19, a communication unit 20, and a drive 21.

  The receiving unit 16 receives and demodulates the broadcast wave signal from the antenna 16A, and acquires MPEG-TS (Moving Picture Experts Group-Transport Stream). The receiving unit 16 extracts program data (program video, audio, subtitle data) to be recorded from the MPEG-TS, and outputs the extracted data to the recording unit 19 via the input / output interface 15. To do.

  The input unit 17 receives a signal from the remote controller, and outputs information representing the contents of the user's operation to the CPU 11 via the input / output interface 15 and the bus 14. In the CPU 11, various processes such as reproduction of a recorded program are performed in accordance with information supplied from the input unit 17.

  The output unit 18 causes the television receiver 2 to display, for example, video of a program received by the receiving unit 16 and decoded by the CPU 11 in software.

  The recording unit 19 includes, for example, a hard disk, and records various data such as a program executed by the CPU 11 and program data supplied from the receiving unit 16 via the input / output interface 15.

  The communication unit 20 communicates with an external device via a network such as the Internet or a local area network. A program broadcast via the Internet may be received by the communication unit 20.

  When a removable medium 22 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is mounted, the drive 21 drives them to acquire programs and data recorded therein. The acquired program and data are transferred to the recording unit 19 and recorded as necessary.

  FIG. 5 is a block diagram illustrating a functional configuration example of the information processing apparatus 1. At least a part of the functional units shown in FIG. 5 is realized by the CPU 11 shown in FIG. 4 executing a predetermined program.

  As shown in FIG. 5, in the information processing apparatus 1, a caption / video acquisition unit 31, a video decoding unit 32, a subtitle position determination unit 33, a region designation unit 34, and a video feature extraction unit 35 are realized.

  The subtitle / video acquisition unit 31 acquires video and subtitle data of a program, and the acquired video data is included in the video decoding unit 32 and the subtitle position information is data representing the subtitle display position included in the subtitle data. Is output to the subtitle position determination unit 33. For example, the program video and caption data received by the receiving unit 16, or the recorded program video and caption data recorded in the recording unit 19 are acquired by the subtitle / video acquisition unit 31.

  The video decoding unit 32 decodes the video of each frame as shown in FIG. 2 from the data supplied from the caption / video acquisition unit 31, and the baseband video information obtained by decoding the video information is a video feature extraction unit. 35.

  The subtitle position determination unit 33 determines whether the display position of the subtitle is within a predetermined range from the normal display position based on the subtitle position information supplied from the subtitle / video acquisition unit 31, and the determination result Is output to the area specifying unit 34.

  For example, the caption position determination unit 33 acquires the position of the center of gravity of the caption area included in the frame of interest based on the caption position information, and calculates the difference between the acquired position and the average position of the caption area. For the caption position determination unit 33, a moving average of the vertical position of the center of gravity of the caption area based on a predetermined position of the frame is given in advance as the average position of the caption area, and This is used to calculate the difference from the position of the center of gravity of the included caption area. In the caption position determination unit 33, such determination is sequentially performed for each frame.

  The area designation unit 34 designates an important area based on the caption position determination information supplied from the caption position determination unit 33, and the important area information, which is information indicating the position and range of the designated important area, is the video feature extraction unit. 35.

  For example, the area specifying unit 34 determines that the difference between the average position of the subtitle areas exceeds the threshold difference and the subtitle position determining unit 33 determines that the subtitle area is included at a position separated by a predetermined distance. Of the entire frame, a certain area including the average position of the caption area is designated as an important area, and important area information representing the position, range, and the like is output to the video feature extraction unit 35.

  Based on the video information supplied from the video decoding unit 32, the video feature extraction unit 35 analyzes the pixel values of the pixels constituting the video, the movement of the subject shown in the video, and extracts the features. At this time, the video feature extraction unit 35 analyzes the video of the important region designated by the important region information supplied from the region designating unit 34 with priority compared with the video of other regions, and extracts the features.

  For example, telop detection processing may be performed, and the presence / absence of a telop, the content of the telop, and the like may be extracted as features. The telop is detected by detecting an edge and recognizing a character. When a plurality of telops are detected by the telop detection process for other regions as well, only the telops in the important region or the telops overlapping the important region are used as feature extraction targets. It is assumed that there is a telop in the important area, and the process may be advanced.

  The feature extracted by the video feature extraction unit 35 is output as a result of extraction to a subsequent configuration, and is used as a reference for selecting a chapter setting or digest playback scene at a predetermined timing.

  Next, processing of the information processing apparatus 1 that designates an important area will be described with reference to the flowchart of FIG.

  This processing is started when the video and subtitle data of the program is acquired by the subtitle / video acquisition unit 31, and the video data is output to the video decoding unit 32 and the subtitle position information is output to the subtitle position determination unit 33. . The video decoding unit 32 decodes the video and outputs video information to the video feature extraction unit 35.

  In step S <b> 1, the caption position determination unit 33 acquires the position of the center of gravity of the caption area included in the frame of interest based on the caption position information supplied from the caption / video acquisition unit 31.

  In step S2, the caption position determination unit 33 calculates the difference between the position acquired in step S1 and the moving average position of the caption area.

  In step S3, the caption position determination unit 33 determines whether or not the difference between the position of the center of gravity of the caption area included in the frame of interest and the moving average position of the caption area exceeds the difference as a threshold, If it is determined that the number does not exceed, the processes after step S1 are repeated. The determination result by the caption position determination unit 33 is supplied to the region designation unit 34.

  On the other hand, if it is determined in step S3 that the difference between the position of the center of gravity of the subtitle area included in the frame of interest and the moving average position of the subtitle area exceeds the threshold value, the area in step S4 The designation unit 34 determines that the difference between the moving average position of the caption area exceeds the threshold difference and the caption position determination section 33 determines that the caption area is included at a position separated by a predetermined distance. A certain area including the moving average position of the caption area in the entire frame is designated as an important area. The important area information indicating the position and range of the important area designated by the area designation unit 34 is supplied to the video feature extraction unit 35.

  In step S <b> 5, the video feature extraction unit 35 extracts features for only the video of the important region specified by the important region information supplied from the region specification unit 34 or for the video of the important region in order. Do. The feature extraction result extracted by the video feature extraction unit 35 is supplied to the subsequent stage, and the process ends.

  Through the above process, it is possible to designate an area that is considered to well represent the features of the video as an important area, and to efficiently extract the features by focusing on the video of the designated important area Is possible.

  In the above, when the position of the subtitle display area is moved, the area including the position of the normal subtitle display area is designated as the important area. When the position of the area where other information is displayed moves, such as when the position of the display area moves or when the position of the logo display area moves, the position of the normal display area of that information May be designated as an important area.

  In the above, the video of the important area is preferentially analyzed, and the features are extracted. However, the analysis itself is performed with the same content and the same level of processing as the other areas. A higher priority may be set for the extracted feature than the feature extracted from the video in the other region, and the extracted feature may be used preferentially for chapter setting or the like.

  The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a program recording medium in a general-purpose personal computer or the like.

  As shown in FIG. 4, a program recording medium that stores a program that is installed in a computer and can be executed by the computer is a magnetic disk (including a flexible disk), an optical disk (CD-ROM (Compact Disc-Read Only). Memory), DVD (Digital Versatile Disc), magneto-optical disk, or removable media 22 which is a package medium made of semiconductor memory, or ROM 12 where a program is temporarily or permanently stored, or recording unit 19 It is comprised by the hard disk etc. which comprise. The program is stored in the program recording medium using a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting via the communication unit 20 that is an interface such as a router or a modem as necessary. Done.

  In the present specification, the steps for describing a program are not only processes performed in time series in the order described, but also processes that are executed in parallel or individually even if they are not necessarily processed in time series. Is also included.

  The embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

It is a figure which shows the information processing apparatus which concerns on one Embodiment of this invention. It is a figure which shows the example of an important area | region. It is a figure which shows the concept of the detection of an important area | region. It is a block diagram which shows the hardware structural example of information processing apparatus. It is a block diagram which shows the function structural example of information processing apparatus. It is a flowchart explaining the process of information processing apparatus.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Information processing apparatus, 31 Subtitle | video acquisition part, 32 Image | video decoding part, 33 Subtitle position determination part, 34 Area | region designation | designated part, 35 Image | video feature extraction part

Claims (5)

Acquisition means for acquiring the position of the display area of the subtitles included in each frame;
Designation means for designating a predetermined area including the average position of the subtitle display area as an important area from among the frames including the subtitle display area at a predetermined distance from the average position of the subtitle display area; Information processing apparatus provided. The information processing apparatus according to claim 1, further comprising a feature extraction unit that performs a process of extracting video features of the important area designated by the designation unit more than a process of extracting video features of another area. . The information processing apparatus according to claim 2, wherein the feature extraction unit performs processing for extracting video features of the important area designated by the designation unit, which is large in time, processing amount, or number of times. Get the position of the subtitle display area included in each frame,
Information including a step of designating a predetermined area including the average position of the subtitle display area as an important area among frames including the subtitle display area at a predetermined distance from the average position of the subtitle display area Processing method. Get the position of the subtitle display area included in each frame,
A process including a step of designating a predetermined area including the average position of the subtitle display area as an important area among frames including the subtitle display area at a predetermined distance from the average position of the subtitle display area A program that causes a computer to execute.

Images