JP2008092594A - Moving image processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a moving image processing apparatus capable of dividing moving images into appropriate metashots irrespective of program types. <P>SOLUTION: The apparatus includes a similarity measuring means 104 for measuring the similarity between divided parts of moving images, a similar-shot specifying means 104 for specifying similar partial parts of moving images, a grouping means 110 for conferring a same group attribute to the similar divided parts of moving images, a metashot generating means 107 for generating a metashot having lead parts of moving images selected by a lead shot selecting means 106 at the head thereof, and a program type determining means 130. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a moving image processing apparatus.

  With the widespread use of high-performance personal computers (PCs) and hard disk recorders, techniques for digitizing and storing video and moving images have been developed. This technology is realized in the form of hardware and software, and is also realized not only for business use but also for home hardware.

  Specifically, for example, video is electromagnetically recorded in a hard disk (HDD) in a PC or a recorder. Therefore, there are advantages that the conventional video tape does not have, such as the ability to start playback of the target program with a small waiting time and the easy deletion of unnecessary programs. Due to such improved convenience, operations such as recording can be performed more easily.

  On the other hand, when a large amount of video is recorded, there is a problem that it becomes difficult to search for a desired scene. Such a problem can be dealt with by shortening the search time by so-called “skipping” the program using a fast-forward function or the like.

  However, such a “skipping” has a new problem of overshooting the scene of interest because the display frames are thinned out in physical units unrelated to the structure of the program content, for example, one frame every few seconds. occured.

  In order to solve such a problem, an image processing technique is used to divide a moving image into partial moving images by image change points (hereinafter referred to as “cut points”) at which the images in the moving image are switched, and shots, Technical research and product development have been carried out to enable skipping for each partial moving image.

  As the video division as described above, it is possible to divide according to the contents of the program, such as commercial and program main part, news program topic change, etc. A large amount of images that cannot be processed are included.

  Many of the generated shots have a short playback time of about several seconds. Thus, when the time length of one shot is extremely short, the effect that the search time can be shortened cannot be expected.

In order to solve this problem, the applicant of the present application has proposed a method for improving the visibility of the list display by omitting the icon display of similar shots (see Patent Document 1). In addition, a method has been proposed in which video is structured in units closer to the original program content by grouping video repeat units into a group of consecutive shots (metashot).
JP-A-9-270006 Aoki et al. “Video Interface Using Hierarchical Icons by Integration of Repeated Shots” (Information Processing Society of Japan Vol.39, No.5 pp.1317-1324, 1998)

  According to the method of the above document, since the method for dividing into appropriate meta shots differs depending on the type of program, it is desired to provide a technique for automatically dividing into appropriate meta shots regardless of the type of program. ing.

  The present invention has been made in view of the above, and an object thereof is to provide a moving image processing apparatus, a moving image processing method, and a moving image processing program that can be divided into appropriate shots regardless of the type of program. And

  In order to achieve the above object, a moving image processing apparatus according to the present invention is divided by a cut detection means for detecting an image change point at which an image content is switched from a moving image, and the image change point detected by the cut detection means. Similarity measuring means for measuring the similarity between partial moving images that are moving images, similar shot specifying means for specifying similar partial moving images based on the similarity measured by the similarity measuring means, Grouping means for assigning the same group attribute to a plurality of similar partial moving images specified by the similar shot specifying means, and similar partial moving images grouped by the grouping means appear in the moving image. And a moving image type determining means for determining the type of the moving image based on the appearance pattern.

  Here, in the present invention, it further comprises shot number comparing means for comparing the number of partial moving images belonging to the same group with a predetermined reference number, and the moving image type determining means is based on the shot number comparing means. The moving image type is determined based on the comparison result.

  In the present invention, the shortest time length comparing means for comparing the reproduction time length of the partial moving image having the shortest reproduction time length among the partial moving images belonging to the same group with a predetermined reference shortest time length is further provided. The moving image type determining means determines the moving image type based on a comparison result by the shortest time length comparing means.

  In the present invention, the longest time length comparison means for comparing the reproduction time length of the partial moving image having the longest reproduction time length among the partial moving images belonging to the same group and a predetermined reference longest reference time length. Further, the moving image type determining means determines the moving image type based on a comparison result by the longest time length comparing means.

  Further, in the present invention, a time length average value calculating means for calculating an average value of reproduction time lengths of partial moving images belonging to the same group, and an average value of the reproduction time lengths calculated by the time length average value calculating means, Average value determining means for determining whether or not the value is within a predetermined reference average time length range, and measuring the number of groups determined by the average value determining means to be within the reference average time length range The moving image type determining means determines the moving image type based on a measurement result by the reference average time group number measuring means.

In the present invention, among the partial moving images belonging to the same group, the partial moving image between the partial moving image first arranged in the moving image and the partial moving image arranged last in the moving image. An inter-shot time length measuring means for measuring an inter-playback time length, and the inter-moving image playback time length measured by the inter-shot time length measuring means is within a predetermined reference inter-moving video time length range. An inter-shot time length determining means for determining whether or not the value is a value, and a group for measuring the number of groups determined by the inter-shot time length determining means to be a value within the reference partial moving image time length range A moving image type determining unit, wherein the moving image type determining unit determines the moving image type based on a measurement result by the reference shot group number measuring unit.

  In the present invention, a group existence range is defined between a partial moving image first arranged in the moving image and a partial moving image arranged last in the moving image among the partial moving images belonging to the same group. Obtained by performing a sum operation on the group existence range specified by the group existence range specifying means and the group existence range specifying means specified for each of the plurality of groups included in the moving image. Meta shot specifying means for specifying the partial moving image included in the range as one meta shot, and the moving image type determining means is based on the appearance pattern of the meta shot specified by the meta shot specifying means Then, the type of the moving image is determined.

  In the present invention, based on the appearance patterns of a plurality of partial moving images belonging to the same group, the interaction frequency indicating the probability that the target partial moving image belonging to the group is a partial moving image indicating a conversation scene is calculated. The portion included in a range in which the interaction frequency is within a reference range when the interaction frequency calculated by the interaction frequency calculation unit is a value within a predetermined reference ratio range. A meta shot specifying unit that specifies a moving image as one meta shot, and the moving image type determining unit is configured to generate the moving image based on the appearance pattern of the meta shot specified by the meta shot specifying unit. The type is determined.

  In the present invention, the moving image type determination means determines the moving image type based on the number of metashots included in the moving image.

  In the present invention, the moving image type determination unit determines the moving image type based on a total value of reproduction time lengths of the meta shots included in the moving image.

  In the present invention, the moving image type determining means determines the moving image type based on the reproduction time length of the metashot having the shortest reproduction time length among the metashots included in the moving image. It is characterized by that.

  In the present invention, the moving image type determining means determines the moving image type based on the reproduction time length of the meta shot having the longest reproduction time length among the meta shots included in the moving image. It is characterized by that.

  In the present invention, the moving image type determining means determines the moving image type based on an average reproduction time length of the metashot included in the moving image.

  In the present invention, the program further includes a program interaction degree calculating unit that calculates the activity of the dialogue with respect to the entire moving image, and the moving image type determining unit determines the interaction degree of the entire moving image calculated by the program interaction degree calculating unit. The moving image type is determined on the basis of this.

In the present invention, it further comprises shot time length calculating means for calculating a total time length of partial moving images belonging to any group in the moving image, wherein the program interaction frequency calculating means includes the time of the moving image. The ratio of the total time length calculated by the shot time length calculation means to the length is calculated as the program interaction frequency.

  In the present invention, it further comprises a shot number measuring means for measuring the number of partial moving images belonging to any group in the moving image, and the program interaction frequency calculating means is configured to calculate the time length of the moving image with respect to the time length of the moving image. A ratio of the number of partial moving images measured by the shot number measuring means is calculated as the program interaction frequency.

  In the present invention, the shot time length calculating means for calculating the total time length of the partial moving images belonging to any group in the moving image, and the number of partial moving images belonging to any group in the moving image Shot number measuring means for measuring the program interaction frequency calculating means, the ratio of the total time length calculated by the shot time length calculating means to the time length of the moving image, and the time length of the moving image The product of the ratio of the number of partial moving images measured by the shot number measuring means with respect to is calculated as the program interaction frequency.

  Also, in the present invention, the moving image type determining means is configured such that a start time of a meta shot that appears in a specified order from the moving image head is a predetermined time from the moving image head or the length of the moving image. The moving image type is determined based on whether the ratio is before or after a predetermined ratio.

  In the present invention, it further includes a dividing unit that divides the moving image into a plurality of temporal sections, and the moving image type determining unit determines the moving image type for each section divided by the dividing unit. It is characterized by that.

  In the present invention, an analysis parameter receiving unit that receives an analysis processing condition corresponding to a moving image is further provided, and at least one of the cut detecting unit, the similar shot specifying unit, and the moving image type determining unit includes the analysis parameter It is characterized in that a cut is detected, a similar shot is identified, or a moving image type is determined based on a reference condition received by the receiving means.

  According to the present invention, it is possible to provide a moving image processing apparatus, a moving image processing method, and a moving image processing program that can be divided into appropriate shots regardless of the type of program.

  Embodiments of a moving image processing apparatus, a moving image processing method, and a moving image processing program according to the present invention will be described below in detail with reference to the drawings. Note that the present invention is not limited to the embodiments. In the embodiment, as an example, processing when a video of a news program is acquired as a moving image will be described. The “shot” described below corresponds to the partial moving image described in the claims.

  FIG. 1 is a block diagram of a functional configuration of the moving image processing apparatus 10 according to the first embodiment. The moving image processing apparatus 10 includes a moving image acquisition unit 101, a cut detection unit 102, a shot section definition unit 103, a similar shot detection unit 104, a grouping unit 110, a meta shot head group determination unit 105, A shot start time determination unit 106, a meta shot generation unit 107, a moving image output unit 108, a statistical processing unit 120, a statistical value holding unit 122, and an interaction frequency calculation unit 124 are provided.

  The moving image acquisition unit 101 acquires a moving image from the outside via, for example, a broadcast program receiver (tuner) connected to the moving image processing apparatus 10. The moving image acquisition unit 101 may acquire an uncompressed moving image. Alternatively, a moving image converted into digital data such as the MPEG format, MPEG-1, 2, 4 or the like which is a standard format for moving image compression may be acquired.

  The moving image acquisition unit 101 converts the acquired moving image into a format suitable for the cut detection unit 102 to process, and passes the converted moving image to the cut detection unit 102. Here, conversion to a suitable format is, for example, processing for expanding (decoding) a compressed (encoded) moving image. In addition, it is a process of converting the size into a necessary and sufficient image size in the process by the cut detection unit 102.

  The cut detection unit 102 calculates the similarity between the image frame input for each image and the image frame input immediately before, and detects an image change point at which the content of the image is switched, that is, a cut point. Further, when a moving image using predictive coding for image compression such as MPEG-2 is acquired, a cut point may be detected by using a variation in the predicted code amount.

  Note that the method by which the cut detection unit 102 detects the cut point is not limited to the embodiment, and may be realized by various known methods. Such a technique is also described in JP-A-9-93588 filed by the applicant of the present application.

  The shot section definition unit 103 defines a set of image frames belonging to a time section surrounded by two cut points arranged at the closest positions detected by the cut detection unit 102 as shots. For example, a cut point is detected immediately before 3 minutes 15 seconds 20 frames, which is a frame that is played back 3 minutes 15 seconds 20 after the start of playback, and the next cut point is detected immediately before 3 minutes 21 seconds 12 frames. In this case, one shot is defined from 3 minutes 15 seconds 20 frames to 3 minutes 21 seconds 11 frames. Here, the reproduction time is a time required from when the video is started until a predetermined frame is reproduced when the video is reproduced.

  The similar shot detection unit 104 detects similar shots with the shot defined by the shot section definition unit 103 as one unit. Specifically, one or more frames included in the shot are selected from each shot. Then, the similarity is measured by comparing these frames.

  For the similarity comparison of shots themselves, the method disclosed in Japanese Patent Laid-Open No. 9-270006 filed by the applicant of the present application can be used. According to this method, the feature amount is calculated in each of the two target frames. Then, the distance between these two feature amounts is calculated. For example, when a 36-division histogram is calculated from the hue values of pixels belonging to each frame and a vector having the frequency of each of the 36 divisions as an element is used as a feature quantity, two feature quantity points in a 36-dimensional space are used. Calculate the distance. This distance is a value corresponding to the similarity, and the smaller the distance, the higher the similarity.

  When the similarity measured in this way is greater than or equal to a predetermined value, these two shots are detected as similar to each other. In this way, similar shots are detected based on the similarity between shots.

  The similar shot detection unit 104 measures the degree of similarity of one shot included in one moving image with all other shots included in the moving image. The degree of similarity may be measured only for a predetermined number of shots that are temporally adjacent to the shot. Note that the similar shot detection unit 104 constitutes similarity measurement means and similar shot identification means of the present invention.

  The grouping unit 110 groups similar shots by assigning the same label to similar shots detected by the similar shot detection unit 104. Based on the information acquired from the grouping unit 110, the statistical processing unit 120 generates statistical information about the group such as the number of appearances of each group in one moving image, for example. The statistical value holding unit 122 holds the statistical information generated by the statistical processing unit 120.

  Based on the statistical information stored in the statistical value storage unit 122, the meta shot head group determination unit 105 selects a group of characteristic shots that should be the first shot of the meta shot from the group generated by the similar shot detection unit 104. select.

  In a news program, since it is often appropriate to use one news item as a meta shot, it is desirable that the beginning of the news item can be detected as the first shot. An anchor person (newscaster / announcer) often appears at the beginning of news items. Therefore, if a scene where an anchor person appears can be detected as the first group of meta shots, an appropriate meta shot can be generated.

  As a method by which the meta shot head group determination unit 105 selects an anchor person shot, that is, a method of determining a meta shot head group, for example, for a plurality of shots belonging to a similar shot group, the number of appearances in a news program, the entire program A method is adopted in which the determination is made based on one or more conditions among conditions such as the wide range of temporal distributions of appearances and the time lengths of shots belonging to the group. A method for determining the head group will be described in detail later.

The meta shot head group determining unit 105 according to the embodiment includes the shot number comparison unit, the shortest time length comparison unit, the longest time length comparison unit, the time length average value calculation unit, the average time length comparison unit, and the shot interval according to the present invention. A time length measuring means, an inter-shot time length comparing means, and a shot position determining means are configured.

  The meta shot head time determination unit 106 acquires the determination result of the head group from the meta shot head group determination unit 105. Based on the determination result acquired from the meta shot head group determination unit 105, the shot that should actually be the head of the meta shot is identified from the shots belonging to the group determined as the head group, and the start position of the identified head shot is determined. The corresponding playback time is defined as the metashot start time.

  Specifically, a shot whose playback time length is equal to or longer than a predetermined length is specified as the first shot. For example, an anchor person may appear in a news item in a dialogue with a guest. In this case, the time that the anchor person appears in the dialogue with the guest is often shorter than the time that the anchor person appears at the beginning of the news item. Therefore, by limiting the shot playback time length to a predetermined length or more, the anchor person shot included in the news item can be excluded from the head shot candidates.

  As another example, the meta shot head time determination unit 106 may specify a shot whose playback time length is equal to or shorter than a predetermined length as the head shot. As another example, the head is based on conditions such as a temporal interval with other shots belonging to the same similar shot group, a temporal context with other shots belonging to another similar shot group, a distribution, an inclusion relationship, etc. A shot may be specified.

  Further, the first shot may be specified based on only one condition selected from the plurality of conditions. As another example, all of the plurality of conditions or two or more selected from the plurality of conditions may be used. The first shot may be specified based on the above condition. The meta shot head time determination unit 106 constitutes head shot selection means of the present invention.

  The meta shot generation unit 107 generates a meta shot starting from the head shot specified by the meta shot head time determination unit 106. Specifically, the same label indicating the same metashot is assigned to each of a plurality of shots continuously arranged from the metashot start time to the next metashot start time.

  In the video of a news program, the time from the beginning of the news program to the time immediately before the first metashot start time appears may be determined as the program opening and labeled as the opening metashot. A shot from the last metashot start time to the end of the news program may be used as a metashot.

  As described above, when the moving image is divided by the meta shot, the division result is output from the moving image output unit 108. The data output in this way is sent to a display device, for example. The display device displays a list of video contents of the moving image based on the meta shot. Or, playback is displayed in meta shot units.

  The moving image output unit 108 outputs the moving image divided into meta shots by the meta shot generation unit 107. The moving image may be output to a display device, for example. In this case, a list of video (program) contents is displayed on the display device. Or, a moving image is viewed in meta shot units.

  In this way, by displaying a moving image corresponding to the section of the meta shot, for example, a screen list for each news item can be created. Further, by operating the “Skip” button on the remote controller, it is possible to view the first shot of the next news item even while viewing a certain news item.

  FIG. 2 schematically shows statistical information held by the statistical value holding unit 122. As shown in FIG. 2, information on each group is held in association with a group ID for identifying a similar shot group. In this embodiment, the information about each group includes the number of shots belonging to the group (hereinafter referred to as “number of times”), the playback time length (shortest) of the shots belonging to the group that has the shortest playback time, The longest playback time length of the shots belonging to the group (longest), the average playback time length of the shots belonging to the group (average length), and the end time of the last shot from the start time of the first shot belonging to the group Playback time length (distribution) and whether or not the shot group is included by other shot groups (included).

  Here, the concept of inclusion will be described. In news programs, anchor person shots that should be the first shot are scattered throughout the entire moving image. For example, among the shots of an anchor person, a relation in which a shot placed between each news item is not sandwiched between two shots belonging to a predetermined group other than the anchor person is defined as an inclusion relation. .

  On the other hand, shots corresponding to the video of each news item are unevenly distributed only within a predetermined playback time range in the moving image. And before and after the news item, anchor person shots are arranged. In this way, a relationship between two shots belonging to a predetermined group is defined as an inclusion relationship.

  From the above, in a news program, it is possible to determine whether or not it is a group of anchor person shots, that is, a leading group, based on whether or not it is an included group.

  Hereinafter, the concept of inclusion will be described more specifically with reference to FIG. FIG. 3 is a diagram schematically showing a moving image. Each rectangle represents one shot. For example, a rectangle attached with A indicates a shot belonging to the A group. That is, shots assigned the same symbol belong to the same group. The horizontal axis is the time axis. That is, the shots 701 and 706 are reproduced in this order. Further, it is assumed that the B shot and the C shot do not exist in positions other than those shown in FIG. 3 in the moving image.

  In FIG. 3, C shots 708 and 709 belonging to group C are both arranged between A shot 704 and A shot 705. The A shot 704 and the A shot 705 are two shots that appear continuously in the time axis direction. In this case, group C is an inclusion group.

  In this way, when all shots belonging to the same group exist between two shots belonging to the same group and appearing continuously in the time axis direction, the group of shots existing between the two shots Becomes a group of inclusions.

  On the other hand, B shot 706 and B shot 707 are both sandwiched between A shots, but A shot 702 is sandwiched between B shot 706 and B shot 707, and B shot 706 and B shot 707 are It does not exist between two consecutive A shots. Therefore, group B is an included group.

  On the other hand, in the A group, for example, the A shot 703 is arranged following the B shot 712, but the B shot is not arranged after the A shot 703. Therefore, it is determined that the group is not included.

  Here, a process in which the metashot head group determination unit 105 selects a head group based on the statistical information stored in the statistical value holding unit 122 will be described with reference to FIGS. 4 and 5.

  FIG. 4 schematically shows a news program. The horizontal axis is the time axis. News programs are played from the top to the bottom. FIG. 5 shows the contents of shots belonging to each group. Group A is an anchor person shot. Group B is a shot of a correspondent stationed in Washington, DC. Group C is a shot where the prime minister who answers is appearing. Group D is a video shot of the prefectural office building. Group E is a shot of an anchor person captured by another camera composition.

  A meta shot starting from the start of a news program is a news summary shot that introduces an outline of the news program. The news summary shot includes a C-shot 710 of the headline with the content of “discussion of the confusing budget committee”, which is the second news item described later. The C shot 710 is an image of the profile of the prime minister who answers.

  Following this, a shot between the shot 702 and the shot 707 is a meta shot corresponding to the topic of the US Congress, which is the first news item.

  A shot 712 is an image in which the anchor person greets and announces the lead portion of the first news item. Then, a scene in which B shots and A shots are alternately arranged continues (713 to 716). This is a scene where a correspondent in Washington, DC and an anchor person have a dialogue. After this dialogue, the video of the American Diet enters two shots (717, 718), and this news item ends.

  Subsequently, a portion between the A shot 720 and the shot 722 is a meta shot corresponding to the second news item. In the meta shot corresponding to the second news item, the A shot 720 is followed by a shot of the Diet Building and a shot of the budget committee discussion room.

  Following this, a C shot 721 is also arranged. The C shot 721 is the same shot as the C shot 710 included in the news summary meta shot.

  A subsequent shot from A shot 731 to shot 732 is a meta shot corresponding to the third news item. The third news item reports a lack of revenue for a local government. It consists of a news scene including D shots 731 and 732 of the prefectural office.

  A subsequent shot from A shot 734 to shot 735 is a meta shot corresponding to the fourth news item. The news items starting from E shots 740, 742, and 744 are exchange rate, stock price, weather forecast, and ending, respectively.

  In the news program as described above, the meta shot head group determination unit 105 specifies the head group based on, for example, the number of shots belonging to the group, that is, the number of times. Specifically, a group that matches the condition “appearance count is 3 or more” is identified as the first group. In this way, a group that appears more than a predetermined number of times is identified as the first group. Thereby, the A group is specified in the news program described with reference to FIGS. 4 and 5. Thus, a desirable group can be specified.

Alternatively, since an extremely large number of anchor persons do not appear, when the number of appearances is equal to or less than a predetermined number, the group may be specified as the first group.

  Further, the head group may be specified on condition that “the shortest length of shots belonging to the same group is 10 seconds or more”, that is, the shortest length of shots belonging to the same group is a predetermined value or more. Furthermore, the head group may be specified on condition that “the longest length of shots belonging to the same group is 21 seconds or longer”, that is, the longest length of shots belonging to the same group is equal to or greater than a predetermined value. .

  Further, even if the first group is specified on condition that “the average value of the lengths of the shots belonging to the same group is 12 seconds or more”, that is, the average value of the lengths of the shots belonging to the same group is a predetermined value or more. Good. Furthermore, the head group may be specified on the condition that the average length of shots belonging to the same group is equal to or greater than a predetermined value.

  In the news program described with reference to FIGS. 4 and 5, Group A is identified from the statistical information described with reference to FIG. 2 under the condition that “the longest length of shots belonging to the same group is 21 seconds or longer”. Is done.

  Further, it may be based on the length from the start time of the first shot belonging to the group to the end time of the last shot, that is, the distribution. Specifically, a group that satisfies the condition “distribution is 3 minutes or more” is specified as the first group. With this condition, the A group and the C group are specified.

  In this case, in order to exclude a group including a special appearance such as the C group appearing at the beginning, the condition of “distribution” is changed to “distribution of shots appearing second and later in the same shot group”. By specifying “]”, the head group can be specified accurately.

  Another example is “shots that are more than the time at which shots belonging to the same shot group are reproduced, that is, the variance of the arrangement, and the average reproduction time of the arrangement is multiplied by a certain coefficient added to that variance. The condition may be “distribution” excluding “.”

  Alternatively, it may be a condition that the group is “included”. Thereby, A group and C group can be specified.

  The conditions for the metashot head group determination unit 105 to select the head group have been described above. However, the head group may be specified based on one or more conditions selected from the above conditions.

  Also, prior to or when a moving image is input to the moving image acquisition unit 101, the analysis parameter receiving unit 190 performs the cut detection unit 102, the similar shot detection unit 104, and the meta shot head group determination unit 105 described above. The conditions (parameters) necessary for each process of the meta shot head time determination unit 106 may be received and supplied to the detection unit and the determination unit.

  For example, an electronic program guide service called EPG or iEPG provides program contents, broadcast channels, start / end times, and the like on the Internet. Similarly, when there is a service that provides analysis parameters on the Internet as part of EPG or iEPG information, the moving image processing apparatus of the present invention sets parameters for detection and determination according to the recorded program. Can be changed.

Specifically, when a specific variety program is input such that a similar title screen is always inserted at each turning point of the topic, the moving image processing apparatus of the present invention sets the program-specific parameter settings before recording, Or download from the Internet during recording. The moving image processing apparatus of the present invention performs meta shots for each topic with higher accuracy by using the downloaded conditions such as “similar shots appearing three times or more and the minimum interval of appearance is two minutes or more”. Can be created.

  The means for downloading analysis parameters is not limited to the Internet. For example, the optimal analysis parameter setting for each program may be supplied in the form of a CD-ROM, a memory card, or the like when new programs are available, such as mid-April and mid-October. When a program is input to the apparatus, the analysis parameter receiving unit 190 may read an optimum parameter setting corresponding to the program from a recording medium and supply it to each detection and determination unit. Also, the optimum parameters recorded on the recording medium are temporarily copied to a recording area (not shown) in the apparatus, and the analysis parameter receiving unit 190 receives the optimum parameters from the recording area when a program is input to the apparatus. May be read and supplied to each detection and determination unit.

  FIG. 6 is a flowchart showing moving image processing in the moving image processing apparatus 10. The moving image processing mainly includes three processes of a shot section definition process, a grouping process, and a metashot generation process.

  First, shot section definition processing is performed. That is, the cut detection unit 102 inputs to acquire image frames one by one (step S202), and the cut detection unit 102 acquires the image frame acquired immediately before the image frame acquired in step S202 and the step S202. The similarity with the image frame is calculated, and a cut point is detected based on the similarity.

  When the acquired image frame is a cut point (step S203, Yes), the shot section definition unit 103 defines a section from the cut point to the immediately preceding cut point as a shot section (step S204).

  The processing from step S202 to step S204 is repeated as described above. When the definition of the shot section for the entire video (program) is completed (step S201, Yes), the shot section definition process is completed and the process proceeds to the grouping process.

  The similar shot detection unit 104 selects a predetermined shot as a reference shot, and determines the degree of similarity between the shot and the target shot to be compared (step S207). If it is determined that the target shot is similar to the reference shot (Yes in step S208), the grouping unit 110 assigns a label for identifying the same group to the target shot and the reference shot. . That is, the target shot and the reference shot are grouped (step S209).

  The processes in steps S207 and S208 are repeated for all target shots for one reference shot. When the processing is completed for all target shots (step S206, Yes), the reference shot is changed, and the processing of step S207 and step S208 is repeated again.

  When the similarity determination process between the reference shot and the target shot is completed for the entire video (step S205, Yes), the grouping process is completed and the process proceeds to the next metashot generation process.

The meta shot head group determination unit 105 identifies the head group based on the statistical information held in the statistical value holding unit 122. Then, the metashot start time determination unit 106 defines the metashot start time based on the start group specified by the metashot start group determination unit 105. When the group to be processed matches the condition of the top group (step S211), the metashot generation unit 107 generates a metashot having the group as the top shot (step S212).

  Steps S211 and 212 are repeated. When the generation of the meta shot is completed for the entire video (step S210, Yes), the meta shot generation process is completed and the moving image process is completed.

Note that there is an analysis parameter reception step (not shown) as described above, and using the optimal parameter setting for each program received from the Internet or the like by the analysis parameter reception step before or during this processing, step S203, S207 and S211 may perform detection and determination processing.

  As described above, since the moving image processing apparatus 10 according to the first embodiment specifies the first shot based on the appearance pattern of shots belonging to the same group, it is possible to avoid generating a metashot that is finer than necessary. it can. As a result, the user can easily search for a predetermined scene.

  The moving image processing in the moving image processing apparatus 10 includes three processes (a part surrounded by a broken line in FIG. 2): (1) shot section definition processing, (2) grouping processing, and (3) metashot generation processing. Has been. In the embodiment, after (1) shot section definition processing is completed for all shots included in the moving image, the processing proceeds to (2) grouping processing. Similarly, after (2) grouping processing is completed for all shots included in the moving image, the process proceeds to (3) metashot generation processing. Instead of this, as another example, by providing a temporary storage area (not shown) in the moving image processing apparatus, the above three processes may be executed in parallel while inputting the video.

  For example, each time a new cut is detected and a shot section is defined, similar shots are determined for the shot section and the past shot section, and a current meta shot is generated based on the previous similar shot determination results. May be. As described above, by executing the processing in parallel, the processing result can be obtained in a very short time after the news program ends.

  FIG. 7 is a diagram illustrating a hardware configuration of the moving image processing apparatus 10. The moving image processing apparatus 10 controls, as a hardware configuration, each part of the moving image processing apparatus 10 according to a program in the ROM 52 and the ROM 52 in which a program for executing the moving image processing in the moving image processing apparatus 10 is stored. CPU 51 for executing image processing, a RAM 53 in which a work area is formed and various data necessary for controlling the moving image processing apparatus 10 is stored, a communication I / F 57 for communication by connecting to a network, and each unit Are provided.

  The moving image processing program for executing the moving image processing in the moving image processing apparatus 10 described above is an installable or executable file, such as a CD-ROM, floppy (R) disk (FD), DVD, or the like. The program is provided by being recorded on a computer-readable recording medium.

  Further, the moving image processing program of the present embodiment may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network.

  In this case, the moving image processing program is loaded onto the main storage device by being read from the recording medium and executed by the moving image processing device 10, and each unit described in the software configuration is generated on the main storage device. It has become so.

  Next, the moving image processing apparatus 10 according to the second embodiment will be described. The moving image processing apparatus 10 according to the second embodiment uses, for example, a shot that does not become the first shot of the meta shot, such as the A shots 714 and 716 in the news item 1 in the news program shown in FIG. To identify. Here, the interaction frequency is a value indicating the appearance frequency of the shot within a predetermined time range including the shot in question whether it should be the first shot. The dialogue frequency will be described in detail later.

  FIG. 8 is a block diagram of a functional configuration of the moving image processing apparatus 10 according to the second embodiment. In addition to the functional configuration of the moving image processing apparatus 10 according to the first embodiment, the moving image processing apparatus 10 according to the second embodiment further includes an interaction frequency calculation unit 124. The interaction frequency calculation unit 124 calculates the interaction frequency based on the information acquired from the shot section definition unit 103, the similar shot detection unit 104, and the grouping unit 110. Further, the meta shot head time determination unit 106 specifies a shot to be the head of the meta shot based on the conversation frequency calculated by the interaction frequency calculation unit 124.

  The interaction frequency calculation unit 124 according to the embodiment includes an interaction frequency calculation unit, a reference range specifying unit, a shot time length calculation unit, a first shot number calculation unit, and a second shot number calculation unit according to the present invention. To do.

  FIG. 9 shows shots 711 to 720 shown in FIG. 9, that is, an interaction scene between the anchor person and the correspondent. With reference to FIG. 9, a description will be given of a process in which the interaction degree calculation unit 124 calculates the interaction degree used when examining whether or not the A shot 714 is the first meta shot.

  First, a range from a central time of a target A shot 714 to a time separated by a specified time (for example, 1 minute, 30 seconds, etc.) is used as a reference range, and shots existing in the reference range are extracted. In the moving image shown in FIG. 9, a shot 717 is extracted from the shot 711. In the present embodiment, shots in which a part of the shot is included in the reference range, such as the shot 711 and the shot 717, are also extracted. Further, shots belonging to any group are extracted from between shot 711 and shot 717.

  Then, the ratio of the total time of the shots belonging to the group to the total time from the shot 711 to the shot 717 is calculated. The ratio of the total time of shots belonging to this group is the interaction frequency.

  In the moving image shown in FIG. 9, shots belonging to any group are shots 712 to 716. Therefore, the result of dividing the total time from the shot 712 to the shot 716 by the total time from the shot 711 to the shot 717 is the interaction frequency for the shot 714. In the moving image shown in FIG. 9, the interaction frequency is 0.88. It becomes.

A high interaction frequency is calculated for the shot A arranged in the news item, like the conversation scene between the anchor person and the correspondent shown in FIG. Therefore, shots other than the first shot can be excluded by not selecting a shot with the interaction frequency equal to or higher than a certain value as the first shot. In other words, the meta shot start time determination unit 106 acquires the interaction frequency from the interaction frequency calculation unit 124, and when the acquired interaction frequency is smaller than a certain value, identifies the shot as the first shot.

  Note that the processing for calculating the interaction frequency is not limited to the above embodiment. In this embodiment, the ratio of shots belonging to the group is calculated using the range from the central time of the target shot to a position separated by a specified time before and after the reference time, but the reference of the specified time is the center of the shot. It may not be time. For example, the position before the time (the beginning of the program) may be a position away from the start time of the target shot by a specified time. Further, the position later in time (the end of the program) may be a position away from the end time of the target shot by a specified time.

  In the embodiment, when a part of the shot is included in the reference range, the shot is extracted. However, instead of this, the entire reference shot includes the entire shot. Only shots may be extracted. As another example, in the case where a part of a shot is included in the reference range, only the playback time length of the part included in the reference range may be the target for calculating the interaction frequency.

  In the embodiment, the “ratio of shots belonging to any group” is calculated, but instead, “the ratio of shots belonging to groups in which all shots exist within the reference range” is calculated. May be. That is, even if the shot belongs to a group, if the shot belonging to the group is arranged outside the target reference range, the shot belonging to the group is not a target for calculating the ratio.

  In the embodiment, the ratio of the reproduction time length is calculated, but instead of this, the ratio of the number of shots may be calculated. Specifically, the number of all shots included in the reference range is measured. In addition, the number of shots included in the reference range and belonging to any group is measured. Then, the ratio of the number of shots belonging to the group to the number of all shots is calculated. This ratio is the interaction frequency. In the example of FIG. 9, the number of shots belonging to any group is 5, and the number of shots included in the reference range is 7. Therefore, the interaction frequency in this case is 5/7.

  As another interactive frequency calculation method, a method of dividing the number of shots included in the reference range and belonging to any group by the length of the reference time is also conceivable. This value represents the number of shots involved in dialogue per unit time. In the example of FIG. 9, 5 (pieces) / 50 (seconds) = 0.1 (pieces / second).

  As another method for calculating the interaction frequency, the product of the ratio of the reproduction time length and the number of shots involved in the interaction per unit time can be used. In the case of FIG. 9, 0.88 × 0.1 = 0.088. In this case, the degree of dialogue increases as the number of shots participating in the dialogue within the reference range increases, and increases as the time taken by the shots participating in the dialogue within the reference range increases. In other words, it can be expected that the larger the number of similar shots repeatedly appearing at a faster speed within the reference range, the larger the numerical value, and an index indicating the activeness of the dialogue.

  In the embodiment, the reference range centered on the shot is specified for a certain shot. Alternatively, the reference range centered on the time may be specified for a certain time. Good.

FIG. 10 shows the interaction frequency calculated by the calculation method described with reference to FIG. The horizontal axis of the graph shown in FIG. 10 indicates time. The vertical axis indicates the interaction frequency. The graph shown in FIG. 10 is calculated for an actual news program. 1001 corresponds to a part where a performer briefly discusses a certain news item in the studio. 1002 corresponds to the part where the guest appears in the studio and the newscaster discusses while interviewing. 1003 corresponds to a sports corner. In 1003, sports news casters repeatedly appear. Furthermore, pitcher shots, batter shots, etc. appear repeatedly as similar shots in the part reporting baseball. This is because even if different batters and different pitchers are used, the camera angles are the same, and therefore, similar shots are determined.

  Since a news caster as an interviewer repeatedly appears in a short time in a scene corresponding to 1002, there is a concern that all these shots are identified as the first shot. Therefore, the meta shot start time determination unit 106 specifies, for example, only the first shot of the shots of the meta shot start group included in the interval as the start shot in the interval in which the interaction frequency is equal to or greater than a specified value. Also good. Thereby, it is possible to suppress the occurrence of a large amount of meta shot head shots from the conversation scene. In the graph of FIG. 10, for example, the interaction frequency 0.2 may be set as the specified value.

  When the interaction frequency is calculated not in shot units but in time units, the time when the interaction level reaches or leaves the specified range may not match the shot boundary. In this case, the metashot boundary and the shot boundary do not have to coincide with each other.

  For example, when the first shot is detected, the time closest to the first shot may be determined as the start time among the times when the interaction frequency becomes equal to or higher than a specified value. Alternatively, the time closest to the first shot among the times when the interaction frequency becomes a specified value or less may be determined as the start time.

  With reference to FIG. 11, the method for determining the metashot boundary time will be described more specifically. FIG. 11 schematically shows the top shot and the interaction frequency for the top shot.

  Reference numeral 1301 denotes a metashot head shot obtained for each shot. Reference numeral 1302 denotes the interaction frequency. Reference numeral 1303 denotes a specified value for determining that a section in which the conversation frequency is equal to or greater than this number is a conversation section.

  In the first shot shown in FIG. 11, the metashot first time when the metashot is defined for each shot is 1304. On the other hand, when the meta shot is defined in units of time, the meta shot head time is 1305. As described above, when the interaction frequency is calculated in units of time, a position different from the shot boundary of the meta shot may be set as the meta shot start time.

  An example of a method of defining a metashot by another method will be described. FIG. 25 shows the duration of each shot and the group names grouped by similar shots for the 61st shot to the 74th shot in the moving image. In the “interaction frequency” portion, as described above, the product of the ratio of the reproduction time length to the reference range and the number of interactive shots per unit time is shown. Here, the “reference range” refers to the section from shot 61 to shot 61 in the column of shot number 61, the section from shot 61 to 62 in the column of shot number 62, and the column of shot number 74. The interaction frequency when the section from shots 61 to 74 is set as the reference range is shown. Here, for convenience, the unit of the numerical value is the reciprocal of minutes, and the numerical value divided by 60 is the reciprocal unit of seconds.

For example, when only the section from the shots 61 to 64 is seen, only the shot 62 and the shot 64 belonging to the group B repeatedly appear in this section, and their total time is 9 seconds. Since the total time from shots 61 to 64 is 27 seconds, the interaction frequency is (9 (
Seconds) ÷ 27 (seconds) × (2 (pieces) ÷ 27 (seconds) × 60 (seconds / minute)) = 1.5.

  In this way, when the reference range is extended, when the shot 61 is set as the start shot, the interaction frequency becomes maximum in the section up to the shot 69. If the minimum value of the interaction frequency that can be regarded as the interaction interval is determined in advance as 2, for example, the interaction frequency 8.1 from the shots 61 to 69 exceeds this value. Therefore, the meta shot head time determination unit 106 sets shot 61 to shot 69 as the conversation section.

  The dialogue section set in this way may be used as a meta shot as it is, and the meta shot start time determination unit 106 may set the start time of the start shot of the meta shot as the meta shot start time. In addition, a shot at the head or end of the conversation section may be used as the head of the meta shot. Furthermore, a shot belonging to any similar shot group and appearing first or last in the dialogue section may be set as the head of the meta shot. Further, the shot that belongs to the group that is determined as the metashot head group by the metashot head group determination unit 105 and that appears first or last in the conversation section may be the head of the metashot.

  FIG. 12 is a flowchart of the moving image processing performed by the moving image processing apparatus 10 according to the second embodiment. In the moving image processing apparatus 10 according to the second embodiment, when the grouping process is completed, the interaction degree calculation unit 124 calculates the interaction degree of the shot to be processed among the shots included in the first group ( Step S220). Then, the process proceeds to step S210. Further, in the process of specifying the top shot (steps S211 and S212), the meta shot head time determination unit 106 specifies the top shot based on the conversation degree calculated by the conversation degree calculation unit 124, and specifies the identified first shot. Is defined as the metashot start time.

  The remaining configuration and processing of the moving image processing apparatus 10 according to the second embodiment are the same as the configuration and processing of the moving image processing apparatus 10 according to the first embodiment.

  Note that if the top shot is specified on condition that similar shots repeatedly appear, a screen 756 in which news items as shown in FIG. 5 are listed may be specified as the top shot. A screen like the screen 756 is often a news item switching in a news program, like an anchor person, so there is no problem even if such a screen is used as the top shot. For this reason, the shot specified as the head shot is not limited to the shot of the anchor person.

  Moreover, a variety program etc. can also be divided | segmented for every corner using the already-mentioned dialog section definition. An example of this method will be described below. FIG. 26 schematically shows a similar shot appearance pattern of a program introducing trivia knowledge. In this program, the talk of the performers in the studio and the trivia introduction video appear alternately, and the name of the person who posted the trivia knowledge on the program is introduced before the trivia introduction video.

  In the studio talk scene, the video of the performer appears alternately. However, there are few similar shots appearing in the trivia introduction video part, or there is a tendency for similar shots to be completed only by the trivia introduction video part. Therefore, when the dialogue section is defined by the above-described method, the studio talk portion that reacts to each trivia knowledge and the partial section of the trivia introduction video become the dialogue section.

For this reason, by selecting a similar shot group that has never been included in the dialogue section (or included only a very small number of times), a characteristic shot is selected for each corner as shown by “M” in FIG. be able to. The similar shot M is a shot that introduces the name of the person who posted the trivia knowledge, and is a shot that follows the similar shot A that the moderator introduces, “That's the next trivia”.

  Since the similar shot M is not included in the conversation section 2501, if the conversation section 2501 is extended by one shot to the subsequent similar shot M, the number of conversations decreases. For this reason, the dialogue section 2501 is defined as shown in FIG. As a result, the similar shot M does not belong to any dialogue section.

  As described above, a shot that belongs to a similar shot group that does not belong to the conversation section is shot at the top of the meta shot, so that even in the variety program, it can be divided for each corner. In addition, when there are a plurality of similar shot groups that do not belong to the conversation section, it may be further narrowed down using the time distribution, average length, or the like.

  The corner division of a variety program by such a method can be used for a certain kind of quiz program. For example, it is a program in which studio respondents discuss after watching the problem video and play the correct video, with a fixed pattern such as “question” before the problem video and “correct answer” before the correct video. In such a case, it is highly possible that shots such as “question” and “correct answer” do not belong to the dialogue section.

  As described above, in the moving image processing apparatus 10 according to the second embodiment, the first shot is specified based on the interaction frequency, so that a more appropriate meta shot can be generated.

  Next, the moving image processing apparatus 10 according to the third embodiment will be described. The moving image processing apparatus 10 according to the third embodiment determines a program type of the acquired moving image. Here, the program type refers to a type such as a news program, a drama, or a sports program.

  FIG. 13 is a block diagram illustrating a functional configuration of the moving image processing apparatus 10 according to the third embodiment. The moving image processing apparatus 10 according to the third embodiment includes a program type determination unit 130 instead of the metashot head group determination unit 105 and the metashot head time determination unit 106 in the moving image processing apparatus 10 according to the first embodiment. Yes. When the similar shots are grouped, the program type determination unit 130 determines the type of the input program based on the temporal distribution of the similar shots. Then, the program type information indicating the program type determined by the program type determination unit 130 is output from the moving image output unit 108 to the external device.

  The external device that has acquired the program type can perform processing according to the program type based on the program type information. When the external device is a recording device such as a hard disk recorder, it may be used to change the recording bit rate or display the program type determined when displaying the recorded program list. . Moreover, you may use for the use of the automatic setting of the determination parameter for cut detection or similar shot detection.

  The program type determination unit 130 according to the embodiment includes the shot number comparison unit, the shortest time length comparison unit, the longest time length comparison unit, the time length average value calculation unit, the average value determination unit, the reference average time group according to the present invention. A number measuring unit, an inter-shot time length measuring unit, an inter-shot time length determining unit, a group measuring unit, a group existence range specifying unit, and a meta-shot specifying unit.

FIG. 14 schematically illustrates statistical information held by the statistical value holding unit 122 according to the third embodiment. Note that the statistical information shown in FIG. 14 is the statistical processing unit 12 for a fictional drama program described later.
0 is the generated statistical information.

  FIG. 15 schematically shows a drama program. Similar to the news program schematically shown in FIG. 4, the horizontal axis is the time axis. Drama programs are played in order from the top to the bottom. The algorithm by which the program type determination unit 130 determines the program type will be described in detail with reference to FIG.

  The program type determination unit 130 determines whether or not the program is a news program, for example, based on whether or not a “group that becomes the first shot of a meta shot” exists. That is, when there is one or more groups that become the first shot, it is determined that the program is a news program. If there is no group that becomes the first shot, it is determined that the program is a program other than the news program.

  Note that the process for determining whether or not “the group to be the first shot of the metashot” exists is the same as the process for selecting the metashot first group by the metashot first group determination unit 105 described in the first embodiment. .

  More specifically, a process of selecting an anchor person shot in a news program is performed. That is, the start of the metashot based on the conditions such as the number of appearances of shots belonging to similar shot groups, the shortest shot time length, the longest shot time length, the average shot time length, and the shot distribution time within a certain range Judged as a shot.

  Here, the process of defining shots 601 to 602 in FIG. 15 as one metashot will be described with reference to FIG. This procedure is the same as the previous document (Aoki et al., “Video Interface Using Hierarchical Icons by Integration of Repeated Shots” (IPSJ Journal Vol.39, No.5 pp.1317-1324, 1998)) It is also described in.

  In FIG. 16, shots belonging to the A group of similar shots appear in the time range 1101. Similarly, group B and group C appear in time range 1102 and time range 1103. When these time ranges are regarded as a set on the time axis and the union is obtained, a time range 1104 is obtained. With the above processing, the time range 1104 (that is, from 601 to 602 in FIG. 16) can be defined as a meta shot.

  After defining the metashot by the above processing, the program type is determined based on the appearance pattern of the metashot.

  Specifically, a group in which the playback time length of the shortest shot in which the playback time length of the shots belonging to the same group is the shortest is 10 seconds or more, that is, the playback time length of the shortest shot is a predetermined value or more. When the number of selected groups becomes 50% or more of the total number of groups included in the entire program, the program is determined as a drama.

  That is, when a group whose playback time length of the shortest shot is equal to or greater than a predetermined value is selected and the ratio of the number of selected groups to the total number of groups included in the entire program is equal to or greater than a predetermined value, the program is selected. Judged as a drama.

  As another example of the above determination conditions, the shortest shot time length is not less than a predetermined value, and the shortest shot time length is not less than a predetermined ratio in the entire program time length. Also good.

In addition, the ratio of the total number of shots belonging to the selected group to the total time length of the program is changed instead of the ratio of the number of selected groups to the total number of groups included in the entire program being a predetermined value or more. The condition may be that it is equal to or greater than a predetermined value. In addition, the ratio of the number of selected groups to the total number of groups included in the entire program is equal to or greater than a predetermined value. It is good also as a condition that it is more than a value. The metashot in this case is the metashot described with reference to FIG.

  Also, the number of meta shots that appear in the entire program, the total playback time of meta shots that appear in the entire program, the playback time length of the meta shot that has the shortest playback time length, and the meta shot that has the longest playback time length The program type is determined based on the reproduction time length, the average reproduction time length of meta shots included in the moving image, and the like. In this way, the program type can be determined based on the appearance pattern of the metashot.

  Alternatively, the average value of the playback time lengths of shots belonging to the same group is calculated, the number of groups in which the average value falls within a predetermined reference average time length range is counted, and the number of counted groups is counted. The program type may be determined based on this. In this way, the program type may be determined based on the appearance pattern of the shots included in the meta shot in the entire program.

  As another example, an inter-shot playback time length is measured between a shot that is first arranged and a shot that is arranged last in a program among shots that belong to the same group. May be counted to determine the program type based on the counted number of groups. In this case, for example, it can be determined whether the program is a news program.

  The program type may be determined based on one of these conditions, or the program type may be determined based on a combination of a plurality of conditions selected from these conditions.

  Also, prior to or when a moving image is input to the moving image acquisition unit 101, the analysis parameter receiving unit 190 includes each of the cut detection unit 102, the similar shot detection unit 104, and the program type determination unit 130 described above. Conditions (parameters) necessary for processing may be received and supplied to these detection units and determination units. As the destination to which the analysis parameter 190 receives parameters, the Internet, a recording medium, and the like are assumed as described in the first embodiment.

  FIG. 17 is a flowchart of the moving image processing in the moving image processing apparatus 10 according to the third embodiment. In the moving image processing of the moving image processing apparatus 10 according to the third embodiment, (1) shot section definition processing and (2) grouping processing are followed by program type determination processing (step S230). In the program type determination process, the program type determination unit 130 determines the program type of the moving image by the above-described process.

  Note that there is an analysis parameter reception step (not shown) as described above, and using the optimal parameter setting for each program received from the Internet or the like by the analysis parameter reception step before or during this processing, step S203, S207 and S230 may perform detection and determination processing.

  The remaining configuration and processing of the moving image processing apparatus 10 according to the third embodiment are the same as the configuration and processing of the moving image processing apparatus 10 according to the first embodiment.

As described above, since the moving image processing apparatus 10 according to the third embodiment can determine the program type based on the appearance pattern of similar shots, it is possible to improve the efficiency of viewing, searching, editing, and the like of video. . In addition, when processing a moving image of the program, processing according to the determined program type can be performed.

  Next, the moving image processing apparatus 10 according to the fourth embodiment will be described. The moving image processing apparatus 10 according to the fourth embodiment defines a metashot using the interaction frequency. FIG. 18 is a block diagram of a functional configuration of the moving image processing apparatus 10 according to the fourth embodiment. In addition to the functional configuration of the moving image processing apparatus 10 according to the third embodiment, the moving image processing apparatus 10 according to the fourth embodiment further includes an interaction frequency calculation unit 124. The program type determination unit 130 determines the program type based on the interaction frequency calculated by the interaction frequency calculation unit 124.

  FIG. 19 is a graph showing the number of dialogues calculated for an actual quiz program, as in FIG. Note that some modifications have been made for the sake of explanation. The black belts 1201 to 1208 on the horizontal axis are scenes where the presenter and the respondent are interacting in the studio.

  In this program, the opening and first question video flows before the black belt 1201. Subsequently, the answer scene of the performer in the studio, the scene where the correct video flows, the scene where the comment is made in the studio, and the scene where the question video of the next question flows are arranged in this order.

  From the graph of FIG. 19, it was found that the degree of dialogue was high in the studio scene and low in the opening video, the question video, and the correct video. This is because similar shots tend not to appear in the sections of the opening, the question video, and the correct video. Therefore, the threshold value 1210 can be set, and a section in which the interaction frequency is equal to or greater than the threshold value can be determined as a studio scene metashot.

  Further, as described above, it may be estimated as “drama / movie” on condition that there is a predetermined number of sections in which the number of conversations is equal to or greater than a threshold value.

  As another example, the total length of meta-shots in which the interaction frequency is equal to or greater than a threshold is calculated, and the calculated value is a “drama / movie” when the calculated value is within a predetermined range. It may be estimated.

  As another example, the condition that the time length of the longest metashot is within a specified range and the condition that the average time length of metashots is within the specified range may be used.

  Also, in sports relay programs such as sumo, baseball, and tennis, there is a tendency to broadcast a combination of videos taken from a plurality of fixed cameras. Therefore, similar shots appear throughout the program.

  Therefore, for example, if a similar shot appears in a time range that is more than half of the program and does not meet the conditions for estimating the news program described in the third embodiment, the program is a sports program. It may be estimated that there is.

  The method for determining the program type is not limited to the above method. An example of a method for determining the program type by obtaining the above-mentioned interaction frequency for the entire program will be described below.

FIG. 27 shows the number of dialogues of the entire program for 9 news programs and 12 variety programs. White squares (□) represent variety programs, and black squares represent news programs. The horizontal axis represents the interaction frequency. The vertical axis represents the ratio of the time at which the first corner title appears divided by the length of the entire program when the method of dividing the variety program of the second embodiment is applied to all programs (including news). . The dialogue frequency is logarithmic for ease of viewing.

  As can be seen from FIG. 27, the news program and the variety program have a clear tendency in the degree of dialogue of the entire program. Therefore, it is possible to discriminate between news programs (area A) and variety programs (areas B and C) with an appropriate degree of interaction as a threshold.

  In the variety program, the three programs in the area C have the corner title screen repeatedly appearing for each corner, but the tendency of appearing downward on the scale of the vertical axis can be observed. Therefore, by setting an appropriate threshold for the time at which a corner title appears, it is possible to discriminate what is appropriate for dividing a variety program into corners and what is not.

  As described above, the type of program may be determined based on the temporal distribution of similar shots, and the specific condition is not limited to the embodiment.

  FIG. 20 is a flowchart of the moving image processing in the moving image processing apparatus 10 according to the fourth embodiment. In the moving image processing according to the fourth embodiment, the conversation frequency calculation process (step S220) is performed immediately before the conversation frequency calculation process (step S220) described in the third embodiment. The processing of the dialogue frequency calculation unit 124 in the dialogue frequency calculation processing is as described above.

  The remaining configuration and processing of the moving image processing apparatus 10 according to the fourth embodiment are the same as those of the moving image processing apparatus 10 according to the third embodiment.

  Next, the moving image processing apparatus 10 according to the fifth embodiment will be described. The moving image processing apparatus 10 according to the fifth embodiment divides a moving image into a plurality of small sections and determines a program type for each small section. Here, the small section is a section that is simply divided by a certain time length such as every 3 minutes or every 30 seconds from the start of the moving image. As another example, a section divided by a certain number of shots such as three consecutive shots and ten shots may be used.

  FIG. 21 is a block diagram of a functional configuration of the moving image processing apparatus 10 according to the fifth embodiment. The moving image processing apparatus 10 according to the fifth embodiment further includes a small section definition unit 240 in addition to the functional configuration of the moving image processing apparatus 10 according to the third embodiment. The small section definition unit 240 divides the moving image into a plurality of small sections. Note that the small section definition unit 240 according to the embodiment constitutes a dividing unit according to the present invention.

  Also, prior to or when a moving image is input to the moving image acquisition unit 101, the analysis parameter receiving unit 190 includes each of the cut detection unit 102, the similar shot detection unit 104, and the program type determination unit 130 described above. Conditions (parameters) necessary for processing may be received and supplied to these detection units and determination units.

  As the destination to which the analysis parameter 190 receives parameters, the Internet, a recording medium, and the like are assumed as described in the first embodiment.

  FIG. 22 is a flowchart of the moving image processing in the moving image processing apparatus 10 according to the fifth embodiment. In the moving image processing according to the fifth embodiment, the small section definition process (step S240) is performed following the shot section definition process (step S204) described in the third embodiment. The processing of the small section definition unit 240 in the small section definition processing is as described above.

  Note that there is an analysis parameter reception step (not shown) as described above, and using the optimal parameter setting for each program received from the Internet or the like by the analysis parameter reception step before or during this processing, step S203, S207 and S230 may perform detection and determination processing.

  When the program type is detected for each small section as described above, for example, “14:00 to 15:00: Interview” or “7:11 to 9:49: Singing” Even in a single program, a type attribute can be automatically assigned to each corner even in a program in which a plurality of components (talk, interview, singing, news announcement, etc.) are combined.

  As a result, when the program itself has multiple corners, it is possible to automatically assign an attribute of each type (news, interview, talk, etc.) so that the user can get a clue. You can easily search the scene you want to watch.

  Other configurations and processes of the moving image processing apparatus 10 according to the fifth embodiment are the same as those of the moving image processing apparatus 10 according to the third embodiment.

  Next, the moving image processing apparatus 10 according to the sixth embodiment will be described. The moving image processing apparatus 10 according to the sixth embodiment determines a program type for each small section based on the interaction frequency. FIG. 23 is a block diagram of a functional configuration of the moving image processing apparatus 10 according to the sixth embodiment. FIG. 24 is a flowchart of the moving image processing in the moving image processing apparatus 10 according to the sixth embodiment. In the moving image processing according to the fifth embodiment, an interactive frequency calculation process (step S220) is performed immediately before the program type determination process (step S230) in the fifth embodiment. The processing of the dialogue frequency calculation unit 124 in the dialogue frequency calculation processing is as described above.

  The remaining configuration and processing of the moving image processing apparatus 10 according to the sixth embodiment are the same as the configuration and processing of the moving image processing apparatus 10 according to the fifth embodiment.

  Since the first shot of the meta shot is selected based on the appearance pattern in which the similar shot appears, there is an effect that an appropriate unit of the meta shot can be generated for a video such as a news program.

1 is a block diagram illustrating a functional configuration of a moving image processing apparatus 10 according to Embodiment 1. FIG. It is a figure which shows typically the statistical information which the statistical value holding | maintenance part 122 hold | maintains. It is a figure which shows a moving image typically. It is a figure which shows a news program typically. It is a figure which shows the content of the shot which belongs to each group. 3 is a flowchart illustrating moving image processing in the moving image processing apparatus 10. 2 is a diagram illustrating a hardware configuration of a moving image processing apparatus 10. FIG. FIG. 6 is a block diagram illustrating a functional configuration of a moving image processing apparatus 10 according to a second embodiment. It is a figure which shows typically the dialog scene of an anchor person and a correspondent. It is a figure which shows the graph of the interaction frequency calculated by the calculation method demonstrated referring FIG. It is a figure which shows typically the first shot and the interaction frequency with respect to the first shot. 12 is a flowchart illustrating moving image processing of the moving image processing apparatus 10 according to the second embodiment. FIG. 10 is a block diagram illustrating a functional configuration of a moving image processing apparatus 10 according to a third embodiment. It is a figure which shows typically the statistical information which the statistical value holding | maintenance part 122 concerning Example 3 hold | maintains. It is a figure which shows a drama program typically. It is a figure for demonstrating the process which defines a metashot. 12 is a flowchart illustrating moving image processing in the moving image processing apparatus 10 according to the third embodiment. FIG. 10 is a block diagram illustrating a functional configuration of a moving image processing apparatus 10 according to a fourth embodiment. It is a figure which shows the graph of the interaction frequency computed with respect to the actual quiz program. 10 is a flowchart illustrating moving image processing in the moving image processing apparatus 10 according to the fourth embodiment. FIG. 10 is a block diagram illustrating a functional configuration of a moving image processing apparatus 10 according to a fifth embodiment. 10 is a flowchart illustrating moving image processing in the moving image processing apparatus 10 according to the fifth embodiment. FIG. 10 is a block diagram illustrating a functional configuration of a moving image processing apparatus 10 according to a sixth embodiment. 14 is a flowchart illustrating moving image processing in the moving image processing apparatus 10 according to the sixth embodiment. It is a figure which shows typically the dialog section setting using a dialog frequency. It is a figure which shows typically the similar shot appearance pattern of a variety program It is a figure which shows typically the program classification determination using the interaction frequency and the corner title appearance time.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Moving image processing apparatus 101 Moving image acquisition part 102 Cut detection part 103 Shot area definition part 104 Similar shot detection part 105 Metashot head group determination part 106 Metashot head time determination part 107 Metashot generation part 108 Moving picture output part 110 Group Conversion unit 120 statistical processing unit 122 statistical value holding unit 124 interaction frequency calculation unit 130 program type determination unit 190 analysis parameter reception unit 240 small section definition unit

Claims (4)

Cut detection means for detecting an image change point at which the content of the image is switched from the moving image;
Similarity measuring means for measuring the similarity between partial moving images that are divided by the image change points detected by the cut detecting means;
Similar shot specifying means for specifying similar partial moving images based on the similarity measured by the similarity measuring means;
Grouping means for assigning the same group attribute to a plurality of similar partial moving images specified by the similar shot specifying means;
A moving image comprising: a moving image type determining unit that determines a type of the moving image based on an appearance pattern in which similar partial moving images grouped by the grouping unit appear in the moving image Image processing device. A program dialogue frequency calculating means for calculating the activity of dialogue with respect to the entire moving image,
The moving image processing apparatus according to claim 1, wherein the moving image type determining unit determines the moving image type based on an interaction frequency of the entire moving image calculated by the program interaction frequency calculating unit. Further comprising a dividing means for dividing the moving image into a plurality of time intervals;
The moving image processing apparatus according to claim 1, wherein the moving image type determining unit determines the moving image type for each section divided by the dividing unit. A leading group for selecting a leading group to which a leading partial moving image to be a leading portion of a metashot consisting of a plurality of temporally continuous partial moving images belongs from a group generated by the grouping means according to a predetermined condition Selection means,
First shot for selecting the first partial moving image from the partial moving images included in the first group based on the appearance pattern in which the partial moving image belonging to the first group selected by the first group selecting means appears in the moving image Selection means,
Metashot generation means for generating a metashot having the head partial moving image at the head;
Shot number comparison means for comparing the number of partial moving images belonging to the same group with a predetermined reference number,
Shortest time length comparison means for comparing the reproduction time length of a partial moving image having the shortest reproduction time length among the partial moving images belonging to the same group and a predetermined reference shortest time length;
Longest time length selection means for comparing the reproduction time length of a partial moving image having the longest reproduction time length among the partial moving images belonging to the same group with a predetermined reference longest time length;
An average time length comparison means for calculating an average value of reproduction time lengths of partial moving images belonging to the same group;
Among shots that belong to the same group, between shots that measure the playback time length between the first moving image arranged in the moving image and the last moving image arranged in the moving image. Time length comparison means,
Shot position determination means for determining whether or not all partial moving images belonging to the first group are arranged between two partial moving images belonging to a group other than the first group;
A reference range specifying means for specifying a reference range having a predetermined time length including a target partial moving image belonging to a predetermined group, or the group based on an appearance pattern of a plurality of partial moving images belonging to the same group Further comprising at least one means of interaction degree calculation means for calculating an interaction degree indicating a probability that the target partial moving image is a partial moving image indicating a conversation scene,
Based on the determination result of the moving image type determination unit, the similarity measurement unit, the grouping unit, the head group selection unit, the head shot selection unit, the meta shot generation unit, the shot number comparison unit, the shortest time At least one of a length comparison means, the longest time length selection means, the average time length comparison means, the inter-shot time length comparison means, the shot position determination means, the reference range identification means, or the interaction frequency calculation means 4. The moving image processing apparatus according to claim 1, wherein the operation reference value is changed.

Images